I promised a tweetorial about this polygenic risk prediction paper but explaining methods and statistics in tweets is easier said than done.
Instead, let me just share how I read a paper like this and what question come up and why.
Instead, let me just share how I read a paper like this and what question come up and why.
Admitted, my prior for being skeptical about this paper was that I simply could not *believe* that 6.6M SNPs matter for CAD risk. This made me check how the SNPs are selected and what their effects (weights) are.
When I found that most of the weights are small, meaning that most SNPs are not relevant for predicting CAD risk, I wondered why LDpred doesn't kick them out. LDpred apparently doesn't assign weights that are really zero (although, the current weights are zero enough to me).
I'll asked on Twitter what other people thought about this. Responses were mixed. From a statistical perspective, I understand that there is no need to leave SNPs out, but I am concerned that a score that is mostly zero-effect SNPs may not be received well by the public.
I was not convinced that 6.6M SNPs are needed, but also not that 6.6K will do better than statistically significant 74 SNPs only. 6.6M SNPs only added 0.015 to the AUC, which is not much and does not imply that the score also works better for identifying high-risk individuals
AUC changes when ROC curves change, but when the score is to be used for selecting high-risk groups, the curves need to change in the lower left corner of the ROCplot. Does the 6.6M score perform better than the 74 SNP score at a specific threshold of interest?
I knew already that the AUC of the CAD 6.6M SNP score was markedly higher than in the preprint version of the paper, from 0.64 to 0.81 an increase in AUC of 0.17(!). I now found that this might be explained by the fact that the current analyses are also adjusted for age and sex.
Earlier prediction studies in the UK Biobank show similar strong effects of age. This is not surprising when you know that the age range of the UKB is wide, from 37-73 years. That is why external validation should always be part of the initial publication, like in GWAS.
Next, I struggled to align the results in Fig. 2c and Table 2, and trying to understand how the table was made "progressively more extreme tails of the GPS distribution were compared." The discrepancies and the 'moving risk' in the comparison group cause much confusion.
And finally, not hindered by any knowledge of cardiology, I wondered how likely it is that so many people would develop CAD without any risk factor. I found that these results cannot rule out that the high-risk CAD patients caused the slight increases in RF prevalences.
So, I am not convinced that polygenic risk scores are ready for inclusion in clinical care. They may be ready, just the relevant data to show their value are not in this paper. Until then, I remain skeptical.
I prepared these slides for journal club. Send me a DM if you want them for your class or journal club. And feel free to change anything you disagree.
And if you want to know more about the basics of prediction research, here is my course manual. cecilejanssens.org/wp-content/upl…
