Thread by @RecheckHealth

Care to kick off the New Year with a TWEETORIAL? Our humble homage to @VPrasadMDMPH will walk you through the evidence on #HPV vaccines and what @Cochranecollab made of it. Let’s first agree on one thing: RCTs are superior to observational studies

On HPV vaccines many observational studies published and hyped Randomized Controlled Trials on #Gardasil #Cervarix failed to demonstrate the expected efficacy though. It is sad, true and rarely discussed. Let’s have a closer look. @garyschwitzer @HealthNewsRevu @senseaboutsci

In 2006 @US_FDA VRBPAC Committee required @Merck to provide data and analyses on Gardasil RCTs FUTURE I and II.
Which outcome would you consider as the most relevant to assess Gardasil efficacy?

CIN 2+ irrespective of HPV type is indeed the outcome that really matters. Let’s now have a look at the intention to treat #ITT analysis results. They were published by @NEJM in 2007 tinyurl.com/y9yr82pp
Measured efficacy in Gardasil RCTs CIN2+ all HPV types ITT?

But wait, they say! #ITT results are bad because RCTs included women who were sexually active and therefore already infected with HPV. If you take young girls who are not infected, the results will be much better.

Okay, so let’s have a look at the results for the per protocol #PP population (naive to vaccine high-risk HPV types at vaccination onset). But, there is a problem. PP results have not been published:
tinyurl.com/y9yr82pp
tinyurl.com/y9wr8c27

Luckily @US_FDA VRBPAC Committee obtained them from @Merck Since years available thanks to a Freedom of Information Act request #FOIA by @JudicialWatch - weirdly these data are never part of the conversation
zenodo.org/record/1434214

Here you go PP results @Merck provided to @US_FDA VRBPAC Gardasil Studies 007 013 015 Analysis of efficacy CIN 2/3+ any HPV type among subgroup of subjects meeting per protocol population for all 4 vaccine-relevant HPV types
Efficacy in Gardasil RCTs CIN2+ all HPV types?

Yep. These are troubling results - even worse than ITT analysis’ because they are statistically not significant. It means you basically cannot measure a difference between vaccinated and control group

Gardasil efficacy different populations provided by @Merck to @US_FDA VRBPAC Efficacy endpoint CIN2+ at best low and unproven in almost all populations zenodo.org/record/1435032 zenodo.org/record/1434214
Do these results suggest Gardasil failed to show the expected global efficacy?

A better subgroup? Here you go: DAP Statistical Data Analysis Plan we obtained from @US_FDA with a #FOIA It shows that analysis of negative to 14 HPV types population (RMITT-2) was added during trials with a new DAP zenodo.org/record/1434216
zenodo.org/record/1434220

So here we have our better subgroup! But it took the place of the pre-specified per protocol #PP analysis. Is this perhaps what #EBM calls outcome switching?
ttps://zenodo.org/record/1434216
zenodo.org/record/1434220

Do you think that in this case outcome switching is fine because we want to vaccine girls before they start to be sexually active?

According to #EBM the right answer is a clear NO. Outcome switching means that you changed hypothesis during the trial because the results were not as expected. It’s bad because it distorts evidence, making clinical trials meaningless thus corrupting medical research @DES_Journal

Some would object that this subgroup IS much better, because it provides ”real world” estimate of the impact of the vaccine. Is this what #EBM calls post-hoc subgroup analysis though?
zenodo.org/record/1434216
zenodo.org/record/1434220

Do you think that in this case to consider the result of a post-hoc subgroup analysis as a solid evidence is fine because we want to vaccine young girls before they start to be sexually active?

According to #EBM, the right answer is a clear NO. The result of a post-hoc subgroup analysis has a strictly exploratory and non-conclusive value; it cannot be interpreted as evidence. @DrSidMukherjee explains very well the issue

Post-hoc subgroup analyses and outcome switching affect Gardasil RCTs Concerning according to #EBM @CebmOxford @stud4BE @bengoldacre @COMPare_Trials Worse: trials publications don’t even mention it
zenodo.org/record/1434216
zenodo.org/record/1434220
Can these flaws impact evidence?

Post-hoc subgroup analysis affect also #Cervarix #RCT PATRICIA There is no mention of the so called TVC-naïve population in @GSK PATRICIA original protocol. When published, data were not correctly presented as post-hoc analyses results

For #Cervarix @GSK investigators published the results for CIN2+ irrespective of HPV type in per protocol analysis, but not for the CIN3+ tinyurl.com/y76nej8k
#SelectiveReporting @TheLancet

All these methodological flaws not acknowledged in @Cochranecollab HPV Vaccines review even though authors group was made aware of them #EBM #HPV @BMJ_EBM ebm.bmj.com/content/early/…
Is this a problem?

and if this is all too much for you, please read (French) the essential article of @boylandry for @ParisMatch it explains it all and is a jargon-free version of this story #HPV #EBM #investigative #journalism ebm.bmj.com/content/early/…
parismatch.com/Actu/Sante/Rie…