Thread by @Ricky_Turgeon

How to Critically Appraise a Trial in 10 Minutes: The NERDCAT Approach: The Tweetorial 1/

2/ The resources:
Go to nerdcat.org for the latest version of all the NERD critical appraisal tools, including NERDCAT-RCT: static1.squarespace.com/static/56a0651…
@medmyths literally wrote an iBook on this, available at: therapeuticseducation.org/sites/therapeu…

@medmyths 3/ We'll use the recent PolyIran trial (ncbi.nlm.nih.gov/pubmed/31448738) to go over the approach & some key concepts

@medmyths 4/ The 1st step before appraising a trial is deciding if it's worth doing. Do you give a HOOT?
Hot topic in your area?
Outrageous conclusions?
Outcome(s) that matter?
Title piques your interest?
If ≥2 HOOTs based on the title+abstract, it's probably worth your time

@medmyths 5/ For PolyIran from my perspective as a cardiology pharmacist, this was a 4/4 HOOTs given. The polypill has the potential to simplify CV risk reduction, & this trial that actively evaluated clinical endpoints rather than surrogate outcomes.

@medmyths 6/ Step 2 is summarizing the study question so that we know exactly what could be answered once we're done appraising. If the trial is properly reported, you should be able to easily do this with the abstract, as done here for PolyIran

@medmyths 7/ Step 3: Mentally trash most of the article (or if you still print, recycling bin), including: The intro (get your background elsewhere), most of the Methods/Stats section (save for >10 min deeper dive if needed), Discussion & Conclusion (someone else's interpretation)

@medmyths 8/ Step 3 turns a 12-page slog into a more manageable ~5 pages. But thankfully, you still won't need to read the rest of the pages start-to-finish. Instead, we'll actively extract what we need to get our answers

@medmyths 9/ Step 4 is the actual "Critical Appraisal" bit, which is all about answering 3 fundamental questions:
(1) Who does this (not) apply to? (generalizability)
(2) Are the results reliable? (internal validity)
(3) What are the results? (absolute magnitude of benefits & harms)

@medmyths 10/ Fundamental Q #1: Who does this (not) apply to?
IMHO, this is the most nuanced part of appraisal. It's the part where you identify if the trial patients are so DISSIMILAR from your patient population (or the patient in front of you) that you cannot apply its result to them.

@medmyths 11/ To help assess generalizability, NERDCAT-RCT has a list of factors to consider. The most important are disease definition & severity. Also assess whether study intervention in your practice (e.g. same/similar drug regimen), including the requisite monitoring & follow-up

@medmyths 12/ We can get most of the information to assess for generalizability by looking at the Inclusion/Exclusion criteria to see who COULD be enrolled, & the baseline characteristics table (classically "Table 1") to see who WAS enrolled.

@medmyths 13/ I like to sum up Table 1 to get a sense of the "typical person" in this RCT. Key things to include are same data points you'd want to know on the ward/in the clinic. In PolyIran, although conducted in resource limited setting, fairly typical older adult 1o prevention patient

@medmyths 14/ Fundamental Q #2 is the search for bias. Download the article PDF & search (PC: control+F, MAC: command+F) for 5 key words: Random; Allocation; Blind (sometimes need to replace with "mask"); Intention; Lost. These should quickly lead you to the relevant section in the text

@medmyths 15/ The relevant text extracts from PolyIran included here.

@medmyths 16/ PSA that allocation concealment & blinding are not the same. Allocation concealment is ALWAYS possible in any RCT & should ALWAYS be done.

If allocation concealment inadequate, blinding is compromised.

@medmyths 17/ PolyIran offers a unique opportunity to see the impact of allocation concealment. Not done in 1st 20% randomized, which was fixed for final 80%. Wide confidence intervals, but point estimates suggest lack of allocation concealment led to exaggeration in estimate of effect

@medmyths 18/ Overall, only real risk of bias (RoB) in PolyIran was non-blinded patients, with possible performance bias. But RoB ≠ presence of bias. I argue most plausible scenario is that control group may be more motivated to follow lifestyle advice to make up for not taking polypill

@medmyths 19/ In this scenario, this would be a "conservative bias", i.e. more difficult for polypill to come out on top (if control group has better adherence to lifestyle advice). So if polypill still found to be beneficial despite this, this strengthens confidence in the trial findings

@medmyths 20/ Final fundamental Q: What are the results. Useful to summarize in 1 table: Benefits & harms, absolute & relative effects. Harms often more difficult to find than benefits, may need to dig through appendices, contact authors, etc.

@medmyths 21/ You'll note I didn't include number needed to treat (NNT). I'm not a fan of this number anymore. Several compelling arguments by @f2harrell. Bottom line, too difficult to communicate benefits:harms with patients. Easier to use absolute event rates + relative risk reduction

@medmyths @f2harrell 22/ e.g. I would summarize PolyIran as follows: In a trial at low risk of bias of ~6800 adults mostly without CVD, use of a polypill containing ASA, statin, ACEI & HCTZ reduced the risk of CV events by 1/3 from ~9% to ~6% over 5 years.

@medmyths @f2harrell 23/23 And that's how I appraise a trial in 10 min. More details on nerdcat.org. Curious to know how others approach this differently

@medmyths @f2harrell 24/23 Final thought: Some people approach appraisal with the goal of trashing a paper without taking anything away from it (find all the flaws to justify staying the same). I used PolyIran as eg here to show how even RCTs with flaws can answer clinical questions & impact practice