,
28 tweets,
9 min read
Read on Twitter
Thread on the reporting of hierarchical outcomes inspired by 1 of the only 2 #Impella RCTs.
A few months ago, @willsuh76 noted that rates of bleeding were not reported in PROTECT-II, a worry with any large bore percutaneous device.
I am familiar with hierarchical significance *testing* of secondary outcomes. As an example, #Partner3 reports secondary outcomes with a hierarchical order to address issues of multiple testing. A Cox model is used for 1st, 3rd, 5th, and 6th listed endpoints. 
For the above, each patient may contribute one endpoint for the specific one being reported.
However, PROTECT II used a variation of hierarchical secondary outcomes and I am not sure how commonly this is used. Readers should be mindful of what this type of reporting represents.
PROTECT-II randomized 448 patients to IABP or Impella for scheduled (non-emergent) high-risk PCI with an LVEF <35%. The study was terminated for futility and the outcomes were reported in 2012. @CircAHA ahajournals.org/doi/10.1161/CI…
The primary endpoint was based on a 30-day observation period (not time-to-event) using a Pearson’s chis^2 test (difference in proportions). 90-day secondary endpoints were also reported.
This is the Table 4 (primary and secondary outcomes) from the @CircAHA manuscript:
Here is the FDA SSED document of the same outcomes (non-hierarchical)
Notice that the event rates are much higher for those lower on the list of endpoints when not hierarchical.
What does hierarchical reporting mean here? It signifies each patient may contribute only ONE event during the period of observation. The event reported is the worst of the hierarchy (order listed).
If Patient X had acute renal dysfunction but then died, only the death is counted as an event. If a Patient Y died, then they did not report other adverse events on the table in the @CircAHA manuscript.
This isn’t necessarily problematic, but it might mislead someone who thinks the information is useful for assessing safety rather than efficacy.
Event rates are lower for secondary endpoints when using hierarchical reporting in this fashion and it’s probably why the FDA didn't emphasize it. Non-hierarchical reporting is important for understanding safety and communicating risks to clinicians/patients.
You cannot tell a patient the risk of repeat revascularization is low with #ProtectedPCI using #Impella when not telling them about events that occurred to the patients that died, or experienced a stroke or MI.
It also is inappropriate to report p-values (tests of proportional differences) between secondary endpoints that depend on each other in Table 4. If anything, cumulative rates of adverse events should be reported as one goes down the hierarchical list of endpoints.
For statistical testing, we frequently require hierarchies when testing differences between treatment strategies.
In a time-to-event analysis, we report the first event but not all events. It would be inappropriate to tally repeated events within single patients unless we use a repeated measures model accounting for clustering within participants.
Now here’s the real kicker that shocked me when reviewing the #Impella data. The FDA patient brochure @abiomedimpella created selectively picks ALL adverse events for IABP but HIERARCHICAL outcomes for Impella, making it seem as though the company’s device is superior.
FYI: CPR rates are higher with Impella in PROTECT II
FDA document on Impella 2.5 approval SSED here:
accessdata.fda.gov/cdrh_docs/pdf1…
accessdata.fda.gov/cdrh_docs/pdf1…
Anyway, thought I'd share my observations after I needed to double-check what I was seeing with @willsuh76 @ADAlthousePhD @drjohnm Thanks for your critical evaluations! Much better than a letter to the editor.
Also, this @ProfDFrancis paper has gotten me in the habit of looking for inconsistencies between tables/prior publications when reviewing manuscripts. bmj.com/content/348/bm…
*Correction: table 3 is ITT and 4 was PP:
Helpful slide from @willsuh76
Unroll @threadreaderapp
