Thread on the reporting of hierarchical outcomes inspired by 1 of the only 2 #Impella RCTs.
A few months ago, @willsuh76 noted that rates of bleeding were not reported in PROTECT-II, a worry with any large bore percutaneous device.
This led me to look at the @FDA evaluation to see if they were reported there. I found it interesting that the FDA tables of outcomes/adverse events included all events, while the @CircAHA paper uses hierarchical for secondary outcomes.
I am familiar with hierarchical significance *testing* of secondary outcomes. As an example, #Partner3 reports secondary outcomes with a hierarchical order to address issues of multiple testing. A Cox model is used for 1st, 3rd, 5th, and 6th listed endpoints.
For the above, each patient may contribute one endpoint for the specific one being reported.
However, PROTECT II used a variation of hierarchical secondary outcomes and I am not sure how commonly this is used. Readers should be mindful of what this type of reporting represents.
PROTECT-II randomized 448 patients to IABP or Impella for scheduled (non-emergent) high-risk PCI with an LVEF <35%. The study was terminated for futility and the outcomes were reported in 2012. @CircAHA ahajournals.org/doi/10.1161/CI…
The primary endpoint was based on a 30-day observation period (not time-to-event) using a Pearson’s chis^2 test (difference in proportions). 90-day secondary endpoints were also reported.
This is the Table 4 (primary and secondary outcomes) from the @CircAHA manuscript:
Here is the FDA SSED document of the same outcomes (non-hierarchical)
Notice that the event rates are much higher for those lower on the list of endpoints when not hierarchical.
What does hierarchical reporting mean here? It signifies each patient may contribute only ONE event during the period of observation. The event reported is the worst of the hierarchy (order listed).
If Patient X had acute renal dysfunction but then died, only the death is counted as an event. If a Patient Y died, then they did not report other adverse events on the table in the @CircAHA manuscript.
This isn’t necessarily problematic, but it might mislead someone who thinks the information is useful for assessing safety rather than efficacy.
Event rates are lower for secondary endpoints when using hierarchical reporting in this fashion and it’s probably why the FDA didn't emphasize it. Non-hierarchical reporting is important for understanding safety and communicating risks to clinicians/patients.
You cannot tell a patient the risk of repeat revascularization is low with #ProtectedPCI using #Impella when not telling them about events that occurred to the patients that died, or experienced a stroke or MI.
It also is inappropriate to report p-values (tests of proportional differences) between secondary endpoints that depend on each other in Table 4. If anything, cumulative rates of adverse events should be reported as one goes down the hierarchical list of endpoints.
For statistical testing, we frequently require hierarchies when testing differences between treatment strategies.
In a time-to-event analysis, we report the first event but not all events. It would be inappropriate to tally repeated events within single patients unless we use a repeated measures model accounting for clustering within participants.
Now here’s the real kicker that shocked me when reviewing the #Impella data. The FDA patient brochure @abiomedimpella created selectively picks ALL adverse events for IABP but HIERARCHICAL outcomes for Impella, making it seem as though the company’s device is superior.
A complete misrepresentation or lying about the statistics.

accessdata.fda.gov/cdrh_docs/pdf1…
FYI: CPR rates are higher with Impella in PROTECT II
FDA document on Impella 2.5 approval SSED here:
accessdata.fda.gov/cdrh_docs/pdf1…
Anyway, thought I'd share my observations after I needed to double-check what I was seeing with @willsuh76 @ADAlthousePhD @drjohnm Thanks for your critical evaluations! Much better than a letter to the editor.
Also, this @ProfDFrancis paper has gotten me in the habit of looking for inconsistencies between tables/prior publications when reviewing manuscripts. bmj.com/content/348/bm…
*Correction: table 3 is ITT and 4 was PP:
Helpful slide from @willsuh76
Missing some Tweet in this thread?
You can try to force a refresh.

Like this thread? Get email updates or save it to PDF!

Subscribe to Boback Ziaeian 🤦🏻‍♂️
Profile picture

Get real-time email alerts when new unrolls are available from this author!

This content may be removed anytime!

Twitter may remove this content at anytime, convert it as a PDF, save and print for later use!

Try unrolling a thread yourself!

how to unroll video

1) Follow Thread Reader App on Twitter so you can easily mention us!

2) Go to a Twitter thread (series of Tweets by the same owner) and mention us with a keyword "unroll" @threadreaderapp unroll

You can practice here first or read more on our help page!

Follow Us on Twitter!

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just three indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3.00/month or $30.00/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!