Thread by @djc795

@rahatheart1 @rafasadaba @ovidiogarciav @Almanfi_Cardio @Coronary4front @FaisalBakaeen @mirvatalasnag @SukhNijjer I enjoy a good observational study as much as anyone, but this is not one of those. Even without reading the full paper, I can see 2 obvious signs that this study is hopelessly confounded based on the original tweet. Challenge to the #CardioTwitter community to find the 2 clues.

@rahatheart1 @rafasadaba @ovidiogarciav @Almanfi_Cardio @Coronary4front @FaisalBakaeen @mirvatalasnag @SukhNijjer 1/x- OK, for those of you have been waiting patiently for my thoughts as to why this study is so obviously confounded, here's the long tweetorial.

@rahatheart1 @rafasadaba @ovidiogarciav @Almanfi_Cardio @Coronary4front @FaisalBakaeen @mirvatalasnag @SukhNijjer 2/x- As you will see, I focused on some very obvious clues that did not require deep insight into statistics. In fact, the issues that concerned me were ones that are obvious to a thoughtful clinician and might actually be missed by a methodologist.

@rahatheart1 @rafasadaba @ovidiogarciav @Almanfi_Cardio @Coronary4front @FaisalBakaeen @mirvatalasnag @SukhNijjer 3/x- First, 30-day mortality in this “low-risk” population was 3.3% with TAVR and 3.0% with SAVR (adjusted OR 1.1). Mortality of 3% is a little high for a true low-risk SAVR group, but mortality of 3.3% is insane for low-risk TAVR.

@rahatheart1 @rafasadaba @ovidiogarciav @Almanfi_Cardio @Coronary4front @FaisalBakaeen @mirvatalasnag @SukhNijjer 4/x- For reference, 30-day mortality in the PARTNER 3 and EVOLUT low-risk trials with TAVR was 0.4% and 0.5%, respectively and the odds ratio compared with SAVR was ~0.4. In the TVT registry in 2021, the 30-day mortality rate for all-comers was 1.0%.

@rahatheart1 @rafasadaba @ovidiogarciav @Almanfi_Cardio @Coronary4front @FaisalBakaeen @mirvatalasnag @SukhNijjer 5/x- These findings indicate clearly that despite propensity matching, the TAVR “low risk” patients were significantly sicker than the SAVR “low risk” patients. There is no other credible explanation for these 30-day mortality results.

@rahatheart1 @rafasadaba @ovidiogarciav @Almanfi_Cardio @Coronary4front @FaisalBakaeen @mirvatalasnag @SukhNijjer 6/x- Second, the divergence of the survival curves starts almost abruptly after 2 years—prior to that, there is very little difference in survival. What could explain this finding?

@rahatheart1 @rafasadaba @ovidiogarciav @Almanfi_Cardio @Coronary4front @FaisalBakaeen @mirvatalasnag @SukhNijjer 7/x- One possibility would be a high rate of valve degeneration starting around 2 years in the TAVR group. However, this would be inconsistent with data from numerous high-quality echo studies showing very low rates of valve degeneration through at least 5 years of follow-up.

@rahatheart1 @rafasadaba @ovidiogarciav @Almanfi_Cardio @Coronary4front @FaisalBakaeen @mirvatalasnag @SukhNijjer 8/x- So, what else could do this? The answer is if the patients who had >2 year follow-up were much sicker in the TAVR group compared with the surgical group. But how could this be the case if the investigators performed propensity matching?

@rahatheart1 @rafasadaba @ovidiogarciav @Almanfi_Cardio @Coronary4front @FaisalBakaeen @mirvatalasnag @SukhNijjer 9/x- Anyone who has been part of a structural heart team for the past 5-7 years knows the answer to this. If a low-risk patient underwent TAVR in 2015-7, they were “low risk” on paper, but not in the clinic.

@rahatheart1 @rafasadaba @ovidiogarciav @Almanfi_Cardio @Coronary4front @FaisalBakaeen @mirvatalasnag @SukhNijjer 10/x- Very likely they had a major comorbidity that was not well-captured in the study database (e.g., liver disease, cancer) or were very frail leading to the decision to offer TAVR.

@rahatheart1 @rafasadaba @ovidiogarciav @Almanfi_Cardio @Coronary4front @FaisalBakaeen @mirvatalasnag @SukhNijjer 11/x- Thus, the “low-risk” patients enrolled earlier in the study (who were the only ones eligible for 3-5 year follow-up, were almost certainly much sicker than the low-risk surgery patients. This is almost certainly why the mortality curves only diverge during the later years.

@rahatheart1 @rafasadaba @ovidiogarciav @Almanfi_Cardio @Coronary4front @FaisalBakaeen @mirvatalasnag @SukhNijjer 12/x- From my perspective, the lesson here isn’t that observational studies are all worthless. But that when we are trying to study treatments where there is a lot of selection bias up front...

@rahatheart1 @rafasadaba @ovidiogarciav @Almanfi_Cardio @Coronary4front @FaisalBakaeen @mirvatalasnag @SukhNijjer 13/x- ... you need to think like an enlightened clinician about how patients end up getting selected for one treatment or another and make sure that all these factors are well-accounted for in the propensity score (or other risk-adjustment technique).

@rahatheart1 @rafasadaba @ovidiogarciav @Almanfi_Cardio @Coronary4front @FaisalBakaeen @mirvatalasnag @SukhNijjer 14/x- And if we understand these clinical concepts, we can often find clues that indicate failure to fully adjust for confounding that render a study invalid before it even gets off the ground.

@rahatheart1 @rafasadaba @ovidiogarciav @Almanfi_Cardio @Coronary4front @FaisalBakaeen @mirvatalasnag @SukhNijjer 15/- This is different from "possible unmeasured confouding" that affects all observational studies. This is clearcut evidence of meaningful unmeasured confounding that invalidates the study's main findings.

@rahatheart1 @rafasadaba @ovidiogarciav @Almanfi_Cardio @Coronary4front @FaisalBakaeen @mirvatalasnag @SukhNijjer 16/x-- The reviewers and journal editors should have identified this and not published a study that is so obviously confounded. IMHO, the authors should have done the same. (END)