Thread by @phlegmfighter

This just out: Physicians do not logically/correctly estimate the probability of outcomes resulting from sequences of events - a thread. #conjunctionfallacy #probability #medicaldecisionmaking #clinicalreasoning #innumeracy #numeracy jamanetwork.com/journals/jaman… @JAMANetworkOpen

In this article, we showed that physicians estimated the probability of two events both occurring as *more* likely than one or both of the individual events. This is logically impossible and consistent with the #conjunctionfallacy. This #bias can lead to catastrophic outcomes

The impetus for the study was an #OBGYN catastrophe that resulted, in part, from this bias. A woman presented in labor w brow presentation. For successful vaginal delivery, 2 events must happen: reversion to deliverable position, and vaginal delivery w/o #caesareansection

If the probability of both events occurring is overestimated, as it was in the real life case, labor will be unnecessarily prolonged waiting for success that is unlikely to come. C-section will be necessary anyway, but later after delay & increased risk #thresholdapproach

We asked two cohorts of BC/BE #obgyn docs to estimate the probability of both events occurring and each individual event occurring. ~75% estimated probability of both events greater than one of individual events. This is logically impossible.

#pulmonologists fared worse, ~88% of them thought the probability of a positive biopsy of a lung nodule was higher than the probability of cancer or the #sensitivity of the biopsy, or both.

It's worse than that, though. A full 93% of respondents across all 3 sub-studies estimated the combined probability to be greater than the mathematically correct *product* of the two independent probabilities. That is a striking degree of #bias

We concluded that physicians either don't know the multiplication rule of probability or don't recognize situations in which it applies. This is very unfortunate because of how frequently physicians are confronted with decisions that hinge on sequences of probabilities

Consider a common scenario: a DNR/DNI elderly man presents with #hemoptysis vs #hematemasis ?#GIB #aspiration #respiratoryfailure and is on #HFNC. #EGD will be risky. Should it be performed? Answer hinges on benefits. What is the benefit of EGD in this patient?

For EGD to be beneficial, patient must have a sequence of probabilities: presence of GIB, EGD able to be completed, EGD able to see lesion (sensitivity - stomach not filled w blood), the lesion has to be "intervenable" (injection/clip etc) & intervention effective

If any one of those things is not true/successful, there will be no therapeutic value of the procedure (though there may be diagnostic value). Even if the probability of each is 70% (highly unlikely to be this high), the overall probability of benefit is 0.7^5 or 17%.

If one of the five conjunct probabilities is 10% the overall probability must be below 10% and is indeed just 2.4% (0.7^4*0.1). If we don't recognize the problem as one comprising a series of probabilities, we may subject the patient to unnecessary risk

In our cases, we did not even attempt to define the "normative" or correct individual probabilities. I made the #pulmonarynodule case using a calculator so #pretestprobability was 80%; sensitivity of transthoracicneedlebiopsy is ~90%. So the "normative answer is ~72% (blue dots)

Only ~ 20% of our respondents gave a combined probability (cancer + positive biopsy) on either side of 72% - the vast majority overestimated it. Such overestimations could cause the relative value of this approach to be misestimated as greater than other options, e.g., surgery

We thought that reversing the order of the questions & asking respondents to estimate the component probabilities first would "debias" the obstetricians. Despite research in problem solving (and my strong opinion) that problem decomposition improves performance, it did not here

As in other studies, the ranges of probabilities given by respondents spanned almost the entire probability spectrum - anywhere from 0% to 98%. Physicians lack calibration. It is no wonder patients complain about different doctors telling them wildly different things

Even where there are strong data to guide probabilistic estimations, as in the nodule case. In the brow presentation case, #UTDOL says probability of brow->deliverable position is 50-70%; the base rate of C-section in US is ~35%. Where threshold for c-section is crossed is murky

In conclusion, many medical decisions involve probability or sequences of probabilities. The tendency of physicians to overestimate the probability of successful outcomes that require multiple steps is worrisome, and is liable to lead suboptimal decisions

Wow. Just Wow: b/c respondents "often provided round numbers (eg, 30% or 40%), calculating the overall estimate should have been easy; However, only 1 physician among 215 correctly estimated the conjunction probability to be exactly equal to the product of the 2 components."

😢 “ the impetus for this research project was a real #OBGYN case…involving patient counseling related to probabilities that was misguided in a way consistent with the conjunction fallacy, resulting in the loss of a child due to injuries sustained during prolonged labor.”