Thread by @dr_dmorgan

How good are doctors at diagnosis?

This is the most relevant paper I have written. Not perfect but addresses a huge issue I think could change medicine if acknowledged
…it has changed how I think about diagnosis

ja.ma/3rQNtjv
@JAMAInternalMed
@drjohnm
1/n

summary:
Clinicians widely overestimated chance of disease especially after testing

Cardiac ischemia after + ECG—EBM 2-11%, median answer 70%
UTI after + urine cx—EBM 0-8.3%, answer 80%
Breast CA after + mammo—EBM 3-9%, answer 50%
Pneumonia after + CXR EBM 46-65%, answer 95%

Gerd Gigerenzer, David Eddy, @StevenWoloshin @arjunmanrai & others asked how well doctors do at the math of understanding diagnosis, and found they aren’t great.

Many of issues w/ real life tests covered by @deeksj @d_spiegel @dan_diekema

We asked questions of diagnosis sense making and test interpretation for boringly common scenarios that often require some degree of patient shared decision making (and therefore require explanation of risk by doctors)

Thanks #NIHHighRisk @NIHDirector for DP2 funding

We assembled a big team that asked many hard questions up front, we finally used survey v34

Great collaborators (those on twitter)
@DKorenstein @AndrewFoy82 @masnick @SWeisenberg @ldscherer @LeykumLuci

We ask for interpretation of scenarios of pneumonia, UTI, cardiac ischemia and breast cancer screening.

All involved varying risk and questioned the use of first line testing.

No tricks, bread and butter cases

The challenge was estimating gestalt probability

We enrolled 553 doctors & NPs/PAs across 8 states w/ 76% response rate
(response high mostly d/t in person enrollment w/ in person follow up—frequently from me or collaborators—this was a LOT of work as survey not easy to complete)

We asked about a common scenario and the chance of a disease before testing and after a positive or negative result.
Patients have nuance, so here is description

We searched lit. for best evidence-based medicine (EBM) answer to compare
The graphs show distribution of answers

Cardiac ischemia
Pretest chance—EBM 1-4.4%, median answer 10%
Post stress + ECG—EBM 2-11%, median answer 70%
Post stress - ECG—EBM 0.4-2.5%, median answer 5%

Urinary Tract Infection (UTI)—case of asymptomatic bacteriuria
Pretest chance—EBM 0-1%, median answer 20%
Post + urine cx—EBM 0-8.3%, median answer 80%
Post – urine cx—EBM 0-0.1%, median answer 5%
After + urine 71.1% would give Abx. After – urine 7.8% would give antibiotics!

Pneumonia
Pretest chance—EBM 25-42%, median answer 80%
Post + chest x-ray EBM 46-65%, median answer 95%
Post – chest x-ray EBM 10-19%, median answer 50%---After negative CXR 27.5% would NOT treat with antibiotics

Breast cancer screening
Pretest chance—EBM 0.2-0.3%, median answer 5%
Post + mammo—EBM 3-9%, median answer 50%
Post – mammo—EBM <0.05%, median answer 5% (no change from pretest!)

Just like @arjun Medicine’s Uncomfortable Relationship With Math ja.ma/39TRPA8
and Casscells (NEJM 1978) when asked

The most common answer was 95%,
BUT 2% was correct

And, these simple decisions each of us makes many times a day, across all clinicians = hundreds of thousands of decisions.
If we’re all biased towards big overestimates, this could explain a big portion of the ~1/3 of medical care that is #overuse #lowvalue care
@lowninstitute

It is not that doctors are bad—but diagnosis and medicine are difficult.

The answer isn’t better math but a gestalt that is generally right

We need to use the clinical evidence we have to make diagnosis more scientific and not a simple “art” done differently by each doctor

Diagnosis is often an afterthought—list icd-10 codes with the right words for $$

If a code is present, make sure patient gets the medication or documentation for performance metrics

@fda & @cms have minimal test oversight @drJoshS
ja.ma/2ReZOBf

The billing/metrics false-simplification of clinical medicine worsens patient care and is a barrier to nuance and #SDM @RichardLehman1

I think the answer lies with clinicians being encouraged to doctor @adamcifu , @Bob_Wachter

The first step is acknowledging we have a problem w/ #diagnosis
leading to a #diagnosticerror of COMission
not described much in @theNAMedicine report nationalacademies.org/our-work/diagn…

@HardeepSinghMD

Solutions may not be that difficult if we:
-teach probability & clinical research > pathophysiology
-guide pretest probability—history/prevalence/exam
-teach test interpretation w/ natural frequencies NOT 2x2 table

ja.ma/3tpejRr
@VPrasadMDMPH
@AdamRodmanMD

cont..
-have reference clinical sensitivity & specificity testingwisely.com
-implement tests w/ dx stewardship ja.ma/2VGYs2F
-understand and accept #uncertainty (@greyscalespaces)