Dan Morgan Profile picture
12 Apr, 20 tweets, 13 min read
How good are doctors at diagnosis?

This is the most relevant paper I have written. Not perfect but addresses a huge issue I think could change medicine if acknowledged
…it has changed how I think about diagnosis

ja.ma/3rQNtjv
@JAMAInternalMed
@drjohnm
1/n
summary:
Clinicians widely overestimated chance of disease especially after testing

Cardiac ischemia after + ECG—EBM 2-11%, median answer 70%
UTI after + urine cx—EBM 0-8.3%, answer 80%
Breast CA after + mammo—EBM 3-9%, answer 50%
Pneumonia after + CXR EBM 46-65%, answer 95%
Gerd Gigerenzer, David Eddy, @StevenWoloshin @arjunmanrai & others asked how well doctors do at the math of understanding diagnosis, and found they aren’t great.

Many of issues w/ real life tests covered by @deeksj @d_spiegel @dan_diekema
We asked questions of diagnosis sense making and test interpretation for boringly common scenarios that often require some degree of patient shared decision making (and therefore require explanation of risk by doctors)

Thanks #NIHHighRisk @NIHDirector for DP2 funding
We assembled a big team that asked many hard questions up front, we finally used survey v34

Great collaborators (those on twitter)
@DKorenstein @AndrewFoy82 @masnick @SWeisenberg @ldscherer @LeykumLuci
We ask for interpretation of scenarios of pneumonia, UTI, cardiac ischemia and breast cancer screening.

All involved varying risk and questioned the use of first line testing.

No tricks, bread and butter cases

The challenge was estimating gestalt probability
We enrolled 553 doctors & NPs/PAs across 8 states w/ 76% response rate
(response high mostly d/t in person enrollment w/ in person follow up—frequently from me or collaborators—this was a LOT of work as survey not easy to complete)
We asked about a common scenario and the chance of a disease before testing and after a positive or negative result.
Patients have nuance, so here is description

We searched lit. for best evidence-based medicine (EBM) answer to compare
The graphs show distribution of answers
Cardiac ischemia
Pretest chance—EBM 1-4.4%, median answer 10%
Post stress + ECG—EBM 2-11%, median answer 70%
Post stress - ECG—EBM 0.4-2.5%, median answer 5%
Urinary Tract Infection (UTI)—case of asymptomatic bacteriuria
Pretest chance—EBM 0-1%, median answer 20%
Post + urine cx—EBM 0-8.3%, median answer 80%
Post – urine cx—EBM 0-0.1%, median answer 5%
After + urine 71.1% would give Abx. After – urine 7.8% would give antibiotics!
Pneumonia
Pretest chance—EBM 25-42%, median answer 80%
Post + chest x-ray EBM 46-65%, median answer 95%
Post – chest x-ray EBM 10-19%, median answer 50%---After negative CXR 27.5% would NOT treat with antibiotics
Breast cancer screening
Pretest chance—EBM 0.2-0.3%, median answer 5%
Post + mammo—EBM 3-9%, median answer 50%
Post – mammo—EBM <0.05%, median answer 5% (no change from pretest!)
Just like @arjun Medicine’s Uncomfortable Relationship With Math ja.ma/39TRPA8
and Casscells (NEJM 1978) when asked

The most common answer was 95%,
BUT 2% was correct
And, these simple decisions each of us makes many times a day, across all clinicians = hundreds of thousands of decisions.
If we’re all biased towards big overestimates, this could explain a big portion of the ~1/3 of medical care that is #overuse #lowvalue care
@lowninstitute
It is not that doctors are bad—but diagnosis and medicine are difficult.

The answer isn’t better math but a gestalt that is generally right

We need to use the clinical evidence we have to make diagnosis more scientific and not a simple “art” done differently by each doctor
Diagnosis is often an afterthought—list icd-10 codes with the right words for $$

If a code is present, make sure patient gets the medication or documentation for performance metrics

@fda & @cms have minimal test oversight @drJoshS
ja.ma/2ReZOBf
The billing/metrics false-simplification of clinical medicine worsens patient care and is a barrier to nuance and #SDM @RichardLehman1

I think the answer lies with clinicians being encouraged to doctor @adamcifu , @Bob_Wachter
The first step is acknowledging we have a problem w/ #diagnosis
leading to a #diagnosticerror of COMission
not described much in @theNAMedicine report nationalacademies.org/our-work/diagn…

@HardeepSinghMD
Solutions may not be that difficult if we:
-teach probability & clinical research > pathophysiology
-guide pretest probability—history/prevalence/exam
-teach test interpretation w/ natural frequencies NOT 2x2 table

ja.ma/3tpejRr
@VPrasadMDMPH
@AdamRodmanMD
cont..
-have reference clinical sensitivity & specificity testingwisely.com
-implement tests w/ dx stewardship ja.ma/2VGYs2F
-understand and accept #uncertainty (@greyscalespaces)

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Dan Morgan

Dan Morgan Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @dr_dmorgan

21 Dec 20
We set out to identify the sensitivity & specificity of common tests for #COVID19 along w/ @dan_diekema , @Anthony98947615 w/ @CDCgov support

A simple enough task, right?

I’ve seen tweets by @DrSidMukherjee @drjohnm @BenMazer @PaulSaxMD @BradSpellberg and others interested
1/n
Looking for comments/criticism

What are we missing? No industry adverts please!

Important papers?
2/n
The @US_FDA has a test comparison site that is incomprehensible to me… but @ASMicrobiology types tell me it reports on analytical sensitivity and LoD for tests
3/n
Read 20 tweets
15 Sep 20
Catch yourself when you say “risks vs. benefits” because you aren’t making a fair comparison.

In @JAMA_current ja.ma/336Lj4Y
& podcast ja.ma/2FAGKI2

@eliowa @drjohnm @d_spiegel @zeynep @VPrasadMDMPH

Why we say it and what we think is better below...
This building block of clinical decisions biases by framing uncertain harm vs. certain benefits and nudges towards treatment

Written with
@DKorenstein @ldscherer
(Over 2 years, i'm embarrassed to admit)
Clearly, the words physicians use have
a critical function in this communication

Referring to harms as “risks” emphasizes that
the unfavorable outcome may or may not happen,
whereas there is no parallel language that highlights
the equally probabilistic nature of “benefits.”
Read 8 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!