Letter: "The Clinician and Dataset Shift in Artificial Intelligence" out in @NEJM w/ @_asubbaswamy , @suchisaria, @kdpsinghlab, @zittrain, @zakkohane, et al

Link: nejm.org/doi/full/10.10…

Full version of Table (w/ details+citations): sgfin.github.io/assets/dataset…

Brief thread: 1/n Dataset shift is a fascinating challenge which IMO poses the single greatest obstacle to durable, high-value medical ML

In this work, we provide a framework for medical dataset shift -- breaking it down into shifts in technology, shifts in population, & shifts in behavior.

2/n

Fun paper: clinicians who’ve never coded use AutoML to train DL classifiers + compare to published models

Touches upon a broader long term question: going forward, what precisely is the role for clinical experts in building medical ML?

(This paper hints at one extreme)

1/9

https://twitter.com/pearsekeane/status/1169832653152866304

I expect an interesting spectrum of answers to the above ?, bc I’ve seen some MLers very critical of notion of non-engs modeling, while others ie @jeremyphoward/@math_rachel advocate more of an “anyone can cook” philosophy (even as they’re gently skeptical of AutoML per se)

2/

I had the pleasure of speaking as part of an AI tutorial at ARVO today. Lots of really great work going on in the field, causing me once again to agree with @EricTopol's take on this.

A few people asked for my slides, so figured I'd make a thread out of it. 1/

https://twitter.com/EricTopol/status/1116364422086180864

Full (22) slides are here sgfin.github.io/assets/slides/…

The primary audience was practicing ophthalmologists who are interested in AI -- hence all the eye imaging cases. But I hope it's interesting more broadly.

Highlights below. 6 questions we must learn to ask:

2/

@KrishnanSid @EricTopol @ScienceTM @amitayer @MatthewPottsMD @NorthwesternEng @NeurosurgeryNM Great paper. And a mini-thread:

50% of CSF shunts fail within 2 years, and each revision requires major brain surgery. ncbi.nlm.nih.gov/pmc/articles/P…

My sister with hydrocephalus had >>100 such procedures, before dying at 26 from a series of infections *acquired from shunt revisions* @KrishnanSid @EricTopol @ScienceTM @amitayer @MatthewPottsMD @NorthwesternEng @NeurosurgeryNM Most people haven't heard of hydrocephalus, but shunt revisions are the most common brain surgery done in children, in part because many hydrocephalus patients have to have so many. In the case of some, like my sister, the very attempt to get the treatment working can kill them.