, 9 tweets, 4 min read Read on Twitter
In last week’s tweetorial* about the area under the ROC curve, I wrote that “there are some exceptions when I would not rely on AUC, but these are rare.”

One of these exceptions is the Gail model for the prediction of breast cancer.

* see here:
When I tell people that risk models with low AUC are unlikely useful, I generally get the Gail model as a counter-example. The AUC of the Gail model is mostly <0.60, which is low and unlikely useful by my standards.
But the Gail model has something most risk models don’t have.
The Gail model has a relatively rare risk factor that has a strong impact on disease risk: family history. Having 1 or 2 first degree relatives with breast cancers increases the risk of disease by several-fold. See all effect sizes (RR) in the table.
Rare risk factors increase the risk for a small group of people but do little for the rest. The risk distribution looks like a ‘normal’ distribution with a long flat high-end tail—if the study is large enough to have enough people with the rare risk factor.
Risk distributions often have high-end tails but when they are less flat, they come with higher AUC.
AUC is a global metric, not insensitive but a measure for the big picture. AUC doesn’t easily show the impact of rare risk factors with strong effects. You may see a steeper ROC curve in the corner of the plot that represents the high-end tail of the distribution.
The Gail model does not always have a low AUC. Here is an exception where its AUC was 0.74. This study was conducted among women attending a Family History Clinic and all had a family history of breast cancer. The rare risk factor was common in this population.
Don't use the Gail model as an example of a great model. It isn't. At their threshold, the true positive rate (0.46) is slightly higher than the false positive rate (0.34). The model identifies women with a family or biopsy history, but does it do more than that? I don't know.
More on this can be found course manual on prediction research, including many references to key methodological papers (update is on its way). The slides of this thread can be downloaded here: cecilejanssens.org/slides-from-tw… … (later today)
Missing some Tweet in this thread?
You can try to force a refresh.

Like this thread? Get email updates or save it to PDF!

Subscribe to Cecile Janssens
Profile picture

Get real-time email alerts when new unrolls are available from this author!

This content may be removed anytime!

Twitter may remove this content at anytime, convert it as a PDF, save and print for later use!

Try unrolling a thread yourself!

how to unroll video

1) Follow Thread Reader App on Twitter so you can easily mention us!

2) Go to a Twitter thread (series of Tweets by the same owner) and mention us with a keyword "unroll" @threadreaderapp unroll

You can practice here first or read more on our help page!

Follow Us on Twitter!

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just three indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3.00/month or $30.00/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!