@Laconic_doc @statsmethods I think Alama has been called out by @GSCollins, I don't know about Public Health England.

Also, I actually never mentioned your name or link to your website to avoid public ridicule
@Laconic_doc @statsmethods @GSCollins That said, I have personally did quite a few things to warn you

First, I send you emails to which you politely and quickly responded. Thanks. You seemed to agree with my critique, but you didn't show any initiative to change it or remove the model
@Laconic_doc @statsmethods @GSCollins Second, I am one of the authors of a reply to the OpenSAFELY study where we specifically mention their model falls short of developing a risk model. You seem to have ignored that and used their multivariable results anyway
@Laconic_doc @statsmethods @GSCollins Third, I am the senior author on a living review on diagnostic and prognostic models for COVID. We have been warning about bad prognostic modeling since March. You seem to be familiar with the publication, but you didn't pick up on the modeling advice bmj.com/content/369/bm…
@Laconic_doc @statsmethods @GSCollins I don't know what more to do to stop models that are poorly developed and not validated to become publicly available or even used.

I'm sure you do not intend to do harm, but we know that bad prediction models *can* do harm, no matter how well intended they are

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Maarten van Smeden

Maarten van Smeden Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @MaartenvSmeden

26 Oct
Today started with email with a new COVID mortality calculator send to a group of researchers

After contacting the developer they explained the calculator uses a *selection* of coefficients from multivariable models published in literature
they had no idea of predictive performance....

but acknowledges the limitations

are you kidding?
there is no doubt this "model" is meant to be used as a prediction tool and it is available online

acknowledging limitations is a really poor substitute for careful development and validation of what is essentially a medical device
Read 4 tweets
14 Oct
HOW DO YOU DEVELOP A NEW PREDICTION MODEL?

This [THREAD] has been long in the making and is arguably overdue

1/138
I'll assume you have some basic knowledge of prediction models and will be relatively short on the technicalities

lets suppose you interested in developing a prediction model for disease X

2/138
There are probably a few dozen prediction models already developed for disease X!

most of them have never and will never be used

so... are you really, really, really sure the world is waiting for a new prediction model for disease X?

/138
Read 4 tweets
16 Sep
the ultimate reviewer #2 bingo card
key citations 👇
unclear analysis aims
stat.berkeley.edu/~aldous/157/Pa…

evidence of absence fallacy
bmj.com/content/311/70…

data dredging
bmj.com/content/311/70…

noisy data fallacy
science.sciencemag.org/content/355/63…
Read 6 tweets
3 Aug
The BMJ just published an editorial about living systematic reviews worth a read, which is new territory for just about everyone bmj.com/content/370/bm…

ICYI, I have a few thoughts to share
We were fortunate to have produced @bmj_latest first living review
bmj.com/content/369/bm…
The aim of our review is (and always was) to give an overview and appraisal of currently available diagnosis and prognosis models related to COVID-19

But this is a fast moving field: from 31 models reviewed in April to 145 models reviewed in our 2nd update published in July
Read 14 tweets
11 Jul
Used to get annoyed by stats consult clients who insisted they needed machine learning for their very large dataset (N of 100s or few 1000s). Now I tell them logistic regression *is* machine learning and everything is great again
And since machine learning is a sub field of AI, logistic regression is also AI. I should have understood this sooner
Logistic regression as statistical model
- prepare data
- estimate model
- evaluate performance
- report

Logistic regression as machine learning
- prepare data
- estimate model
- evaluate performance
- report
Read 4 tweets
9 Jun
Was asked for personal favorite resources for improving methods and statistics skills. I promised to make it a thread, so here it is

1/n
I work in medical research, so that is going to be my focus here too. But I’d like to think the resources are relevant to a wider audience

This list should not be taken as a guide to become a statistician, nor is it a must-read list for all academics (obviously)

2/n
My personal view is that medical research would benefit from involving trained statisticians earlier and more frequently; not from everyone trying to become one

Here are some good arguments by @statsepi:
medium.com/@darren_dahly/…

And some more: medium.com/@darren_dahly/…

3/n
Read 20 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!