Our paper dropped: Performance Metrics for Comparative Analysis of Clinical Risk Prediction Models Employing Machine Learning. We show 'commonly reported metrics may not have sufficient sensitivity to identify improvement of
#ML models…’ @CircOutcomes ahajournals.org/doi/abs/10.116…
@CircOutcomes Risk models are ubiquitous now. In this paper, we 'propose the use of a comprehensive list of performance metrics for reporting and comparing clinical risk prediction models.’ Time to expand the metrics. @CircOutcomes ahajournals.org/doi/abs/10.116… @YaleMed @YaleCardiology @AHAScience
@CircOutcomes @YaleMed @YaleCardiology @AHAScience We review a wide range of options for assessing the performance of risk models and demonstrate the neccessity of a comprehensive view in any evaluation. Paper was led by Chenxi Huang. Also with @jbmortazavi; SL Normand; @jspertus @CesarCaraballoC @Dr_BowTie65 @DrJRums
@CircOutcomes @YaleMed @YaleCardiology @AHAScience @jbmortazavi @jspertus @CesarCaraballoC @Dr_BowTie65 @DrJRums "Such a suite of metrics provides a comprehensive evaluation and comparison of the [risk] models, which is a critical step toward understanding the performance of a model when
implemented in practice.” @CircOutcomes
@CircOutcomes @YaleMed @YaleCardiology @AHAScience @jbmortazavi @jspertus @CesarCaraballoC @Dr_BowTie65 @DrJRums Finally kudos to @CircOutcomes team for their help in improving the manuscript. I am an advocate for #preprints, but I am also an advocate for thoughtful, constructive peer review. And this article is open to the public, which is great because it seeks to set standards. @bnallamo
@CircOutcomes A premise of our paper is... traditional metrics of risk models may miss some of the advantages of #ML #AI models that capture better the risk of some patients, esp at the extremes of risk. In a precision approach, getting it right on each person matters. @YaleMed @YaleCardiology

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Harlan Krumholz

Harlan Krumholz Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @hmkyale

28 Sep
"The @PCORI Board’s approval of $61.3 million will enable eight PCORnet CRNs to optimize their infrastructure resources and capacities to support PCORnet in its third phase." What is the total spent on PCORnet to date? What are the deliverables to date? pcori.org/news-release/p… Image
@PCORI And asking w/respect, just think the accounting of resources spent & deliverables provided is good hygiene for funders. And what resources from the effort are publicly available for others to leverage? Since this is such a big investment, regular reports on ROI is useful. @PCORI
@PCORI And this might be available, I just didn’t know where to look. By the way, similarly with @NIH, who previously reported that almost half of their funded trials were never published. Quarterly reports on trial funding and deliverables (results reported), for example, also great.
Read 4 tweets
30 Aug
With much talk about wearables & devices picking up unappreciated atrial fibrillation, I was intrigued by this randomized trial of effect of an implantable loop recorder to detect afib on outcomes. These slides are from #ESCCongress presentation. Kudos Jesper Svendsen and team. Image
The team identified participants with a high risk of stroke and randomized them to an implantable loop recorder (Reveal LINQ by Medtronic), with a primary outcome of stroke or systemic embolism. Question: would better detection of afib improve outcomes? Image
They randomized 1:3 - so most people were in the control group. 1501 randomized to the implantable recorder (and 1420 received it, that will be important later) and 4503 in the control group (and none crossed over and received the recorder). They followed them for median 65 mos. Image
Read 18 tweets
27 Aug
CardioMEMS. Expensive, invasive device. Initial trial contaminated. FDA approves & it sells. Definitive trial now completed yrs later. No sig benefit. However, authors, look only at pre-COVID results, claim benefit, conclude it's worthwhile. #ESCcongress thelancet.com/action/showPdf…
Authors write: 'In conclusion, haemodynamic-guided management
across spectrum of ejection fraction & symptom severity was safe but did not reduce a composite of mortality and total heart failure events.” This seems very straightforward. We should stop using it. #ESCCongress2021
Then they say… "in a pre-COVID-19 analysis, a benefit of haemodynamic-guided management on the primary outcome, driven by a decrease in heart failure hospitalisations, was shown… supporting benefits of haemodynamic-guided management in patients with chronic heart failure."
Read 7 tweets
27 Aug
Remarkable win streak for SGLT2i drugs continues…now specifically for heart failure with preserved ejection fraction (HFpEF), where effective therapies are scarce. Empagloflozin benefits one of every ~30 people treated over median of 26 mos. nejm.org/doi/full/10.10… #ESCCongress
What gives me confidence about the finding is it is consistent w/what we have seen in subgroups of other studies, but not yet in a dedicated trial. Kudos @JavedButler1 Stefan Anker and team. This is truly a landmark in heart failure care. #ESCCongress2021 #ESCcongress @escardio
@JavedButler1 @escardio Another notable about this EMPEROR-Preserved trial… empaglifozin won across a range of outcomes, across all the subgroups (incl people w/o diabetes), and w/o evidence of safety issues (more serious safety issues in the placebo group). Very welcome set of findings. #ESCcongress
Read 6 tweets
19 Aug
Illuminating the disparities… "During COVID-19, Black & AI/AN persons had highest excess all-cause mortality IRs among <25yrs & 25–64 years; among ≥65 yrs, largest excess mortality occurred among Black & Hispanic persons.” @CDCgov @jeremyfaust @YaleMed cdc.gov/mmwr/volumes/7… Image
@CDCgov @jeremyfaust @YaleMed We learned a lot in researching this study…using the metric we pioneered (I credit @YNHH CORE team and @jeremyfaust) we show: 'Black persons had highest excess mortality IR among <25 yrs with 14.1 excess deaths per 100,000 person-years in 2020, followed by AI/AN persons (6.5)."
@CDCgov @jeremyfaust @YaleMed @YNHH Also, "Among adults aged 25–64 years, the highest total excess mortality IR was among AI/AN persons (221.1), followed by Black (133.4), NH/PI (124.9), Hispanic (98.5), White (51.2) and Asian persons (30.2).” These differences are huge and demand our attention.
Read 7 tweets
8 Aug
For States and regions in the US that have low vaccination rates because people don’t want it, wouldn’t the federal gov’t be better off waiting until they ask for help rather than trying to push/force vaccinations on them, which just seems to increase resistance.
In this construct, the federal gov’t would say, we are ready and eager to help with vaccines, funding, people - let us know when you are ready. This, in place of being characterized as forcing people. De-politicize it… feds just wait to be invited to help.
The key is to make vaccines accessible and free to everyone; make trustworthy information about the vaccines abundant and clear. And let people make their choices. Be the sun and not the wind, and people will eventually take off their coats.
Read 6 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!

:(