Thread by @venkmurthy

Major updates to NY Times COVID evidence tracker in first 24 hours.

But still lots of concerns.

**Thread**

nytimes.com/interactive/20…

They have eliminated strong evidence category.

Ventilation & proning now called "Widely Used"

Dexamethasone & remdesivir are now the only 2 w/ "Promising Evidence"

Anticoag (previously strong evidence) moved to mixed evidence (two grades down) along with most everything else.

Dialysis is now gone.

Why list ventilation without dialysis?

Overall, this is still *deeply* problematic.

Why are "Widely Used" given a higher rating than RCTs?

They have also not separated strength of recommendation from quality of evidence.

Something can have really good data but a small effect and so get a weaker recommendation than something like dialysis in really severe AKI that would be hard to randomize.

Not clear why they have picked some things to rate and not others either.

There isn't much COVID specific evidence for some of these.

Also critically, the fact that there were such massive changes within first 24 hours shows how poorly thought out this was.

Rather than updating, this should have been taken down with a serious mean culpa.

Synthesizing studies like this is very hard work. It takes lots of time and study to build up the required expertise. Even then experts disagree and thus we often have many experts working together with special methods to build consensus and rate evidence.

Often having different types of expertise is required (different medical fields, allied health, stats, pharmacology, economics, etc).

Even then we get different ratings by different societies in some cases (consider European and American ratings).

Thus this is hard even for experts who spend months.

NYT reporters cobbling together something overnight without process or expertise is downright dangerous and seems from the outside to be arrogant.

H/t to @dnunan79 for pointing out the update.