David Nunan Profile picture
Sep 30, 2020 15 tweets 5 min read Read on X
Have to say I’m somewhat surprised by these pronouncements, and modus thereof (RR without AR etc), particular given the expertise.

The replies here on methods are important, especially this:


⚠️ of “more research” feel cursory.

#classiccovid
Also not pre-registered and certainly not PRISMA compliant in pre-print version.

Hmmm....
Also, did the authors check if their review was needed?

There’s a living systematic review published back in August with sound methods:
acpjournals.org/doi/10.7326/M2…

Conveniently its last search date just preceded the 3 studies including in this pre-print.
There are currently 12 pages of systematic reviews on hydroxychloroquine returner in @epistemonikos

epistemonikos.org/en/search?q=Hy…

Many of these looking at prophylaxis.

#racetobethewinner
#classiccovid
Another is the fact that no consideration has been given to the certainty of the evidence e.g. risk of systematic bias (2 of the studies are pre-prints), random error (amount of information, precision). Given one of the authors has written the book on causal inference = ironic.
So @_MiguelHernan has updated the MA to include this latest study: jamanetwork.com/journals/jamai…

If we include the only two studies that have been peer-reviewed then the MA looks like this.

That's before we again look at the overall certainty of evidence. Image
Also, I did that meta-analysis in 1 min. This shows the absolute pointlessness of MA's outside of the context of a properly conducted and reported systematic review, that includes an assessment of the certainty of evidence (i.e. GRADE or similar).

So poor.
If the risk of bias of the trials is high, then we downgrade the evidence from high to mod.

We have to make a decision as what a clinically meaningful effect here is? 25% RR is considered by some as 'meaningful' for estimates of non-life threatening outcomes eg risk of COVID +ve
In pre-print MA - CI crosses meaningful (low as 0.61) but also includes trivial/no benefit. I would downgrade here by one again meaning we low quality/certainty of evidence.

For my MA, would consider downgrading two levels as shows potential meaningful harm (so very low qual)
I haven't the time to look at whether there is indirectness (the studies don't include relevant/similar populations, intervention, comparator, outcomes) for the research question.

Probably also fails on the optimal information size (enough info) as here we have <300 events
Only thing it (the totality of evidence) doesn't get downgraded on is 'consistency' (results go in same direction, overlapping CIs etc).

Publication bias also needs looking at. No idea what the authors of the pre-print did on this.
How many trials on hydroxychloroquine have been registered/non-registered.

How many of these unpublished? Though the fact that the existing trials are all null trials does indicate that studies with p>0.05 are being published.
A good SR+MA would do all these checks.

It's a pre-print so we can give benefit of the doubt.

But a press-release tweet of a single point estimate that considers little of these CRITICAL issues from a MA that wasn't pre-registered + poor reporting = almost ignorable.
A tweet to nicely end this thread on:

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with David Nunan

David Nunan Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @dnunan79

Feb 29
Lots of takes on this. Many are what people have anyway regardless of this paper, such is the general diet/nutrition discourse, and particularly around UPF.

For those of you interested in what an actual evidence-based approach to this paper looks like, buckle up🧵

1/
In my view it is a good example of pervasive issue in the way most medical/health research is interpreted, and this even includes the authors themselves.

What’s the issue?

IGNORING UNCERTAINTY!

2/
A key function of a systematic review is to inform as to how certain we can be that the available data provides an observable truth.

There are different methods a review team can take. One of the most commonly used in med/health research is GRADE.

3/ Image
Read 26 tweets
Sep 22, 2022
"It's going to protect YOU..."
"It will reduce YOUR risk of a heart attack, cancer, diabetes etc".

We see this all the time when it comes to medical treatments and health interventions.

I'm going to show why 99.9% of the time this type of phrasing/framing ("YOU/R") is wrong🧵
When people hear these phrases with "YOU" / "YOUR" in them, it's most likely they will perceive the benefits & risks of treatment/intervention in relation to their own personal benefit/risk. The thing is, we do not, & cannot, know your own personal benefit/risk.
The vast majority of the time we only have information on groups of people (samples). Benefits/risks in this context relate to the numbers of people (with a similar health profile) among the group who either do or do not experience a given health outcome.

An example to help...
Read 17 tweets
Feb 8, 2022
ON MISINFORMATION

Here's the thing. You cannot police it in a fair, reproducible & representative way.

The reason being is the lines between misinformation and genuine uncertainty soon become blurred.
Trust me, I tried.

(ps. does this tweet count as misinformation)
I mean, look at Twitters own criteria.
"...widely available, authoritative sources'

Trouble is these sources are dealing mainly with evidence/information in the Green zone.
Lots of examples of the difficulty in this area.

Here's @cochranecollab being censored for spreading "harmful false information" (for the 2nd time)
Read 9 tweets
Apr 22, 2021
So here's the thing with this. I posted a tweet out of frustration of seeing an organisation promote evidence from a commentary piece as if it were proof a question had been answered. No uncertainty. Proof.
/1
The same organisation didn't mention a systematic review attempting to answer the same question (which happened to show uncertainty). Yes, the organisation was aware of the review's existence.
This signals, to me, an agenda. It's also a good example of the state of things.
/2
The state of things in relation to evidence and "evidence-based". It's a state I worry about, hence the original tweet.
This isn't about the topic/question. It's about the principle. Of promoting selected information as if it is proven fact, ignoring info that might contest.
/3
Read 15 tweets
Feb 5, 2021
Our latest publication revisits a well-known problem: reporting of relative effect estimates without absolute effects in journal publications of clinical trials:

ebm.bmj.com/content/early/…

🧵on this issue
("nah, show me the findings": )
First an intro to the problem. A practical example is probably best here. Take a look at the image. How many more people are at increased risk bowel cancer? Image
I'll be nice by making it multiple choice:
Read 46 tweets
Jan 2, 2021
Lots of folk already commented on “1 or 2 dose”. One thing that comes up often is “efficacy”, followed by numbers like “80%”, 95%”. In many cases, it reads as if folk think this is how much YOUR chance of getting the virus is reduced by. That’s incorrect.
These figures are actually the relative risk reduction (RRR) of infection with the vaccine. Eg. 2000 people without Covid-19 - 1000 vaccinated (group 1), 1000 not vaccinated (group 2).
200 people (20%) in group 2 get Covid-19
10 people (1%) in group 1 get Covid-19
= 95% RRR
The absolute effect is 19% (difference between 1% and 19%).

Another way of putting it = in 1000 people who don’t have Covid and are not vaccinated, 200 will catch it.
If the same 1000 people had the vaccine, 10 would get it, meaning 190 will be spared.
Read 11 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(