Thread by @ProfDFrancis

One terrifying possibility from the NIAID press release, pointed out to me by @venkmurthy

Did they say "randomized to remdesivir" or "who received remdesivir"?

Science is about focusing on the CORRECT details.

Facing the wall of bullshit that has been reported in the media about COVID, proper scientists know to ignore almost all of it.

Science is like windshield on a roadster.

Without it you end up with a faceful of crap.

Someone comes to you with "data" about an exciting new treatment?

Check whether it is an RCT.

If it isn't, punch them in the face.

Brutally (but humanely).

Next on the checklist is to make sure it is not a non-randomised trial disguised as a randomised trial.

Suppose your family member is a bit dehydrated and they don't seem keen to drink fluid.

You bring them to the Golden Nugget Clinic (Proprietors William Suh and Darrel Francis).

We normally make a fortune treating asymptomatic pain, but numbers are down at the moment.

What would you want them to receive at the clinic?

Fortunately, we have not yet reached peak unscrupulousness, and even though we have shelf-fuls of Impellas going out-of-date, we resist the temptation of increasing your relative's cardiac output that way.

We go for something conventional, although disappointingly inexpensive.

"It's sterilized," I reassure you as I infuse the life-giving fluid into your loved one.

All goes smoothly, until you notice this at the bottom of the fluid.

As you stare, the bug waves cheerily back at you, as if wishing your loved one a quick recovery.

You chase me into my office, and I ram myself against the door and shout back replies against your torrent of abuse.

"Sterile? Sterile? There is a whole menagerie of creepy crawlies in that saline!"

"Yes," I reply, confidently. "The saline is sterilised. Then, of course, I used some to water my plants. But don't worry, I poured the excess back into the bag."

Was the saline indeed sterilised?

Is it good enough to know that it was indeed sterilised, as advertised?

Why not?

When people tell people that something is sterilised, the purpose could be to convey

(a) a fact about a PREVIOUS event in the history of its existence, or

(b) a fact about its CURRENT status (i.e. still sterile, not cycled through a plant pot since)

Which would YOU mean?

And any sensible person does.

People who don't are either insane or deceptive, and should have this fact tattoo'd on their forehead, if they work in a hospital.

The same applies to RCTs.

If I say a trial is a RANDOMIZED controlled trial, I mean it is STILL randomized, now.

If people are changed between arms, it is not randomized any more.

EVEN IF ONLY A LITTLE BIT CHANGED.

Just like *one* passage through a plant pot destroys any sterility of the iv fluid.

And just *one* living creepy-crawly is enough evidence to reject the sterility belief.

The only reason to bother to randomize patients is to do analyse them according to their randomized arms.

There is *no point* bothering to randomize if you are going to shuffle them between arms afterwards. You may as well make up the results. Save time and money.

That is why I refuse to let my group write in any paper that we analysed data based on "intention to treat", unless forced to by reviewers.

I consider it as stupid as appending, to our address at the top of our paper, after "UK":

The World
The Solar System
Milky Way Galaxy
Local Group of Galaxies (Left dumbbell)
Virgo Galaxy Supercluster
Laniakea Supercluster
The Universe

Where else could "London, UK" be?

Likewise, how else would we analyse an RCT, other than by the randomly allocated arms?

However some people do allow arm-switching analyses which inevitably make the therapy look better.

I despise them, but accept they exist.

Like murderers and thieves.

Just I wouldn't let a murderer look after my child, or a thief look after my wallet, I wouldn't read an arm-switching analysis of an RCT.

I am *trusting* that proper scientists, of the calibre that NIAID/NIH would take on, would also consider this obvious.

That they would not de-randomize a randomized trial, just as they would not de-sterilise a sterile i.v. fluid bag.

So (in their case) I am happy to ASSUME that they are just using the terms "received" and "randomized" as synonyms, just as I am happy to ASSUME they didn't spit into the remdesivir.

I personally don't need them to tell me.

However if you doubt them, you should ask them!

I realise I should provide two clarifications.

1. I am saying I believe the NIAID trial to be randomized, intention-to-treat

This belief is based on my experience of how experienced professional scientists talk about research, especially on such important and dangerous matters.

It's only a belief. I might be wrong. But, just like a doctor listening to a patient describe an unclear history, I have come to a professional judgement.

If I turn out to be wrong (when they issue more info), I will be shocked and disappointed, and NIH, NIAID, Tony Fauci, the trialists and especially the DSMB, will have lost a lot of credibility in the eyes of me and of millions of scientists worldwide.

2. My second point is that some people are wondering why it matters whether they analyse it Intention-To-Treat or on some other bullshit method, like As-Treated or Per-Protocol or various other euphemisms for "Sorry, we ran the trial data through a plant pot."

Here is why it matters.

Suppose William runs an RCT, of "Suh's Health-enhancing Infusion Therapy" versus placebo.

The S.H.I.T. protocol requires 30 days of infusion of homeopathic bleach, or crystal UV therapy of the toenails, or something like that.

The key is that you have to have the 30 days, otherwise you didn't try hard enough. (Coincidentally it is $10,000 per day)

We randomize 200 patients, 100 to S.H.I.T. and 100 to placebo.

Outcome is assessed at 90 days.

In the placebo arm, 20 people die, mostly at around 10-20 days.

In the S.H.I.T. arm,

*also* 20 people die, mostly around 10-20 days.

The analysis is conducted independently by that great statistician: me. (Qualifications: zero)

"They didn't get S.H.I.T., so you can't count them," says William, when he hands over the raw data.

"That's not true, William, they got something, not zero."

"No, I didn't mean they didn't get shit, I meant they didn't get S.H.I.T., the whole S.H.I.T. protocol."

"Oh OK"

Based on that high-level scientific discussion, we delete 19 of the 20 patients in the S.H.I.T. arm, because they didn't get the whole treatment protocol.

In fact a couple died on the first day, before the first infusion!

They really didn't get shit.

To be on the safe side, we analysed it both ways.

We are very scientifically careful, at the Golden Nugget Clinic.

Analysis 1. "Per protocol: kick them out if they didn't get complete S.H.I.T."

Control: 20 died, 80 lived. 20% Mort
Active: 1 died, 80 lived. 1.25% Mort.

P value:

Wow.

Phone up Dr Fauci and tell him to get back on that Sofa!

P = zero point zero zero zero zero!

But we can do better!

Remember the two people who were going to have S.H.I.T. but didn't even get shit? They died before the first infusion?

Suppose we stick them in the opposing arm. After all, they didn't get shit!

If we put those 2 into the control arm, which of the following tables would apply?

A
22, 20
1, 80

B
20, 82
3, 80

C
20, 80
3, 78

D
20, 80
3, 80

This is option 2, namely "As-treated" analysis.

It is even more wrong than the "Per protocol" analysis because it exaggerates the benefit even more.

P value?

So *that* is why I consider the *only* form of analysis for an RCT is intention-to-treat.

The only time you ever see another type of analysis is when the intention-to-treat is neutral and the investigator is trying to salvage some credit with the funding company by mangling it.