A Thread.
My column in @newslaundry criticised the RCT of Coronil done by Patanjali on, among other grounds, its small sample size. Many have asked me to say what sample size I would have recommended. So here is a #Tweetorial in RCTs and sample sizes. newslaundry.com/2021/02/22/pat…
Imagine a conversation between an Ayurvedic Researcher (R) and a Statistician(S) . It might go something like this:

R: "hey I need a sample size calculation. In my last trial I had 100 patients and the critics laughed, ‘it’s too small’, they said. "

S: Okay lemme help.
S: What is the outcome measure you seek to study?

R: Virus clearance at Day 3

S: I see. So, what is your best guess of the virus clearance rate at Day 3 in the absence of any treatment?

R: I dont know. We think its is 50%
S: That sounds high, I hope you are counting the days properly What is Day 0 for you?

R: Day 0 is when my subject comes to me with a positive RT-PCR.

S: tha's when YOU 1st have evidence of virus +ve status. It’s not the first day of infection. Have you looked at Lead-time bias?
S: While you think abt that, Lets go with the 50% estimate. Now, What is the smallest change on 50% that you think is clinically important NOT to miss?
R: From 50% background rate of clearance at day 3 we think a 70% clearance would be a useful improvement.
S: So lets agree a difference of 20% either way from the background 50% would be the minimum difference that you would not want to miss detecting..
R: (looking confused ).. I dont get this 50% +/- 20% business. Its an Ayurvedic drug how can it be worse than a placebo?
S: Look, park that for a bit. Can we agree that you’ll follow the usual 5% alpha error rate and 20% beta error rate.

R: What do they mean?

S: Okay, listen carefully this can be a difficult tghing to grasp at first.
S: Well the 5% alpha error means that in the experiment you’re about to do, assuming your coronil is no better or worse than a placebo, you are willing to run a 5% risk that you (erroneously) conclude that Coronil is different than the placebo . Hence p<0.05, get it?
R: And the 20% beta error rate?

S: That means that if Coronil is in reality not the same as placebo then you're happy with a 20% chance that the experiment you are about to embark on will fail to show a difference and lead U to conclude (erroneously) that Coronil is worthless.
R: But hang on, Coronil is an Ayurvedic drug. It cannot be worthless. Ayurveda has been around for millennia. The Vedas have described it.

S: Every researcher says something like that. " My drug HAS to work", they say. But you're a scientist now about to do an experiment.
S: You have to think like a scientist, keep an open mind. Be sceptical. Be objective. But if you are very sure that coronil is as nectar from the Gods, why do the experiment at all? You have to be open to all possibilities when you do an experiment. Agnostic about the result.
R: This is all so confusing. I just want to prove that my Coronil works, tell me a number for the sample size.
S: The trial you are going to do does not PROVE anything, it is merely to gather evidence for or against the null hypothesis that Coronil is the same as a placebo.
R: That’s not what my Acharya told me. He wants to publish a paper in Phyto-something to show that coronil does some good. Why don't you give me the sample size number that I need to recruit.

S: Oh well okay then, let me help, Let me recap where we have got to.
S: You think the background rate of virus clearance at day 3 is 50%. You don't want to miss a difference of 20% either way. And we agree that we fix the alpha eror rate at 5% and the Beta at 20%.
R: Right. If you say so.
S: No No YOU gave me those numbers. It's your experiment.
S: I think 20% is too high. Surely a 15% diff is worth detecting. Lets set it at 15%, please. If Coronil really offers a 65% viral clearance at day 3 you want a to design an experiment NOT to miss that big a difference with more than a 20% prob.
R: okay lets go with that.
S: Brilliant. I’ll plug those numbers into Epicalc this free software from EpiInfo from the guys at Emory University School of Public health in Atlanta, Ga.

R: The guys who wrote this software at Emory; do they know anything about Ayurveda?

S: I guess not.
S: but then they dont know much about cancer or modern medicine either, they are statisticians you see, they design statistical experiments and do the analysis. An RCT is a statistical experiment.

R: okay lets go!

S: Ive plugged those numbers and the sample size I get is (pic) Image
S: You'll need to recruit about 360 patients, 180 in each arm of your trial to be able to demonstrate a +/- 15% (or greater) departure from the background 50% virus clearance that you expect in the absence of any treatment.
R: That’s bonkers, Coronil works, it says so in all the ancient texts I’ve read. I am not going to do a trial in 360 patients give half of them a placebo that might let them die. This statistics is all a western con trick!!!!
S: I am sorry you feel that way. That’s what I said at the beginning. If you’re so sure Coronil works and you cant face the prospect of putting it to the test then you should market it on the basis of faith newslaundry.com/2020/11/11/ayu…
Note: this entire tweet thread is fictional. All researchers react with shock and disbelief when a statistician gives them a sample size number. But just as you need a BIG telescope to detect a far away star, you need a large sample to detect small effects.

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with JayEnAar

JayEnAar Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @GorwayGlobal

23 Feb
I am delighted beyond imaginable that #DishaRavi has been granted bail. But I find a few aspects of the Court's order both inexplicable and worrying.

Let me explain,
First, @ndtv reports that the Judge said, "...it cannot be presumed by resorting to surmises or conjectures that she also supported the secessionist tendencies or the violence caused on 26.01.2021.."

Q. What evidence did the Judge have to label the violence as "secessionist".
If the Judge was relying on the police charge that the violence was by #Khalistani agents then he should have been careful to qualify it as "allegedly secessionist violence".
Read 6 tweets
21 Feb
Those bemoaning the rise in #PetrolPrice in India shd know that @nsitharaman can do nothing about it. You see, India's public finances are in a dire state. We now have the accounts to Dec 2020 - i.e. 9 months of FY 2020-21. Let's focus on the receipts. See this pic. Image
Start with the last line:Total receipts. Pro-rata to the Budget estimates for 2020-21 of 22.46 Lakh crores (LC) at 9-month mark, the receipts are short by 5.6 lakh crores. That is a 34% shortfall. looking at the detail of Line 4. Non-debt capital receipts we get this information Image
Effectively, there has been ZERO disinvestments of the Gvt's stake in Public sector banks and financial institutions, a 77% shortfall in the disposal of Govt's equity holdings.
Read 6 tweets
6 Feb
@Barristotle and I have formally written to the MP for Harrow East to point out the falsehoods and misleading statements in the infographic tweeted below. We asked him to consider the facts, and withdraw the tweet and apologise. Unacceptable for a British MP to spread falsehoods
It is an open letter so I shall tweet the contents of the letter as screen shots:
1.
2.
Read 7 tweets
6 Feb
Aaji Shukla in a remarkable, eye-opening analysis on the enormous political significance of Modi's Farm Laws miscalculation. Screenshots follow. thecitizen.in//index.php/en/…
Read 6 tweets
5 Feb
A tweet thread.
Re: #FarmersProtest There is a great deal of ignorance and misinformation about the freedoms currently available to Indian farmers and big business houses., even before the #farmLaws were dreamt up by the #ModiGovt that is causing so much turmoil in India.
Take one example. When McDonald's (NYSE: MCD; Market Cap USD 158 Bn) opened in India, they found the Indian potato was not the right kind for their french fries. The Govt did not allow the import of American or European potatoes. What did they do?
Read on:
McDonald India went to the enterprising Indian farmer and did a deal "Grow this variety of potato, to these specs and we'll guarantee to pay this price and what's more we'll pick up your crop come what may". No #mandis no #APMC. #ContractFarming with guarantees.
Read 16 tweets
4 Feb
When looking at financial numbers the context and change over time is important. So a fiscal deficit in Year 20X1 of 5% can only be understood by looking at the time trend over a few years. So here's some time trend charts about #Budet2021 for #India.
1st the fiscal deficit Image
The fiscal deficit is the gap between A)Total expenditure by the Government and B) the sum of revenue receipts (tax + non tax), loan recoveries, other receipts(these are mainly PSU disinvestments). This is the gap that has to be plugged by the Government borrowing money
#Budget2021 forecasts that the interest the Govt expects to pay for accumulated Govt debt (+ any new borrowing in-year) at 8.1 lakh crores). This makes most sense when seen as a proportion of revenue expenditure. Image
Read 5 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!