4 Jan, 31 tweets, 8 min read

SO MUCH Continued CONFUSION about Rapid Antigen Tests - Now about Asymptomatics.

A New @CDC_gov @CDCMMWR study concludes Ag tests didn't work in Asymptomatic ppl. Simply put, It's wrong

Here I got in depth on Rapid Tests & how to evaluate them

1/x
The study found rapid Ag tests missed 40% of PCR +ve Asymptomatics.

BUT... FAILED to state that 100% of the misses were in ppl with NO CULTURABLE VIRUS (plus ALL had Ct values >30)

Rapid tests CAUGHT 100% of the infectious ppl!
cdc.gov/mmwr/volumes/6…

2/x
That's right!

In the very same study that concludes rapid Ag tests do not work in asymptomatic people, the authors failed to state the rapid Ag tests SUCCESSFULLY CAUGHT 100% of ASYMPTOMATIC PEOPLE who had likely contagious virus.

3/x
In this thread I'll try to place that study in context, as well as discuss another highly publicized study from Liverpool,UK which I believe is hitting up against similar issues-in a less explicit way-causing an unnecessary loss in confidence in a rapid Test.

4/x
First, Some definitions:

Definitions (i):

Ct=Cycle threshold

High Ct (40) = VERY LOW
Low Ct (10) = VERY HIGH

Each 10 Ct's DOWN = 1000x Higher virus

Eg. If Ct 40=100 viruses, Ct 30 = 100K, 20 = 100MM, 10 = 100BB

5/30
Definitions (ii)

Ag = Antigen: a piece of virus. An Ag test looks for physical part of virus-like looking for a person's eyes

RNA = genetic code of a virus. A PCR COVID19 test looks for virus RNA, like a detective looks for DNA of a human.

6/30
Definitions (iii)

Culture = lab test to determine if there is "live" (likely contagious) virus in the sample

"+ve" & "-ve" simply mean "positive" & "negative"

So, a sample that is "culture +ve" means it has "live" virus detectable in it.

7/30
Definitions (iv)

A sample that is "culture -ve" means there's no discernable live virus in sample.

If a Sample is PCR +ve & Culture -ve it is very likely that the PCR is detecting old remnant RNA. I explain more below.

8/30

Here I show the actual PCR +ve Asymptomatic data from the paper.

As is PLAINLY evident, the rapid Ag tests detected 100% of Culture +ve PCR +ve specimens.

*Culture +ve on decent assay should be MINIMUM bar to consider ppl likely infectious

9/30
So HOW does this go so wrong?

How does a study get so badly distorted??

Simple, We must always DEFINE WHAT IS THE GOAL OF THE ASYMPTOMATIC TESTING!

In case of asymptomatic screening of ppl with no reason to think their infectious, it is to find CONTAGIOUS people.

10/30
The goal of Asymptomatic screening is NOT to medically diagnose ppl as having remnant non-infectious virus RNA.

NO! Leave that for medical testing.

Ppl w remnant RNA usually do NOT need to be isolated

**We MUST recognize PCR STAYS POSITIVE LONG AFTER INFECTIOUS**

11/30
If NO symptoms, you're testing for contagiousness.

We must stop thinking about tackling this pandemic w medicine but rather w PUBLIC HEALTH tools.

We defined the reasons for testing in @ScienceMagazine here w/ @K_G_Andersen

science.sciencemag.org/content/early/…

12/30
The @CDCMMWR study linked above failed to define why they were testing and thus got confused.

They said the tests missed 60% of the asymptomatics when in fact they caught 100% of the asymptomatic PCR +ves that they intended to detect!

The culture positive ones!

13/30
***I implore anyone studying rapid Ag tests to first DEFINE WHY YOU'RE TESTING, and from that, choose the metrics that make sense.

This is crucial.

If your unsure what your goals are, don't do the study or find someone to help. I'll help....

14/30
There've been many many rapid Ag test studies by ppl who haven't defined their goal. These unsurprisingly tend to ultimately come to a conlclusion that Rapid Ag tests don't work.

15/30
Meanwhile, studies by experts in this field have largely concluded these tests work well

The study that instigated this thread shows how WELL these tests work- though the data was interpreted with a slightly bent lens - discussed above.
cdc.gov/mmwr/volumes/6…

16/30
Here are a few tips on interpreting Rapid Ag Test studies

a) Define the goal. If goal is screening contagiousness, then study should put any rapid Ag tests in context of virus culturability OR attempt to measure Rapid Ag test sensitivity stratified on viral RNA load

17/30
b) Virus Culture +ve does not necessarily mean someone is infectious.

Rather, it should be considered IMIO as a minimum bar to entry that someone might be.

If not culture +ve, then from a public health lens, the person is most likely NOT a major public health threat

18/30
c) Yes, some virus culture systems are more sensitive than others. Of course. But most will do fine to get at this broad metric of whether to consider a sample to have enough live virus to consider likely contagious and a public health threat.

19/30
d) If evaluating against PCR Ct values - do NOT assume consistent Cts across labs.

Some can be wildly different from next!

20/30
For ex, many studies have found a Ct value of ~30 generally reflects the edge of culturability

Thus Ct 30 is often used as useful Ct cutoff for rapid Ag tests to shoot for

This is OK as a guide, but can be way off

This MMWR Study is a decent example of Ct 30 cutoff

21/30
If the PCR test being used to validate rapid test is calibrated with a lower Ct value (for whatever reason) this can make a rapid Ag test look much worse than it is.

I believe a real world example is currently sowing confusion in Liverpool, UK

22/30
In most studies that defined Ct ~30 as useful target for culturability/Rapid Ag tests, median Ct values of Asymptomatics is usually ~28-31:

Here is a mishmash of Studies showing Asymptomatic Cts - note medians are all around 30

23/30
In these studies 👆, rapid Ag test worked well up to Ct of ~30, or even better.

This is important b/c a very public study in Liverpool is showing outlier results where rapid Ag test looks like it is performing poorly... but I think the Ct distribution may be at play.

24/30
This study in Liverpool, UK is finding that Rapid Ag tests are performing well only up to a Ct value of about 22.

This is way different than other assessments of this same test - which have shown it works to Ct ~30, as expected.

However...

25/30
Inspection of Ct value distributions in Liverpool suggest shift in PCR Ct values:

The median Ct of Asymptomatic screening is a HUGE OUTLIER w median Ct of only ~21!

This is Exceptionally LOW for random screen of Asymptomatics - as shown in the figure a few tweets up.

26/30
The Ct result is either from a very rare chance of finding most asymptomatic people right at the peak of their infectivity (unlikely given the narrow time frame for this)

OR

More reasonably, the PCR lab may have a readout shifted down by about ~5-10 Cts.

27/30
There is other evidence for this.

The lab in the study claims a Ct value of ~30 equals ~100 viral RNA particles/mL

In MOST studies/labs, a Ct value ~100 RNA particles/mL is usually a Ct value of ~40.

Here is the Liverpool study
assets.publishing.service.gov.uk/government/upl…

28/30
Thus, in the Liverpool study, it is likely that the rapid Ag test is performing as we expect, Just that the PCR Ct values are shifted down from what we're used to seeing.

The study SHOULD evaluate vs culture to know for sure.

29/30
To CONCLUDE - we have to be very very careful when evaluating Rapid Antigen Tests.

We must DEFINE the purpose of the test. Determine the best comparison test for evaluation based on that definition, and only then report out how well the test is working.

30/31
Anything short of that ends up running into major problems with these tests.

This paper here is one of the best papers in terms of methods to evaluate rapid tests. IMO.

• • •

Missing some Tweet in this thread? You can try to force a refresh

This Thread may be Removed Anytime!

Twitter may remove this content at anytime! Save it as PDF for later use!

# More from @michaelmina_lab

4 Jan

On False POSITIVES of rapid antigen tests.

This 🧵 is on SPECIFICITY or the issues of False POSITIVES and rapid antigen tests.

Many people are concerned that these fast, inexpensive tests cause too many false positives and will overload PCR labs...

1/x
Rapid antigen tests can have 98%-99.9% specificity meaning between 0.1% and 2% of tests run might be falsely positive.

If prevalence is low, even a 99.9% specificity could mean many of the positive tests are falsely positive.

But w rapid tests, there are rapid solutions!

2/x
Many ppl instinctively worry that a false positive will mean an erroneous 10 day isolation and that huge numbers of people are going to require laboratory based PCR confirmation tests and this will overload the system.

No, this is stuck-in-our-ways thinking...

3/x
4 Jan
@apoorva_nyc @TakeWeightOffMD Seen a lot of responses talking about partial immunity here.

But, Unlike antibiotics, the comparison has to consider even greater “only partial immunity” among those who have had no vaccine by the time they get infected.

1/x
@apoorva_nyc @TakeWeightOffMD The immune response is going to develop whether you’ve been vaccinated or not. If you’ve had no vaccine, then there is going to be much more “only partial immunity” that the virus gets to play around with and “test out”

2/x
@apoorva_nyc @TakeWeightOffMD We have to very careful to not forget what the baseline is here. W antibiotics, the Bacteria is only exposed to a partial dose if youre on antibiotics. With immunity, the virus is exposed to a partial dose whenever you’re not already protected while immunity builds up.

3/x
3 Jan
Dear @CDCgov - Why can't you learn?

The paper concludes rapid Antigen tests had only 40% sensitivity in Asymptomatics

But failed to state they had 100% SENSITIVITY for what matters - culture +ve viable virus in Asymptomatics

PCR, not Ag, is wrong tool for Asymptomatic screens
The problem is simple

PCR remains +ve for long time after infectious

Meanwhile Ag tests are +ve only when infectious

If testing Asymptomatics randomly (vs. repeatedly) - they are MOST likely to be found post-infectious -> EXPECT Ag to be -ve MOST of time someone is PCR +ve
The very paper @CDCgov links to that concludes rapid Ag tests do not perform well in Asymptomatics literally shows 100% sensitivity for culture +ve samples in Asymptomatics

Massive failure to not emphasize this for asymptomatic screening.

cdc.gov/mmwr/volumes/6…
31 Dec 20
Dec 31 2020

Nearly 4000 #COVID19 deaths per day in US

Vacc are many months away for most ppl

Rapid paper strip tests *could* be in everyone’s home today to curb spread immediately

Write to your congress ppl and demand they be a priority

1/
We make it easy for you to do this and have collected signatures from leading scientists requesting congress act on these - Set aside funds that amount to <1% of the most recent stimulus bill to get these tests to everyone, quickly and for months.

Rapidtests.org
FDA / others think the populous is not capable of having simple tests at home & making responsible decisions

This is insane - plus we have expensive over the counter tests that require much technology - so I guess FDA feels the wealthy can use their own data but poor cannot. BS!
31 Dec 20
Great article bc it feels a bit haphazard (by intention) and thus captures the confusion of the current status of vaccine rollouts across the world.

But I want to talk about a specific quote in the article...

1/x
sciencemag.org/news/2020/12/i…
Many scientists have been frustrated that UK extended time between vaccine doses to ~12 weeks - in effort to get as many people as possible at least a single dose.

This makes sense for optimizing public health. But for optimizing individual protection it is controversial.

2/x
One scientist in the article said:

“I wouldn't want to be sitting around for 12 weeks waiting for the second dose with the partially protective vaccine,”

Under normal optimal circumstances, this opinion might be the right opinion, but In midst of a pandemic, it is faulty..

3/x
30 Dec 20
UK has authorized AZ/Oxford vaccine.

Importantly: UK is leading the way in taking a bold public health approach (vs individual health) to getting the population vaccinated - Prioritizing distribution of 1 dose & delaying dose 2 up to 12 weeks.

1/x bbc.com/news/health-55…
This bold move is going to cause ppl to scrutinize the decision. But in this pandemic, too few governments have acted swiftly to limit spread - bc quick action sometimes means imperfect data.

Here, robust data isn’t likely available for a 12 week delay for dose 2

But...

2/x
This was a good decision that

a) need not be permanent and suggests the UKG is willing to think with agility here and

b) likely will be the better overall public health approach rather than letting doses sit in freezers while others die and

3/x