Thread by @michaelmina_lab

THREAD **IMPORTANT** (IMO)

SO MUCH Continued CONFUSION about Rapid Antigen Tests - Now about Asymptomatics.

A New @CDC_gov @CDCMMWR study concludes Ag tests didn't work in Asymptomatic ppl. Simply put, It's wrong

Here I got in depth on Rapid Tests & how to evaluate them

1/x

The study found rapid Ag tests missed 40% of PCR +ve Asymptomatics.

BUT... FAILED to state that 100% of the misses were in ppl with NO CULTURABLE VIRUS (plus ALL had Ct values >30)

Rapid tests CAUGHT 100% of the infectious ppl!
cdc.gov/mmwr/volumes/6…

2/x

That's right!

In the very same study that concludes rapid Ag tests do not work in asymptomatic people, the authors failed to state the rapid Ag tests SUCCESSFULLY CAUGHT 100% of ASYMPTOMATIC PEOPLE who had likely contagious virus.

3/x

In this thread I'll try to place that study in context, as well as discuss another highly publicized study from Liverpool,UK which I believe is hitting up against similar issues-in a less explicit way-causing an unnecessary loss in confidence in a rapid Test.

4/x

First, Some definitions:

Definitions (i):

Ct=Cycle threshold

High Ct (40) = VERY LOW
Low Ct (10) = VERY HIGH

Each 10 Ct's DOWN = 1000x Higher virus

Eg. If Ct 40=100 viruses, Ct 30 = 100K, 20 = 100MM, 10 = 100BB

5/30

Definitions (ii)

Ag = Antigen: a piece of virus. An Ag test looks for physical part of virus-like looking for a person's eyes

RNA = genetic code of a virus. A PCR COVID19 test looks for virus RNA, like a detective looks for DNA of a human.

6/30

Definitions (iii)

Culture = lab test to determine if there is "live" (likely contagious) virus in the sample

"+ve" & "-ve" simply mean "positive" & "negative"

So, a sample that is "culture +ve" means it has "live" virus detectable in it.

7/30

Definitions (iv)

A sample that is "culture -ve" means there's no discernable live virus in sample.

If a Sample is PCR +ve & Culture -ve it is very likely that the PCR is detecting old remnant RNA. I explain more below.

8/30

Back to thread:

Here I show the actual PCR +ve Asymptomatic data from the paper.

As is PLAINLY evident, the rapid Ag tests detected 100% of Culture +ve PCR +ve specimens.

*Culture +ve on decent assay should be MINIMUM bar to consider ppl likely infectious

9/30

So HOW does this go so wrong?

How does a study get so badly distorted??

Simple, We must always DEFINE WHAT IS THE GOAL OF THE ASYMPTOMATIC TESTING!

In case of asymptomatic screening of ppl with no reason to think their infectious, it is to find CONTAGIOUS people.

10/30

The goal of Asymptomatic screening is NOT to medically diagnose ppl as having remnant non-infectious virus RNA.

NO! Leave that for medical testing.

Ppl w remnant RNA usually do NOT need to be isolated

**We MUST recognize PCR STAYS POSITIVE LONG AFTER INFECTIOUS**

11/30

If NO symptoms, you're testing for contagiousness.

We must stop thinking about tackling this pandemic w medicine but rather w PUBLIC HEALTH tools.

We defined the reasons for testing in @ScienceMagazine here w/ @K_G_Andersen

science.sciencemag.org/content/early/…

12/30

The @CDCMMWR study linked above failed to define why they were testing and thus got confused.

They said the tests missed 60% of the asymptomatics when in fact they caught 100% of the asymptomatic PCR +ves that they intended to detect!

The culture positive ones!

13/30

***I implore anyone studying rapid Ag tests to first DEFINE WHY YOU'RE TESTING, and from that, choose the metrics that make sense.

This is crucial.

If your unsure what your goals are, don't do the study or find someone to help. I'll help....

14/30

There've been many many rapid Ag test studies by ppl who haven't defined their goal. These unsurprisingly tend to ultimately come to a conlclusion that Rapid Ag tests don't work.

15/30

Meanwhile, studies by experts in this field have largely concluded these tests work well

The study that instigated this thread shows how WELL these tests work- though the data was interpreted with a slightly bent lens - discussed above.
cdc.gov/mmwr/volumes/6…

16/30

Here are a few tips on interpreting Rapid Ag Test studies

a) Define the goal. If goal is screening contagiousness, then study should put any rapid Ag tests in context of virus culturability OR attempt to measure Rapid Ag test sensitivity stratified on viral RNA load

17/30

b) Virus Culture +ve does not necessarily mean someone is infectious.

Rather, it should be considered IMIO as a minimum bar to entry that someone might be.

If not culture +ve, then from a public health lens, the person is most likely NOT a major public health threat

18/30

c) Yes, some virus culture systems are more sensitive than others. Of course. But most will do fine to get at this broad metric of whether to consider a sample to have enough live virus to consider likely contagious and a public health threat.

19/30

d) If evaluating against PCR Ct values - do NOT assume consistent Cts across labs.

Some can be wildly different from next!

Though much discussion about this - it is oddly frequently ignored when assessing Rapid Ag tests.

20/30

For ex, many studies have found a Ct value of ~30 generally reflects the edge of culturability

Thus Ct 30 is often used as useful Ct cutoff for rapid Ag tests to shoot for

This is OK as a guide, but can be way off

This MMWR Study is a decent example of Ct 30 cutoff

21/30

If the PCR test being used to validate rapid test is calibrated with a lower Ct value (for whatever reason) this can make a rapid Ag test look much worse than it is.

I believe a real world example is currently sowing confusion in Liverpool, UK

22/30

In most studies that defined Ct ~30 as useful target for culturability/Rapid Ag tests, median Ct values of Asymptomatics is usually ~28-31:

Here is a mishmash of Studies showing Asymptomatic Cts - note medians are all around 30

23/30

In these studies 👆, rapid Ag test worked well up to Ct of ~30, or even better.

This is important b/c a very public study in Liverpool is showing outlier results where rapid Ag test looks like it is performing poorly... but I think the Ct distribution may be at play.

24/30

This study in Liverpool, UK is finding that Rapid Ag tests are performing well only up to a Ct value of about 22.

This is way different than other assessments of this same test - which have shown it works to Ct ~30, as expected.

However...

25/30

Inspection of Ct value distributions in Liverpool suggest shift in PCR Ct values:

The median Ct of Asymptomatic screening is a HUGE OUTLIER w median Ct of only ~21!

This is Exceptionally LOW for random screen of Asymptomatics - as shown in the figure a few tweets up.

26/30

The Ct result is either from a very rare chance of finding most asymptomatic people right at the peak of their infectivity (unlikely given the narrow time frame for this)

OR

More reasonably, the PCR lab may have a readout shifted down by about ~5-10 Cts.

27/30

There is other evidence for this.

The lab in the study claims a Ct value of ~30 equals ~100 viral RNA particles/mL

In MOST studies/labs, a Ct value ~100 RNA particles/mL is usually a Ct value of ~40.

Here is the Liverpool study
assets.publishing.service.gov.uk/government/upl…

28/30

Thus, in the Liverpool study, it is likely that the rapid Ag test is performing as we expect, Just that the PCR Ct values are shifted down from what we're used to seeing.

The study SHOULD evaluate vs culture to know for sure.

29/30

To CONCLUDE - we have to be very very careful when evaluating Rapid Antigen Tests.

We must DEFINE the purpose of the test. Determine the best comparison test for evaluation based on that definition, and only then report out how well the test is working.

30/31

Anything short of that ends up running into major problems with these tests.

This paper here is one of the best papers in terms of methods to evaluate rapid tests. IMO.

https://twitter.com/michaelmina_lab/status/1327720372318441473?s=20