Jon Deeks Profile picture
11 Sep, 10 tweets, 2 min read
Follow-on explanation from RSS statement

Fair questions about the actual figures used, and how this fits with repeated cheap testing as proposed by Michael Mina and others.

Key is that the Mina proposals (and Moonshot) only look at one side of the coin - the benefits

1/10
Moonshot does not consider the harms that mass testing will do through false positives.

In screening you always need to balance the benefits (the true positives) against the harms (mainly false positives)

So what determines how many false positives there will be?

2/10
Key figures that matter:

1)Prevalence of COVID
2)Specificity

An illustration - bear with my assumptions.

~3,000 cases in UK detected per day at the moment
National rate is 5 per 100,000

Double to allow for undetected asymptomatics –
Real rate of 10 per 100,000

3/10
Spreadsheet looks at numbers of TP and FP
for tests with 60%. 80% and (dream on) 100% sensitivity

and 99% specificity (so 1% false positives)

Highlighted row shows true and false positives when prevalence is 10 per 100,000

You can see sensitivity doesn't really matter

4/10
For every true case detected there will be between 100 and 170 false positives.

Testing 10 million a day there would be 100,000 false positives.

In a week, that would be 600,000 (we will rest on Sundays), the population of Sheffield (sorry Sheffield for picking on you).

5/10
What happens to these people? What happens to their close contacts?

These are the harmful consequences that are omitted from all thinking for Moonshot.

We could test them again with another PCR test, but we can’t really do 200,000 PCR tests a day at the moment anyway

6/10
And we will do it all go through it again next week, and the week after, and the week after ....

7/10
Reducing false positives (increasing specificity) is the only way to improve this. Halve false positives to 0.5% and the harms will halve.

8/10
SO Mina and Moonshot are right that tests with any sensitivity can catch disease and help, and the more times you use them the more they will catch

but they are wrong in ignoring the accumulating harms that will occur with every repeat of testing.

9/10
Establishing specificity of these tests when used in the community is essential to get a grip on how big the problem would be.

Not the values from the cleanest most professional laboratory setting, but when they are used at home.

Is only 1% false positive optimistic?

10/10

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Jon Deeks

Jon Deeks Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @deeksj

7 Sep
Only evidence for accuracy of UK-Rapid Test Consortium Covid-19 AbC test is flawed

Over a week ago with @sarahboseley I asked for transparent reporting of RTC study that says test has 99.40% accuracy

But NOTHING has been made available or explained

1/16
theguardian.com/commentisfree/…
This is the totality of information about the study that is public - from Abingdon Health’s website - its three sentences.
abingdonhealth.com/uk-covid-19-ra…

But three sentences are enough to see the flaw.

2/16
Selecting only EuroImmun +ves biases sample to those with easy to detect antibodies. EuroImmun assay only has ~70% sensitivity - those with low or no antibodies are highly likely to have been excluded.

Then it's as easy as ABC to get a good estimate of sensitivity!

3/16
Read 16 tweets
9 Aug
AGREE - very important that we clearly distinguish between different types of studies. I discussed this at House of Lords select COVID-19 investigation (below - Q52 9th June) and have seen similar confusion across figures for many tests
1/10
committees.parliament.uk/work/293/the-s…
Not so clear this is due to profession, but "pathway" of studies exist - vary on what the participants/samples are, what the target condition is (and ref std), what RQ is, and what conclusions can be drawn. Def need clarified terminology to distinguish.

2/10
Lab people want tests which are likely to give the correct result for the sample that they have been presented with, and want numbers which tell them that. PH and clinicians want to know how well the test will work in the person/patient in front of them. Which rarely is as well
Read 10 tweets
25 Jun
Antibody tests for identification of current and past infection with SARS‐CoV‐2 - Deeks, JJ - 2020 | Cochrane Library cochranelibrary.com/cdsr/doi/10.10…

Living Cochrane Review of Antibody Tests – 1st version now published. Twitter summary to save you reading the 300+ pages
#1 Search date of 27 April 2020. Scanned 10,965 COVID-19 publications and pre-prints. Included 54 studies of test accuracy of COVID-19 antibody tests. Some studies looked at multiple tests, so total of 89 test evaluations including 15,976 samples (8,256 from COVID-19 patients)
#2 Most research studies done in China on tests developed and manufactured in China. Data found for 27 commercial tests and 25 in-house assays. 61% were lab assays (requiring venous blood), 39% with point-of-care assays (possible finger-prick blood)
Read 22 tweets
28 May
TRACK-AND-TRACE: MANY PCR-ves WILL HAVE COVID-19

Government advice (gov.uk/guidance/nhs-t…)

“If your test is negative, you and other household members no longer need to self-isolate”

BUT chances of COVID could be 19% in this group
@petermbenglish @drhelmac @PHE_UK @drhelmac
DATA (*more needed*)

Of staff in UK hospital 43% with week off for COVID symptoms were RT-PCR+ve.

Sensitivity of RT-PCR - no clear data - stated to be as low as 70%.

Investigate worst case 43%, compare with 24% best case
Sens: 70% to 85%
Spec: 98% (makes little difference).
TECHNICAL STUFF #1 – NEGATIVE LIKELIHOOD RATIOS (LR-)

LR- = (100-sens)/spec
Read 7 tweets
16 May
21 WAYS TO SPIN RESULTS FROM A COVID DIAGNOSTIC TEST ACCURACY (morning thoughts whilst walking the dog - please add others I've missed). I'm putting this out to help us critique the "number theatre" being thrown at us.
#diagnostics #COVID19 @deb_cohen @MHRAgovuk @drhelmac
1/22
#1 Use a small sample size (and never report how big it isn’t). Use phrases like"we detected all cases" the test is "100% accurate" without mentioning it is only from 29 samples.
2/22
#2 Talk about the total sample size when interpreting results about sensitivity (especially if you have tested 5000 blood donors to estimate specificity and only 29 to estimate sensitivity)
3/22
Read 23 tweets
14 May
1/9 Where’s the evidence for the ROCHE test claims?

All I can find is the IFU document. fda.gov/media/137605/d…

@deb_cohen @phe_uk @MHRAgovuk @sarahboseley I'll summarise what I have found. Please share other studies and results if you can find them. We've searched hard.
2/9 Study design

“A total of 204 samples from 69 symptomatic patients with a PCR confirmed SARS‑CoV‑2 infection were tested with the Elecsys Anti‑SARS‑CoV‑2 assay. 1 or more consecutive specimens from these patients were collected after PCR confirmation at various time points.”
3/9 (that's all the detail available - standard two sentences of detail - STARD reporting guidelines clearly not applied).
Read 9 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!