Jon Deeks Profile picture
4 Dec, 13 tweets, 3 min read
So the next new Covid-19 testing technology to look at is the OPTIGENE LAMP test - as covered here in the Guardian.

1/13

Experts question claimed accuracy of Covid-19 saliva tests theguardian.com/world/2020/dec…
The main body of the report is here, appendicies with data tables can be obtained by email

2/13

gov.uk/government/pub…
The study looked at using the test on RNA extracted from swabs and saliva, and directly on swabs and saliva. Key results are here:

3/13 Image
So false negative rates increase from 5% with RNA extracted from swabs (which looks good), to 21% with direct saliva (not so good). But the Government said that missing 21% is OK.

I really wonder what their criteria for good are. Let me know if you've seen then stated.

4/13
But there's a twist, linked to that word "spiked". Its not normal to put spiked samples in a clinical study. Not at all normal. Clinical studies are all about knowing how well a test works in people like us, not manufactuered samples

5/13
59 spiked samples were in the direct saliva group. Despite most of them having low (Ct>=33) viral loads, the test detected the virus in 91% of them. Of the real samples with similar viral loads virus was only detected in 13% of them. So spiked samples are different

6/13
I corresponded with the authors at the beginning of the week suggesting they presented results without these, as I didn't want to have to write more critical tweets or get quoted in the Guardian saying somebody had done something bad AGAIN.

7/13
I suggested it would have been much better just to report results from real people. But they haven't done it.

So PLEASE can somebody do a test evaluation study which I can tweet really nice things about? That part of my vocabulary is severely underused at the moment.

8/13
So I've removed the spiked results for you.

Removing them gives an overall sensitivity for direct saliva of 76% - so 24% of cases missed. Compared to 95% - so 5% missed with RNA extracted swabs - so 5 times as many get missed.

9/13
Now grouped by viral load

Sensitivity for direct saliva is
94% if Ct<25; 71% if 25<=Ct<33; 13% if Ct>=33,

compared to
100% if Ct<25, 95% if 25<=Ct<33; 74% if Ct>=33
for RNA extracted swabs.

10/13
Get the Appendicies and you will also see worrying variation in sensitivity between the centres for direct saliva

Birmingham 100% (n=16)
Southampton 87% (n=76)
Hampshire 84% (n=19)
Lighthouse lab 54% (n=41)
Manchester 47% (n=15)

11/13
These data suggest a large (5 fold) loss of sensitivity using LAMP on direct saliva, and worrying substantial unexplained differences between centres.

And the test really can't detect when viral levels are low and misses over a quarter when viral levels are moderate

12/13
But I am still pondering how the Technical Validation Group thinks its OK to throw 59 spiked samples into a "clinical study". The DHSC also says there's nothing wrong in doing that.

That certainly wouldn't get past peer review at any reputable journal I deal with.

13/13

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Jon Deeks

Jon Deeks Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @deeksj

3 Dec
Mass testing in Liverpool MISSED ~50% of infections and ~30% with high viral loads. Results are in this Government document, but no actual numbers or details are given.

Absolutely URGENT that @DHSCgovuk reports full data today and HALTS IMPLEMENTATION

gov.uk/government/pub…
This test is NOT FIT FOR PURPOSE for the Government's plans

It is totally unsafe to use these tests to decide somebody does not have COVID nor “infectious”.

If it were a drug surely this would warrant an immediate withdrawal from use.
Missing 30% with high viral loads is NOT SAFE. PHE studies said missed <5% - so this is more than 6 times as many.

You cannot risk people with high viral loads visiting their elderly relatives.
Read 5 tweets
24 Nov
@BBCr4today invited me to speak about lateral flow tests this morning (can listen at 0650). All government plans are now about using tests to "release" people - this depends on reliability of negative result. This is what the manufacturer says:

1/16
The PHE Porton Down - Oxford University report included this graph showing sensitivity would be 58% when the test is delivered by a trained test-and-trace centre staff member. This is in symptomatics - not asymptomatics.

2/16
The Government's 77% sensitivity combines the other two groups - tests done by Porton Down lab staff, and tests done by NIHR Research Nurses - i.e. the experts and completely ignores the 58% group. That seems completely wrong.

3/16
Read 16 tweets
11 Nov
INNOVA test – time for critical appraisal of the report

What does the report say about the test and should we believe it?

1/20

ox.ac.uk/sites/files/ox…
There are a lot of data and studies reported - difficult to get your head round. Grateful to CI for talking to me this afternoon. He has worked at incredible speed to meet deadlines and says full report is forthcoming – there is more data and description to be added.

2/20
Phase 2 – spiked samples in controlled laboratory conditions

Phase 3a – samples from hospitals tested in controlled laboratory conditions

These do not tell us how well tests work in real world – important to do to move forward to what happens next, but pass by them now.

3/20
Read 21 tweets
10 Nov
How well is the Innova test working in Liverpool?

This is a speculative calculation informed by data available from Liverpool. The source is the Liverpool mayor - reported 23,170 tests done, with 0.7% positive (so about 162) here.

liverpoolecho.co.uk/news/liverpool…
Presume specificity 99.6%

No sensitivity data for this test - @UKPHE has data but not public

I estimate sensitivity of 75% based on independent evaluations done for WHO of other similar LFIA tests (range from 50% to 90%) - but in symptomatic patients

finddx.org/covid-19/sarsc…
Using a prevalence of 400 per 100,000 you get the following 2x2 table:
Read 5 tweets
1 Nov
How well will these 15-minute rapid tests Boris mentioned work? (quick answer – we don’t know)

The Innova test was mentioned this week in Telegraph.
telegraph.co.uk/news/2020/10/1…

and I presume was on Boris’s list of the tests last night.
We are updating our Cochrane review and have been through our searches. There are no pre-prints or published studies of this test.

There are 2 studies – 1 from the company Instructions for Use (IFU) and 1 from PHE Porton Down.

Test is made by Xiamen Biotime Biotech in China.
Quick Critical appraisal of PHE Porton Down study:
1)Is the study relevant? Can’t tell.
2)What do the results mean? Can’t tell.
3)Should we believe the results? Can’t tell.
(please give us the full report!!!)
Read 11 tweets
1 Nov
The PM claimed that they have validated these new 15min antigen tests – what is the @dhsc @phe validation process?

15 Aug ministers commissioned new process at PHE Porton Down in collaboration with Oxford University. Protocol here (updated 23 Oct).

gov.uk/government/pub…
3 phases:

1 document review

2A Test 60 spiked saliva samples (n=15x4 dilutions)+71 -ve samples

2B Test against seasonal coronaviruses

3 Lab study of 1000 negatives and 200 positives sourced by Oxford University Hospitals. -ves fresh (<48hr) saliva samples +ves frozen
“Phase 3 findings will be reported to the Oversight Group, with DHSC and ministers using this information and any recommendations to inform potential purchasing decisions.”

Little detail is public – full results below - criteria for “pass” and “fail” for Phase 3 not stated
Read 6 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!