The study looked at using the test on RNA extracted from swabs and saliva, and directly on swabs and saliva. Key results are here:
3/13
So false negative rates increase from 5% with RNA extracted from swabs (which looks good), to 21% with direct saliva (not so good). But the Government said that missing 21% is OK.
I really wonder what their criteria for good are. Let me know if you've seen then stated.
4/13
But there's a twist, linked to that word "spiked". Its not normal to put spiked samples in a clinical study. Not at all normal. Clinical studies are all about knowing how well a test works in people like us, not manufactuered samples
5/13
59 spiked samples were in the direct saliva group. Despite most of them having low (Ct>=33) viral loads, the test detected the virus in 91% of them. Of the real samples with similar viral loads virus was only detected in 13% of them. So spiked samples are different
6/13
I corresponded with the authors at the beginning of the week suggesting they presented results without these, as I didn't want to have to write more critical tweets or get quoted in the Guardian saying somebody had done something bad AGAIN.
7/13
I suggested it would have been much better just to report results from real people. But they haven't done it.
So PLEASE can somebody do a test evaluation study which I can tweet really nice things about? That part of my vocabulary is severely underused at the moment.
8/13
So I've removed the spiked results for you.
Removing them gives an overall sensitivity for direct saliva of 76% - so 24% of cases missed. Compared to 95% - so 5% missed with RNA extracted swabs - so 5 times as many get missed.
9/13
Now grouped by viral load
Sensitivity for direct saliva is
94% if Ct<25; 71% if 25<=Ct<33; 13% if Ct>=33,
compared to
100% if Ct<25, 95% if 25<=Ct<33; 74% if Ct>=33
for RNA extracted swabs.
10/13
Get the Appendicies and you will also see worrying variation in sensitivity between the centres for direct saliva
These data suggest a large (5 fold) loss of sensitivity using LAMP on direct saliva, and worrying substantial unexplained differences between centres.
And the test really can't detect when viral levels are low and misses over a quarter when viral levels are moderate
12/13
But I am still pondering how the Technical Validation Group thinks its OK to throw 59 spiked samples into a "clinical study". The DHSC also says there's nothing wrong in doing that.
That certainly wouldn't get past peer review at any reputable journal I deal with.
13/13
• • •
Missing some Tweet in this thread? You can try to
force a refresh
Mass testing in Liverpool MISSED ~50% of infections and ~30% with high viral loads. Results are in this Government document, but no actual numbers or details are given.
Absolutely URGENT that @DHSCgovuk reports full data today and HALTS IMPLEMENTATION
@BBCr4today invited me to speak about lateral flow tests this morning (can listen at 0650). All government plans are now about using tests to "release" people - this depends on reliability of negative result. This is what the manufacturer says:
1/16
The PHE Porton Down - Oxford University report included this graph showing sensitivity would be 58% when the test is delivered by a trained test-and-trace centre staff member. This is in symptomatics - not asymptomatics.
2/16
The Government's 77% sensitivity combines the other two groups - tests done by Porton Down lab staff, and tests done by NIHR Research Nurses - i.e. the experts and completely ignores the 58% group. That seems completely wrong.
There are a lot of data and studies reported - difficult to get your head round. Grateful to CI for talking to me this afternoon. He has worked at incredible speed to meet deadlines and says full report is forthcoming – there is more data and description to be added.
2/20
Phase 2 – spiked samples in controlled laboratory conditions
Phase 3a – samples from hospitals tested in controlled laboratory conditions
These do not tell us how well tests work in real world – important to do to move forward to what happens next, but pass by them now.
This is a speculative calculation informed by data available from Liverpool. The source is the Liverpool mayor - reported 23,170 tests done, with 0.7% positive (so about 162) here.
No sensitivity data for this test - @UKPHE has data but not public
I estimate sensitivity of 75% based on independent evaluations done for WHO of other similar LFIA tests (range from 50% to 90%) - but in symptomatic patients
and I presume was on Boris’s list of the tests last night.
We are updating our Cochrane review and have been through our searches. There are no pre-prints or published studies of this test.
There are 2 studies – 1 from the company Instructions for Use (IFU) and 1 from PHE Porton Down.
Test is made by Xiamen Biotime Biotech in China.
Quick Critical appraisal of PHE Porton Down study:
1)Is the study relevant? Can’t tell.
2)What do the results mean? Can’t tell.
3)Should we believe the results? Can’t tell.
(please give us the full report!!!)
2A Test 60 spiked saliva samples (n=15x4 dilutions)+71 -ve samples
2B Test against seasonal coronaviruses
3 Lab study of 1000 negatives and 200 positives sourced by Oxford University Hospitals. -ves fresh (<48hr) saliva samples +ves frozen
“Phase 3 findings will be reported to the Oversight Group, with DHSC and ministers using this information and any recommendations to inform potential purchasing decisions.”
Little detail is public – full results below - criteria for “pass” and “fail” for Phase 3 not stated