INNOVA test – time for critical appraisal of the report
What does the report say about the test and should we believe it?
1/20
ox.ac.uk/sites/files/ox…
What does the report say about the test and should we believe it?
1/20
ox.ac.uk/sites/files/ox…
There are a lot of data and studies reported - difficult to get your head round. Grateful to CI for talking to me this afternoon. He has worked at incredible speed to meet deadlines and says full report is forthcoming – there is more data and description to be added.
2/20
2/20
Phase 2 – spiked samples in controlled laboratory conditions
Phase 3a – samples from hospitals tested in controlled laboratory conditions
These do not tell us how well tests work in real world – important to do to move forward to what happens next, but pass by them now.
3/20
Phase 3a – samples from hospitals tested in controlled laboratory conditions
These do not tell us how well tests work in real world – important to do to move forward to what happens next, but pass by them now.
3/20
Phase 3b FALCON - relevant
Symptomatic and asymptomatic in real conditions. Dry Swab are test per instructions by research nurse at site. VTM data tested in controlled laboratory conditions – less relevant.
Only PCR+ so only sensitivity
4/20
condor-platform.org/condor_workstr…
Symptomatic and asymptomatic in real conditions. Dry Swab are test per instructions by research nurse at site. VTM data tested in controlled laboratory conditions – less relevant.
Only PCR+ so only sensitivity
4/20
condor-platform.org/condor_workstr…
Phase 4 - School studies - not relevant
Only PCR’d those who were Innova test positives, so can’t really be used beyond estimating failure rates.
But very worrying 16.8% failure rate in school 1 points to batch or storage problems with the test.
5/20
Only PCR’d those who were Innova test positives, so can’t really be used beyond estimating failure rates.
But very worrying 16.8% failure rate in school 1 points to batch or storage problems with the test.
5/20
Phase 4 – Army, PHE staff, hospital staff
All tested in controlled laboratory settings which will have minimised error rates. Report shows that training of tester makes a difference, these have got higher estimates of test specificity. Few cases to estimate sensitivity
6/20
All tested in controlled laboratory settings which will have minimised error rates. Report shows that training of tester makes a difference, these have got higher estimates of test specificity. Few cases to estimate sensitivity
6/20
Phase 4 – Testing centre – really important data here.
Will be mainly from symptomatic though as they are the people going to testing centres.
So FALCON and Testing centre are where the important data are.
7/20
Will be mainly from symptomatic though as they are the people going to testing centres.
So FALCON and Testing centre are where the important data are.
7/20
Results FALCON
Fig 3 sensitivity 73% (95% CI 65-80) for research nurses and 79% (73-84) PD lab technicians
Figure 2 show sensitivity 77% (72%-81%) (includes Phase 3a lab data)
Lower 62% (55%-69%) for Ct>25 (but avoid over interpret ing Ct – not standardised or precise)
8/20
Fig 3 sensitivity 73% (95% CI 65-80) for research nurses and 79% (73-84) PD lab technicians
Figure 2 show sensitivity 77% (72%-81%) (includes Phase 3a lab data)
Lower 62% (55%-69%) for Ct>25 (but avoid over interpret ing Ct – not standardised or precise)
8/20
Results - Test centre
“Self-trained” data in Fig 3 -CI described this person as experienced in taking swabs at test centre who followed instructions to use the LFIA – not untrained public – closest to a soldier here.
Sensitivity 58% (52% to 63%).
Quite a bit lower.
9/20
“Self-trained” data in Fig 3 -CI described this person as experienced in taking swabs at test centre who followed instructions to use the LFIA – not untrained public – closest to a soldier here.
Sensitivity 58% (52% to 63%).
Quite a bit lower.
9/20
Results – Test centre
Specificity was 99.4% (99.1% to 99.9%) in the testing centre with the trained swabber. I think it wrong to add in the schools as LFIA negatives were not verified. Other groups are all laboratory tested.
10/20
Specificity was 99.4% (99.1% to 99.9%) in the testing centre with the trained swabber. I think it wrong to add in the schools as LFIA negatives were not verified. Other groups are all laboratory tested.
10/20
Summary #1
Little matches intended use of
(1) asymptomatic
(2) locally trained health care worker
(3) verified as COVID or not.
Rely on symptomatics (most of FALCON and the Phase 4 Test Centre). Self-Trained Test Centre most closely matches, followed by the HCW data
11/20
Little matches intended use of
(1) asymptomatic
(2) locally trained health care worker
(3) verified as COVID or not.
Rely on symptomatics (most of FALCON and the Phase 4 Test Centre). Self-Trained Test Centre most closely matches, followed by the HCW data
11/20
Summary #2 Sensitivity
58% (52 to 63) from swabber in test centre
73% (65 to 80) from our research nurse in FALCON.
Difference is troubling
Between 1 in 2 and 1 in 4 current cases of Covid-19 will be missed.
Other tests are better. See finddx.org/covid-19/sarsc…
12/20
58% (52 to 63) from swabber in test centre
73% (65 to 80) from our research nurse in FALCON.
Difference is troubling
Between 1 in 2 and 1 in 4 current cases of Covid-19 will be missed.
Other tests are better. See finddx.org/covid-19/sarsc…
12/20
Summary #3
Those getting negative results need to know Covid risk is reduced, but they could still have Covid, and get Covid tomorrow or next week.
Harmful for them to think they are Covid free - especially if they now cuddle their granny.
13/20
Those getting negative results need to know Covid risk is reduced, but they could still have Covid, and get Covid tomorrow or next week.
Harmful for them to think they are Covid free - especially if they now cuddle their granny.
13/20
Summary #4 Specificity
Relatively consistent in field studies.
Test centre 99.4% (99.1% to 99.9%).
Hard to think you would get better, but prev problem remains
0.6% getting false positives means 6 in 1000, or 138 in 23000 – so many test positives could be false.
14/20
Relatively consistent in field studies.
Test centre 99.4% (99.1% to 99.9%).
Hard to think you would get better, but prev problem remains
0.6% getting false positives means 6 in 1000, or 138 in 23000 – so many test positives could be false.
14/20
Summary #5 Faint lines
CI remarked on “faint lines” needing further investigation, linked to false positives Soldiers in Liverpool are apparently using a “can’t call” category, which is proving more popular than the test positive category.
15/20
CI remarked on “faint lines” needing further investigation, linked to false positives Soldiers in Liverpool are apparently using a “can’t call” category, which is proving more popular than the test positive category.
15/20
Summary #6 Failure rates
That 16% failure rate in School 1 is a big concern. There was apparently no problem with process followed – it was the test not showing the control lines. No current explanation, but concerns that there could be bad batches of test.
16/20
That 16% failure rate in School 1 is a big concern. There was apparently no problem with process followed – it was the test not showing the control lines. No current explanation, but concerns that there could be bad batches of test.
16/20
Overall #1
Regardless of value of mass screening, is this really strong enough evidence to purchase this test?
We lack good data on implementation as intended – really important that Liverpool is doing this, but unclear of the process being used to verify results.
17/20
Regardless of value of mass screening, is this really strong enough evidence to purchase this test?
We lack good data on implementation as intended – really important that Liverpool is doing this, but unclear of the process being used to verify results.
17/20
Overall #2
Participants really need to be properly informed about the remaining risk of Covid in those who are test negative.
Very dangerous to hear interviews on the news saying that they were off to visit their granny in a care home now they know they are negative.
18/20
Participants really need to be properly informed about the remaining risk of Covid in those who are test negative.
Very dangerous to hear interviews on the news saying that they were off to visit their granny in a care home now they know they are negative.
18/20
Overall #3
And how on earth can we get to a safe “test-and-release” strategy with a test which can miss up to 1 in 2 cases?
IMHO ABSOLUTELY NO CHANCE!
Need to engage a magician not a statistician for that.
19/20
And how on earth can we get to a safe “test-and-release” strategy with a test which can miss up to 1 in 2 cases?
IMHO ABSOLUTELY NO CHANCE!
Need to engage a magician not a statistician for that.
19/20
19 tweets was enough!
20/20
20/20
Sorry error here - this FALCON study link is here and it is in hospitalised patients all with COVID-19
condor-platform.org/condor_workstr…
condor-platform.org/condor_workstr…
• • •
Missing some Tweet in this thread? You can try to
force a refresh
