16 Sep, 10 tweets, 3 min read
More iAbra statistical nonsense

Following from the Telegraph at the weekend, I have now seen the data from iAbra for their 20 second "Holographic microscopy" saliva test being trialled at Heathrow.

Start here in the FT, then see below.

1/10

ft.com/content/e7a279…
In FT I said:

“we are in a pandemic, people are dying from the disease, and a company decides that it is reasonable to mislead us all to make their test look like the best thing available."

2/10
"Legally, they can probably get away with this, but there cannot be any consideration that this is morally acceptable.”

They have said my characterisation was “incredibly unfair”.

See what you think.

3/10
The product claim made by iAbra says:

Virolens®, a new screening device for COVID-19, can deliver results in just 20 seconds with 99.8% sensitivity and 96.7% specificity.

4/10
Sensitivity first

99.8% sensitivity means that if you had a 1000 people with Covid, it is positive in 998 of the people. iABRA has a different take on what sensitivity is.

Their sum which gives 99.8% is 156751940/157000000.

So what are these huge numbers?

5/10
They say they are “counts” of “virons” averaged across 3 samples. I presume they are counts of virus particles.

156751940 is the number from their machine. 157000000 is the expected number from some computation based on the replication rate (no further info given).

6/10
So the 99.8% relates to a quantity (viron count) measured in 3 artificial samples as a percentage of the value that they expected to see.

This is not an estimate of sensitivity but an assessment of measurement accuracy of their test

7/10
Specificity is worse.

Actually they call is Specitivity, so spelling is a problem too.

96.7% specificity means that of your sample of a 1000 people who are being tested for COVID but have nothing or other things, it is correctly negative in 967 of the people.

8/10
iAbra has a different take on what specificity is.

Their sum which gives 96.7% is 13232/13681

Again it is a ratio of a virus particle count to an expected value (doesn’t say how many samples).

Again it is an assessment of the measurement accuracy of their test.

9/10
If none of the clever people at iAbra know about sensitivity and specificity then they should not be working in this field.

If they do know, they are throwing sand in our eyes, which is morally indefensible.

What do you think?

10/10

• • •

Missing some Tweet in this thread? You can try to force a refresh

This Thread may be Removed Anytime!

Twitter may remove this content at anytime! Save it as PDF for later use!

# More from @deeksj

22 Sep
BMJ Editorial @bmj_latest on why Moonshot is scientifically unsound.

with Tony Brookes and @AllysonPollock

has just been published here

bmj.com/cgi/content/fu…

#Moonshot @UoB_IAHR @unibirm_MDS

1/10
Moonshot plans uses point of care tests (POCT) approved for home use which don’t yet exist.

Proposals use computer modelling not empirical evidence. Understanding model assumptions is critical.

Widely claimed model is based on using LESS ACCURATE TESTS. This is WRONG.

2/10
The model assumes new POCT is positive in people who have INFECTIOUS Covid-19

and negative in both those who don’t have COVID-19 infection at all

and negative in those who have NON-INFECTIOUS Covid-19.

(infectious means you can pass the virus to somebody else).

3/10
18 Sep
What type of COVID-19 test do you think this is? @MHRAgovuk please have a look
Some antigens are proteins, so looks like if could be an antigen test, and a saliva one as well, and it is CE marked.

It's a dream ...
But it is actually an antibody test. Misleading?

(antibody tests have antigens (proteins) inside that detect the antibodies - so they are telling us what is inside it rather than what it detects).

Dream over.
11 Sep
Follow-on explanation from RSS statement

Fair questions about the actual figures used, and how this fits with repeated cheap testing as proposed by Michael Mina and others.

Key is that the Mina proposals (and Moonshot) only look at one side of the coin - the benefits

1/10
Moonshot does not consider the harms that mass testing will do through false positives.

In screening you always need to balance the benefits (the true positives) against the harms (mainly false positives)

So what determines how many false positives there will be?

2/10
Key figures that matter:

1)Prevalence of COVID
2)Specificity

An illustration - bear with my assumptions.

~3,000 cases in UK detected per day at the moment
National rate is 5 per 100,000

Double to allow for undetected asymptomatics –
Real rate of 10 per 100,000

3/10
7 Sep
Only evidence for accuracy of UK-Rapid Test Consortium Covid-19 AbC test is flawed

Over a week ago with @sarahboseley I asked for transparent reporting of RTC study that says test has 99.40% accuracy

But NOTHING has been made available or explained

1/16
theguardian.com/commentisfree/…
This is the totality of information about the study that is public - from Abingdon Health’s website - its three sentences.
abingdonhealth.com/uk-covid-19-ra…

But three sentences are enough to see the flaw.

2/16
Selecting only EuroImmun +ves biases sample to those with easy to detect antibodies. EuroImmun assay only has ~70% sensitivity - those with low or no antibodies are highly likely to have been excluded.

Then it's as easy as ABC to get a good estimate of sensitivity!

3/16
26 Aug
We have published our next Cochrane review of Covid-19 tests:

Rapid, point‐of‐care antigen and molecular‐based tests for diagnosis of SARS‐CoV‐2 infection - Dinnes, J - 2020 | Cochrane Library cochranelibrary.com/cdsr/doi/10.10…
Includes pre-prints and published studies available up to the end of May. We found 18 study cohorts with 3198 unique samples, of which 1775 had confirmed SARS‐CoV‐2 infection.

Due to the importance of these tests we currently updating to end of July and will republish in weeks.
Most of the data included here are from remnant sample studies, didn't use the tests according to their instructions, and were shockingly poor in other ways.

Hoping for better evidence in the update
21 Aug
BBC News - Coronavirus antibodies tests 'put public at risk' bbc.co.uk/news/health-53…

Our review of websales of COVID-19 antibody tests reported after @BBCNewsnight with @deb_cohen @charliehtweets last night. Joint work Uni Birmingham @UoB_IAHR @TERG_UoB and Uni Warwick.

1/11
Full report is available as a pre-print led by @siantphillips2

Information given by websites selling home self-sampling COVID-19 tests: An analysis of accuracy and completeness medrxiv.org/content/10.110…

2/11
A simple search of UK and US websites (end of May) found 27 websites selling 41 tests direct to user home self-sampling and testing for COVID-19. Seems a particular UK problem as 39 tests were for sale in UK and only 2 in US.

3/11