Tweet

Jon Deeks FMedSci

15 Jul, 9 tweets, 3 min read

This new study suggests LFTs in primary care have sensitivity of 95% and specificity of 89%, but it is FLAWED.

These results are misleading because of PARTIAL VERIFICATION BIAS

A quick lesson ….

1/9
sciencedirect.com/science/articl…

Participants were first tested with LFTs – 810 positive and 1736 negative. The investigators choose to test 217 of the 1736 negatives with PCR – that’s 1 in 8. This wasn’t a random sample as they were influenced by clinical characteristics as well as the test result.

2/9

The sensitivity / specificity calculation is based on all LFT+ves and 12.5% of LFT-ves as follows:

3/9

No correction made for sampling 100% of positive LFTs but only 12.5% of negative LFTs.

If it were a random sample the correction for sampling would multiply false negatives and true negatives by 8.

Sensitivity would be 72% (not 95%) and specificity 98.5% (not 89.1%).

4/9

BUT –

as it is not a random sample, we should not even do this (although it clearly shows the magnitude of bias that this could create).

We do not know whether the informative sampling would increase or decrease this bias.

5/9

https://twitter.com/deeksj/status/1373999012529987592

We previously saw the same error on a study from Wales back in March. When the authors were aware of the problem they appropriately deleted that component of their study from their report.

https://twitter.com/deeksj/status/1373999012529987592

6/9

The Royal Statistical Society Diagnostic Working Group report discusses these issues on pages 51-52. here

rss.org.uk/policy-campaig…

7/9

Sampling only a proportion of LFTs negatives can be used in an efficient design provided that the sample is representative and the sampling is corrected for in the analysis, as we did in the Birmingham study of Innova.

8/9

journals.plos.org/plosbiology/ar…

Studies where individuals choose to provide data after they have signed up for the study (e.g. current DHSC Innova studies where individuals getting PCR are sent home with an LFT, or where people using LFTs are asked to complete a home PCR) will be at risk of this bias too.

9/9

• • •

Missing some Tweet in this thread? You can try to force a refresh

This Thread may be Removed Anytime!

Twitter may remove this content at anytime! Save it as PDF for later use!

More from @deeksj

Jon Deeks FMedSci

@deeksj

7 Jul

@dhscgovuk

Even more data on LFTs out today.

@dhscgovuk released report of studies of Innova and Orient Gene, and their interpretation of findings.

Includes unpublished studies

BUT Clear evidence of post hoc interpretation of results based on naïve definition of infectiousness.

1/10

Long link is here:

gov.uk/government/pub…

2/10

@dhscgov

@dhscgov define

HIGH viral load as >1,000,000 RNA/ml and appear to consider that these are the only cases which matter.

10,000 to 1,000,000 is LOW (not moderate)

<10,000 MINIMAL.

This is despite acknowledging there is no cut-off that categorises people as infectious

3/10

Read 11 tweets

Jon Deeks FMedSci

@deeksj

22 Jun

@dhscgov

Results from the LIVERPOOL EVENT PILOTS have been published on line and in the media. Somehow I missed these coming out. cultureliverpool.co.uk/event-research…

No official report from @dhscgov as per normal.
Seems important evidence is being delayed once again.

1/7

The bottom line is that the events were safe.

Kudos to Liverpool PH Team.

But detail is interesting to see why they were safe.

2/7

First the infection rate in Liverpool was very low when the events were held

Negative LFTs required for entry. 5/13263 positive and excluded. Same-day PCR found 4 people positive who had attended with false negative LFTs. So 5/9 were picked up by LFT – 44% missed.

3/7

Read 7 tweets

Jon Deeks FMedSci

@deeksj

17 Jun

@educationgovuk

What do we known about ORIENT GENE used in the Daily Contact Testing Trial by the @educationgovuk and @DHSCgovuk?

There have been claims that this test is as good as others and has been reviewed by @MHRAgovuk for use in assisted testing. This is not right

1/10

The process does not make sense.

The MHRA never review products for assisted testing as they are professional use tests, which go through the self-certification process to get a CE-IVD mark.
MHRA doesn't go near this process.

2/n

In fact ORIENT GENE is not even on the MHRA register of products which is a requirement. You can check here - both for the product and manufacturer (sorry for the messy link).

3/n

aic.mhra.gov.uk/era/pdr.nsf/na…

Read 11 tweets

Jon Deeks FMedSci

@deeksj

17 Jun

@MHRAgovuk

IMPORTANT: MHRA New standards for LFT self tests for SARS-CoV-2 in people without symptoms.

@MHRAgovuk just published new Target Product Profile.

This is a guidance document based upon the best available evidence and independent expert opinion.

1/12

gov.uk/government/pub…

Important to note that the MHRA see these tests as for detecting “Current Infection”. That is detecting active infection, but is bigger than checking whether people are infectious. It also includes people who are pre-infectious.

2/12

They give guidance on reference standards to detect this. There are challenges here in establishing scientifically valid clinical reference standards. RT-PCR is regarded as acceptable which is what most are using.

3/12

Read 13 tweets

Jon Deeks FMedSci

@deeksj

29 Apr

The 1st 2 weeks of school testing did find MORE FALSE POSITIVES than TRUE POSITIVES - data are finally public.

Proportion false were 62% and 55% in these 2 weeks. Of 2304 positive tests, 1353 were likely false, with 1 positive per 6900 tests done.

@RoyalStatSoc

Remember that the law made kids, teachers, families and bubbles had to isolate for 10 days even when the PCR came back negative, despite the @RoyalStatSoc spending days trying to explain the issues to @DHSCgovuk.

Never ever say tests don't do harm.

To be clear as some still can't see the problem. 2 weeks testing in every school only found 954 true cases. To find each one used 6884 tests (cost £144,564) and 1.4 false negatives were put into unnecessary isolation with bubble and family- 500 person days if a bubble affected 36

Read 5 tweets

Jon Deeks FMedSci

@deeksj

28 Apr

Well here is some real data from 27 health care workers who picked up symptomatic Covid-19 infections whilst being tested twice weekly using RT-PCR. The paper develops models to look at impact of testing frequency, but also hypotheses on LFT performance
bmcmedicine.biomedcentral.com/articles/10.11…

There is excellent presentation of the original data in the paper, as well as results from the models. Kudos to the authors.

Plots of Ct values from PCR are provided, which show that RT-PCR with Ct<37 had a peak sensitivity for detecting infection of 77% 4 days after infection.

No LFTs were used in the study, but the paper considered test results with Ct<28 and Ct<25 as illustrations of how LFTs might perform. At Ct<28 the red spots are +ve LFT results, the black are -ve LFT results, peak sensitivity is 64% at 4.3 days post infection.

Read 5 tweets

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Share this page!

Jon Deeks FMedSci

Try unrolling a thread yourself!

More from @deeksj

Jon Deeks FMedSci

Jon Deeks FMedSci

Jon Deeks FMedSci

Jon Deeks FMedSci

Jon Deeks FMedSci

Jon Deeks FMedSci

Did Thread Reader help you today?

Like this author's thread?