Thread by @drg1985

There's widespread confusion over false results in both #COVID19 tests and #cervicalcheck. Medical testing can be deeply counterintuitive: A HIV test is 99.99% accurate, yet for most of us, a positive test is only 50% likely to imply HIV - is it clear why?

..confusion is understandable; first we need to understand three key ideas in testing. The first is SENSITIVITY: This is how likely a test is to correctly identify a disease if you have it. A sensitivity of 90%, for example, means that the test catches 9/10 cases (2/n)

..the second idea is SPEICIFICTY: How likely a test is to give you a clear if you don't have the condition. A specificity of 95% means, for example, that if you have a totally healthy group, the test will still tell you 5% have the disease when they don't (3/n)

..sensitivity and specificity are functions of the test itself. But there's a third, equally important factor: prevalence / incidence. How common the disease is, the likelihood you actually have the illness in question. Let's go back to HIV example to see how they interplay (4/n)

The HIV test is ~100% sensitive (captures virtually all true cases) & 99.99% specific. Incidence of HIV in non-IV drug users is 1/10000; if 10000 people get tested, one has it, flags positive. In the remaining 9999, there's one false positive. So what happens? (5/n)

..we're have 2 positive results, only 1 of which is positive. False positive rate: 1/2, or 50%. You can see it in the frequency tree here, but the critical lesson is this: false positive (and negative) rates are a function of prevalence too, & don't make sense in insolation (6/n)

..even with HIV, we can see this: In a high risk population (IV drug use) incidence is 150/10000. If 10000 get tested, 150 true positives, 1 false positive - your false positive rate falls to 0.66% - I've simulated false positive rate with incidence here for illustration (7/n)

Note that HIV test doesn't tend to have false negatives because sensitivity is ~100%. Most medical tests not this good: take a #COVID19 test - PCR Sensitivity 0.98, Specificity 0.9999 (estimated) - you can see false positives and negative rates vary markedly with incidence (8/n)

Those simulations, by the way, shows why the claim by COVID deniers / conspiracy theorists that cases are all false positives is so wrong: false negatives a bigger problem than phantom false positives. They are effectively denying COVID exists, despite the evidence (9/n)

...this also brings to mind confusion over #cervicalcheck - if we consider only LBC (the old school smear), this test has a sensitivity of 0.75 and specificity of 0.90, with a grouped prevalence of CIN2+ of 2%. If you imagine 1000 women tested, you'd expect the following: (10/n)

1000 women tested:
20 CIN2+, 15 detected (True +) , 5 missed (False -)
980 clear, 882 cleared (True -), 98 flagged (False +)

False positive rate: 98/(98+15) = 86.73%
False negative rate: 5/(5 + 882) = 0.56%

The chances of a false positive are >> than a missed case (11/n)

This by the way is a hugely life-saving test, false negatives are low but unavoidably non-zero- check out recent articles by @russellnoirin to see how much, especially with new HPV methods. Here's an old Irish times piece I did on it too for context (12/n) irishtimes.com/opinion/confus…

The crux with ALL testing is that it is impossible to create a perfect test, and false results are unavoidable, no matter how good a test you have. This is by the way why it is a really poor idea to just throw a barrage of tests at someone without good reason to do so! (13/n)

It's also vital to realise false positive and false negative are potentially misleading terms: they don't apply to tests themselves, they describe interplay of test and prevalence, and change with incidence. Personally I think they are terms which just cause more confusion (14/n)

Anyway, I hope this is somewhat useful when rates get thrown around. I've started doing visual explainers on Instagram and I just did one on this topic if it helps: instagram.com/p/CMIkAI6nLbm/ (15/15, n = 15)