Thread by @jamewils

How reliable are these indigenous Rapid Anti Body Tests with these sensitivity and specificity values?

Maths has a solid answer!

#thread

https://twitter.com/namita_kohli/status/1259829991186984966

How you will instill an interest in mathematics rather than fearing the subject?

Lots of that to do with your school teachers, how they teach you maths.....some makes it an interesting experience of approaching our everyday problems, but many make it a nightmare!

Epidemiology predictions and testings rely a lot on probability theories and differential equations.

So its characteristics can be easily understand if you have a solid mathematical foundation.

Clinical tests always use sensitivity and specificity to indicate its accuracy!

So let's first understand what both these terms means!

Sensitivity of a clinical test refers to the ability of the test to correctly identify those patients with the disease.

Specificity refers to the ability of the test to correctly identify those patients without the disease.

If a test mentions that it is having a sensitivity of 92% it means that, it can detect the 92% of the infected population from a set of population.

Many people got confused with this parameter.

Most struggle to explain complex maths in layman language.

I seen that when my techie friends here tried to enlighten you with the very dangers of relying fingerprint for Aadhaar as a unique feature.

So let me try to decipher this concept as simple as I can, of course, I briefed lawyers & politicians for many years, so I learnt the art!

Before that you should know 4 important terms, first learn two of that

TRUE POSITIVE: the patient has the disease and
the test is positive.

FALSE NEGATIVE: the patient has the disease
but the test is negative.

So a test's sensitivity return you these two results

So if you test 100 people ALL HAVING DISEASE and your test is having 92% sensitivity means it will identify 92 people as having disease while show 8 people who have disease as negative though they have the disease!

92 are true positive while 8 are called false negatives

So how the sensitivity is calculated - it is calculated as True Positive/(True Positive + False Negative) = 92/(92+8) = 92%

Most important thing many people did not follow is that is not a blind test, but you are testing 100 people with known disease not (disease + non-disease)!

Now learn two new terms too:

TRUE NEGATIVE: : the patient does not have the
disease and the test is negative

FALSE POSITIVE: the patient does not have the
disease but the test is positive.

Unlike we tested 100 diseased people before, here testing 100 people with NO DISEASE

A test claiming 97% specificity identifies here 97 people out of 100 who are not having disease or with a "NO" signal. But it wrongly identify that 3 people with an "YES" means they are wrongly identified with having a disease, though they are perfectly all right!

So they calculate Specificity = True Negative/ (True Negative + False Positives)

ie, Specificity = 97/(97+3) = 97%

But what is the problem with these numbers in real world?

In real world, you don't have all are infected with disease or you are disease free, in both cases, you don't need a test at all, because you know how they are!

So take the same number of people. You have 100 people but you don't know how many of them have disease or not!

Let's assume that in this population, 4% is having COVID, the same % COVIDS identified in India now.

So we want to see how much % of the positive results fall inaccurate and how much % of the negative results fall inaccurate!

Here maths comes to our help, Bayes' Theorem

Most of you learnt that in High School blindly just regurgitating what your maths teacher told you without understanding much of its practical applications in our real life!

This is that boring equation all of us byhearted blindly.

I don't want to teach you statistics or probability here!

I don't want to you make you boring like tech boys bored you when they flag how Nilekkanni's Aadhaar will be big tool of exclusion!

Even learnt judges of Supreme Court or many brightest in the bar did not get it!

That time you laugh at the them when they told you about false positive and negatives...

You thought that poor people who will deny ration will suffer because of this small % error.

Why should I worry?

That was a question of the subsistence of that poorest of poor, not urs!

U did not get when people like
@digitaldutta
@prasanna_s
@iam_anandv
flagged the very dangers of Aadhaar linked to everything especially it was linked to criminal records and rations and all and sundry.

@ShekharGupta painted them as Khan market & Manu Joseph derided them

Yes, I purposefully digressed here, because I want you to understand the concepts and when it is applied to a large population, how these terms false positives and false negatives decide life and death for many!

Ur curious here, because COVID not going to discriminate, thanks!

Now let's come to our problem, I will simply it for you to understand even if you are not a maths hate fellow!

We are testing 100 people, we assume that out of that 4 people have COVID.

Our goal is to find through the new RAT kit who is having COVID, it can be u or me!

Forget to tell you one more thing.

There is a fundamental difference between rRT-PCR test and Rapid Anti-body Test, which is a serological tests, as it is a blood test and look at serum - an amber-coloured, protein-rich liquid which separates out when blood coagulates.

So in rRT-PCR tests, a molecular tests, you look for the viral RNA of the virus - not making you confused, you look for the exact virus - means a positive test results means that you are having COVID

In RAT, you look for IgM & IgG antibodies, where positive means an diff meaning

It detects whether the IgM & IgG antibodies are developed in your blood stream to resist the attack of the virus and the positive here means, you can be certified as Corona resistant and some issue “immunity passports.” to u, so u can go back to work without fearing COVID!

There are enough materials available from reputed journals and institutions. Just Google, enough information is available, but always look who is providing the information and whether the source is credible or not.. That is it.

Now come back to our maths problem

As I take a small sample it returns fractions, so assume here I am testing 1000 people and 4% or 40 people out of that group has COVID.

Let us first consider all that 40 infected people tested with here. 92%, ie, 37 called as TRUE+VEs and 3 people tested negative, FALSE -VE

What about the rest of 960 people who were tested, as the specificity is 97%, your test returns 931 as TRUE NEGATIVE while 29 are labelled as FALSE POSITIVE.

So now look at the earlier tweet

Total positive identified - 37 (True +ve) + 29 (False -ve) = 66 instead of real 40!

U should be really re-read the tweets from here onward to understand the nuances!

What is the % of +ve tests were inaccurate here?

= False +VE/ Total Positive got through test

= 29/66 = 43.9%!

This is the error for a sensitivity of 92% when you have look for 4% infected!

Now look at how much % of -ve tests were inaccurate?

True Negative - 931
False Negative - 3
Total Negative identified - 934

% of -ve tests were inaccurate = 3/934 = 0.34%!

If you extend this to a larger population also this inaccuracy remain CONSTANT THROUGHOUT

What this means? The test who looks for antibodies can have an error factor of 43.9% when you look for them.

Sametime, if you are infected but not yet developed antibodies or not yet infected, its error value is a meagre 0.34%!

That is why its value is limited to research alone

Since false positives are high, you should do an rRT-PCR to confirm that whether you have infection or not.

Your negative result doesnot rule out that ur virus free, as it may take some days to develop antibodies, so you should in quarantine!

BUT ALL THIS COMES WITH A BIG RIDER

The results we discussed above is with a BIG ASSUMPTION THAT THERE ARE 4% INFECTED PEOPLE.

So we try to find out, if 4% are real infected, how much will be the error factor in positive and negative.

If you vary the assumed infection, the figures will also dramatically vary!

So you can vary the percentage of infected population, of course, an unknown, and play with Baye's theorem to find out how this error values will change!

Okay, I done that exercise for you, for this sensitivity and specificity of the indigenous RAT kit

Let me introduce another big IF here, the labs when they claim the sensitivity and specificity, they are testing it on a very less number of samples.

Therefore, these values should also be taken with a pinch of salt, till it is augmented with supplemental data

So before discussing the next two variables, kindly review what we discussed so far.

If sensitivity and specificity of a test is constant, and if your % infected people in a population is CONSTANT, population size WILL NOT INFLUENCE the error factor of the results at all!

In nut shell, if you suspect that 4% of your population is infected and you proceed with a test has 92% sensitivity & 97% of specificity, 44% of your positive results will be ERRONEOUS!

When infected population quantity changes, error also changes!

Now discuss other two terms!

Positive Predictive Value (PPV) and Negative Predictive value (NPV)

PPV gives an idea to clinicians ‘How likely is it that this patient has the disease (here antibody in case of PCR its virus) given that the test result is positive?’

PPV = True +VE /(True +VE + False +VE)

So in our case of 1000 people

True Positive - 37
False Positive - 29

There fore, PPV - 37/(37+29) = 56%

That means if ur test result indicate +VE, the person has a 56% chance to have anti-body.

The NPV of a test answers the question:

‘How likely is it that this patient does not have the disease given that the test result is negative?’

NPV = true negative/(true negative + false negative)

In our case

True Negative - 4656
False Negative - 3

NPV = 4656/(4656+3) = 99.7%

So along with sensitivity & specificity, PPV and NPV are given by manufactures of kits to understand the reliability index

Now I got you more confused with discussing all nuances - take your own time and read slowly.

In nutshell, you should understand that if the test claims 92% sensitivity and we target for 4% people, the person likely to have disease if test return a positive value is ONLY 56%!

Let me do a small tweak and change the sensitivity to 95% keeping rest constant, your PPV will be 57%

If sensitivity is 99%, PPV changes to 58%

if sensitivity is 99.9%, PPV remains at 59%

If sensitivity is 90%, PPV will be 56%

This is when we keep all other value constant

So in every test, they decide the sensitivity and specificity keeping in mind for a SPECIFIC TARGETTED PURPOSE in mind, RAT is for detecting anti-body with 50-55% confidence level.

It is a low cost test, so high NPV value ensures negative cases with high confidence

So if 50-56% range error is there in the positive value, you can go for a RT-PCR to confirm them after eliminating a major chunk of possible negatives, thus it saves a huge cost!

But the catch is if ur infected but not developed anti-bodies, the NEGATIVE did not ensure ur safe

Hence ICMR rightfully advised to follow up every positive result with a confirmatory rRT-PCR and whoever finds -ve with this test should be kept in quarantine to see that whether they are infected or not.

So each test parameters are tweaked for an intended purpose.

rRT-PCR is with high PPV value, because if rRT-PCR detects you are positive, that leaves a very narrow margin of error for positive.

But in RAT, is tweaked with high NPV value, where its purpose it entirely different.

What this teaches you with respect to unique soln of Aadhar?

When you try for a unique ID for different type of purposes which are very contradictory in nature in its very predictability of results, that is a sales product, a bogus one, sleight of the hand!

Yep, a sales man sells his product as universal solution!

I will expalin

Two purpose of Aadhaar - one is PDS ration, other is for ATM use - both places use finger print - for which any query return false negatives & false positives!

PDS - goal is to see that people should not excluded due to high degree of FALSE NEGATIVES due to error factor

ATM - goal is to avoid impersonation and see that not a false person should not access ur account with a wrong finger print - so ur goal is to reduce maximum FALSE POSITIVES

So if you are accessing finger print for different purposes it needs different criteria

More serious one is this Aadhar fingerprint data is used by the police to identify a person using a crime scene fingerprint - If it returns a false positive, and then it is ur will be harassed thorough the years long due process to prove that ur innocent.

So PDS & Crime confiict

Aadhaar cannot be used same time to access PDS and your bank account (AEPS) because of this conflict between false negatives and false positives. If you tweak this in favour of PDS, you compromise on financial transactions and vice versa.

A midway compromises both!

Even with 99.9% sensitivity and specificity of finger print data you will either deny PDS for lakhs and lakhs of people or you will compromise their financial dealings or this data is used for crime detection, innocents will be trapped in due process, big punishment!

Greedy Koramangala techies very well know about this, but they push it with all means - by coercive sales pitch, Nilekanni's aura, invest in all media & buy opinion makers, invest in legal think tanks who pens our policies for all governments - who invests in them?

Investors are the global fintechs, big pharmas, big vaccine lobbies - they identify the influencers like Nilekkani who can influence and swing the policy making decisions, who draw respect of our so called educated elite majority lack critical thinking & selfish!

If my time permits, I will write this as a blog post, where i will try to make it more cohesive rather than these random rumblings, as I digressed many times from the core topic. ;)

END OF THE RANTS.