1) This thread by @jposhaughnessy beautifully describes how important it is to draw the right inferences from your observations/available data.
I want to connect this to a topic I feel has not been talked about enough -- the role of probabilistic inference in Covid-19 testing.
I want to connect this to a topic I feel has not been talked about enough -- the role of probabilistic inference in Covid-19 testing.
2) Most people in the US today don't know if they have Covid-19 or not.
Some estimates say 3% of the US population has it. Others say 30% of the population has it.
Nobody really knows for sure.
Some estimates say 3% of the US population has it. Others say 30% of the population has it.
Nobody really knows for sure.
3) Of course, there are Covid-19 tests available.
But they're not perfect either.
Some estimates say they're only about 60% accurate. Others say they may be 80% accurate.
Nobody really knows for sure.
But they're not perfect either.
Some estimates say they're only about 60% accurate. Others say they may be 80% accurate.
Nobody really knows for sure.
4) So here's the question: if you test positive for Covid-19, what are the chances that you actually have it?
And if you test negative, what are the chances that you *don't* have it?
These are questions of inference: given a test result, how much can you infer from it?
And if you test negative, what are the chances that you *don't* have it?
These are questions of inference: given a test result, how much can you infer from it?
5) Let's lay down some assumptions.
Say 10% of the US population has Covid-19.
And say the tests are 75% accurate.
That is, if you have Covid-19, the test has a 75% chance of returning positive and a 25% chance of returning negative. And the reverse if you don't have Covid-19.
Say 10% of the US population has Covid-19.
And say the tests are 75% accurate.
That is, if you have Covid-19, the test has a 75% chance of returning positive and a 25% chance of returning negative. And the reverse if you don't have Covid-19.
6) Here's a matrix of possible outcomes: 
7) And here's a Venn diagram showing how the US population (roughly 300M people) would fall into these outcomes (at our 10% infection assumption):
8) There are ~300M people in the US.
We've assumed 10% infection; that's ~30M people with Covid-19.
But if all these people were tested, only ~22.5M will test positive because the test is only 75% accurate. The other ~7.5M will test negative even though they have Covid-19.
We've assumed 10% infection; that's ~30M people with Covid-19.
But if all these people were tested, only ~22.5M will test positive because the test is only 75% accurate. The other ~7.5M will test negative even though they have Covid-19.
9) Suppose you test positive. What are the chances that you actually have Covid-19?
Around 75%, right? After all, the test is 75% accurate.
Read on.
Around 75%, right? After all, the test is 75% accurate.
Read on.
11) Look at the Venn diagram again.
The diagram shows there are ~90M people who test positive. But only ~22.5M of them have Covid-19.
So, even if you test positive, your chances of having Covid-19 are only about 25% (22.5M/90M).
Not 75%.
Surprised? That's probability.
The diagram shows there are ~90M people who test positive. But only ~22.5M of them have Covid-19.
So, even if you test positive, your chances of having Covid-19 are only about 25% (22.5M/90M).
Not 75%.
Surprised? That's probability.
11) OK, what if you test negative? Then, how likely are you *not* to have Covid-19?
Again see the Venn diagram. Of the ~210M who test negative, about ~202.5M don't have Covid-19.
So, if you test negative, there's a 96.43% chance you don't have Covid-19.
Again see the Venn diagram. Of the ~210M who test negative, about ~202.5M don't have Covid-19.
So, if you test negative, there's a 96.43% chance you don't have Covid-19.
12) This means you can have so much more confidence in a negative test outcome than in a positive test outcome.
This is fairly well-known. Any introductory statistics textbook or course will cover it.
But you'd be surprised how many doctors don't know enough statistics.
This is fairly well-known. Any introductory statistics textbook or course will cover it.
But you'd be surprised how many doctors don't know enough statistics.
13) What if you do more than 1 test?
Intuitively, if you do many tests and they all agree, your confidence in the result should increase, right?
That's right.
Say you do 3 tests, and they all turn out positive. How likely are you to have Covid-19?
It turns out, about 75%.
Intuitively, if you do many tests and they all agree, your confidence in the result should increase, right?
That's right.
Say you do 3 tests, and they all turn out positive. How likely are you to have Covid-19?
It turns out, about 75%.
14) The table below shows how likely you are to have Covid-19, as more and more tests turn out positive.
At 1 test, your confidence is just 25%. But if you do 7 tests and they all turn out positive, your confidence increases to more than 99.5%.
At 1 test, your confidence is just 25%. But if you do 7 tests and they all turn out positive, your confidence increases to more than 99.5%.
15) And this other table shows how likely you are *not* to have Covid-19, as more and more tests turn out *negative*.
With just 1 test, you're already at 96.4% confidence. And at 3 tests, you get to more than 99.5% confidence.
With just 1 test, you're already at 96.4% confidence. And at 3 tests, you get to more than 99.5% confidence.
16) The key lesson here is: probabilistic inference is hard.
Our intuition doesn't always guide us well.
We have to systematically analyze the situation. Draw matrices and Venn diagrams. Run computer simulations.
Only then, we can have confidence in our inferences.
Our intuition doesn't always guide us well.
We have to systematically analyze the situation. Draw matrices and Venn diagrams. Run computer simulations.
Only then, we can have confidence in our inferences.
17) As always, there's a mathematical theory for this kind of probabilistic reasoning.
It's called Bayes' Rule.
If you're mathematically inclined, the Wikipedia article on it is pretty good:
en.wikipedia.org/wiki/Bayes%27_…
It's called Bayes' Rule.
If you're mathematically inclined, the Wikipedia article on it is pretty good:
en.wikipedia.org/wiki/Bayes%27_…
18) Congrats on reaching the end of another one of my long threads!
I'll leave you alone now.
/End
I'll leave you alone now.
/End
