Thread by @thattai

In coming days, it's important for the public to know that experts looking at the same data can legitimately disagree.

E.g. Consider the debate about whether community transmission has begun. Due to Bayes' Theorem, the answer depends on what you already believe.

Thread.

1/9

Assume the ideal case: don't consider asymptomatic carriers, let test results be instant.

We test 1,000 symptomatic cases and all are negative. This means there's no community transmission right?

Wrong. To interpret the result, we need to make a bunch of assumptions.

2/9

First we need to define some level of C.T. to look for. In the high-incidence group of symptomatic int'l travellers & contacts, about 2% are positive (283 out of 15,701 tested).

Let's assume that incidence due to C.T. among all symptomatic people is a tenth of that: 0.2%.

3/9

Note that 0.2% incidence among all symptomatic people is a *very high* rate, which would overwhelm the healthcare system. Such a high value should be easy to detect, if it is occurring. If we can't detect it, that means we could easily miss lower values of C.T.

4/9

The qPCR test can also have high false positive and false negative rates (FPR, FNR). But with clinical evaluation and repeated tests, net FPR and FNR can be low. Let's assume FPR = FNR = 0.0002, which is *very low* (about 3 of the current 283 cases would be false positives).

5/9

So, we're looking for C.T. of q = 0.2%, at FPR = FNR = 0.002. Out of N symptomatic cases tested, all are negative.

Now, instead of asking "Is C.T. occurring?" we must ask "What is the chance that C.T. is occurring, given this data?" Bayes' Theorem allows us to do this.

6/9

But there's a catch: Bayes' Theorem requires us to feed in our *prior belief* that C.T. is occurring. Even if experts agree on all other parameters, they may disagree on this.

Each expert starts with some prior belief, and updates this depending on how many test negative.

7/9

No matter what someone's initial belief is that C.T. is occurring, as more negative results are seen, this degree of belief will decrease. Eventually, it will reach some low level (let's say 0.01 to be conservative) at which point we can declare "No C.T."

8/9

The graph at the top of the thread shows exactly this: for each possible initial degree of belief that C.T. is occurring, it shows how many negative tests it would take to convince someone that C.T. is *not* occurring.

Experts can legitimately disagree, given the same data.

9/9