Thread by @jimbuctwit

TEST POSITIVITY - My view.

Disclaimer: I'm not an epidemiologist nor statistician. This is my understanding of the topic.

A simple internet search will yield a LOT of explanations with good graphics so you might consider going that route if I don’t provide what you need.
…

I am going to sacrifice brevity and density for clarity, so this could end up being a long thread. My apologies in advance. Someone who knows how might unroll this for us.

In my haste I won’t have diagrams or graphics so you’ll have to use your imagination at times.

…

This thread is obviously about the coronavirus pandemic so I won’t specifically refer to virus all that much. You know what I’m talking about.

To start, imagine a community of 1000 people in which 100 people are randomly infected. We’ll test a random sample of 100.
…

And don’t worry about transmission and incubation times and all that. For now this is just a group of people with an infection which they are not spreading to others. We’ll discuss the spreading of infection later.

Also don’t worry about false negatives and such.
…

In our test sample of 100, 10 people will test positive.
On day 2 let’s test 200. We get 20 positives. Yikes, the # of positives doubled in one day!
On day 3 we’re worried so we test 300. Oh boy, 30 positives.

This is an alarming pattern of new cases 10-20-30.
…

But you that know nothing has changed in the community, still just 100 of 1000 are infected. Increase in cases is simply a function of more testing.

If you divide each day’s new cases by the # of tests, they all represent 10%. 10/100 = 30/300.

That is what TPR is all about.
…

TPR is Test Positivity Rate.

TPR is the proportion of tests completed on a given day that are positive. In other words, the # of new cases divided by the # of tests performed.

It “equalizes” the day-to-day variability in the number of tests performed one day to the next.
…

In the real world cases are not evenly distributed around the community and the sample of people tested is far from random. In South Dakota most testing has been done on people with symptoms, a group that is likely to be positive.

This pushes TPR up.
…

If disease (dz) is stable or slowly spreading,TPR will stay about same but when dz spread amplifies, more people get sick & want a test. In setting of limited testing the proportion of tests done that are positive will naturally rise. This is a good way to monitor dz activity.
..

So that is the basics of TPR and if no big changes are made re who is allowed to get a test, and people with symptoms can get a test when needed then TPR is a good way to keep track of dz activity.

But what happens if testing strategy changes?
…

Over the summer DoH started doing “surveillance” testing - mass testing in groups that are expected to be negative.

They are monitoring groups, like healthcare workers or nursing home residents, to try to catch dz early & stop spread. This is good, and the more the better.
…

But this big influx of new tests that are expected to be negative inflates the denominator in the TPR calculation.

If TPR was typically 10% and suddenly the # of negative tests is doubled then TPR will drop to 5% implying dz activity has improved when it really hasn’t.
…

To acknowledge impact of surveillance testing in SD, DoH, in early August, started to separately report the # of test done on people for the FIRST TIME to distinguish those from the # of tests done repeatedly on the same person.

I’m guessing here, but I think that’s right.
…

So, now we have unique persons, first-time testers (FIRSTERS) and repeat testers (REPEATERS).

DoH reports the # of new cases each day. This # can be divided by the # of FIRSTERS to get 1 TPR, or the TOTAL number of tests (FIRSTERS + REPEATERS) to get a different TPR.
…

FIRSTERS are probably mostly people who are recently infected and symptomatic who are seeking a diagnosis. This is an infection-rich group.

The TPR using firster tests in the demoninator (I’ve been calling this person-TPR, or pTPR) surely OVERESTIMATES community prevalence.
…

FIRSTERS also includes those who might be required to get “cleared” for a surgery, travel, or some reason not based on concern they have infection (no symptoms, no close contact).

These could significantly dilute TPR in areas with a low amount of testing.
…

REPEATERS are tests done on anyone who’s been tested before, for whatever reason. Some employees might get tested every week. Some nursing home residents might get tested every month. Over and over, never really expecting result to be positive.
…

Some REPEATERS are like my mom. She had to go to hospital over the summer from her assisted living place. Tested NEG on admission, had to test NEG to go to rehab, tested NEG upon admit to rehab, surveillance tested NEG while there, and again NEG to go back to AL.
…

That’s 5 NEGS in a few weeks. Those negative results don’t really help define dz spread in the community and actually contribute to a false sense of low dz activity.

BUT…

As dz spread & prevalence in a community rises due to transmission in the community, repeat surveillance testing, esp among employees but also in NH staff and residents, etc. would show more positives in those groups. REPEATER positives would help define comm spread.
…

THE PROBLEM in the situation when a REPEATER tests positive is that when they do, their result adds to the new case COUNT but does NOT add to the FIRSTER DENOMINATOR because that person had already been counted, the first time they were tested.
…

To be clear, if you tested in the past and were negative you were counted as a first-time tester. Your test was included in the denominator for that day. Now, if you later test a second time and are positive your result is added to the new case count but not the denominator.
…

This makes the firster TPR, or pTPR, look worse than it really should. And this effect becomes more pronounced as disease activity in the community increases and more & more repeaters test positive. Despite that caveat, pTPR is a very important indicator of dz amplification.
…

TPR calculated with ALL results (firsters and repeaters) in the denominator is also important because repeat testing in individuals who do move about in the community does assess the community-at-large. But, ALL tests TPR does UNDERESTIMATE community prevalence.
…
a few more...

So,
pTPR overestimates dz prevalence and spread, &
ALL-tests TPR (aTPR) underestimates.

The real status of dz activity is somewhere between the 2 extremes.
Comparing pTPR in 1 location to aTPR in another is wrong.
Comparing pTPR to pTPR over time is fine. Same with aTPR.
…

Both measures of dz activity are vulnerable to inaccuracy when testing strategies are changed or intermittently blended. For example the adding in of results of mass testing that do not require qualifying symptoms could briefly throw results off a trend.
…

Ideally, DoH would report new positives discovered among FIRSTERS separately from those among REPEATERS, but at some point we need to step back and just look at the big picture. Both methods are showing an upward trend, meaning dz is spreading.
…

Summary:

Each method of calculating TPR has merits & faults.
Each is fully legitimate - 1 is not better than other.
It’s important to understand limitations.
Different reporting agencies & sites use different methods.
Be careful comparing 1 method to a different method.

This graph shows pTPR and all-tests TPR in SD. Ignore the data glitch spike on blue line. You ca see that both are increasing, and that is what is concerning. The faint red line marks 5%, the goal recommended to indicate adequate testing.

Finale:

That was a lot of words. In my mind I could imagine a lot of great graphics to show the concepts but I didn’t want to take the time to create them. Please LMK if you see errors or omissions. LMK if you have a question.

Remember: BIG PICTURE. Reality is in between.