Thread by @kareem_carr

THINK LIKE A DATA SCIENTIST:

Are you frustrated with how organizations like the CDC and the WHO are handling the pandemic? Do you wish they did a better job of following the data?

If so, read on... 👇

One of the earliest lessons of the pandemic was covid outbreaks can get really bad really quickly. While the costs of over-responding are easy to predict like unnecessary financial losses and physical discomfort, the costs of under-responding are harder.

Some areas got away with relatively small outbreaks. Others experienced tremendous disruptions to their healthcare system and significant losses of life.

One solution to the asymmetry of over-responding vs under-responding is to bite the bullet. Knowingly over-responding to covid is a simple strategy that's easy to implement. It's a plan with very little uncertainty. A lot of governments did this.

Customizing our responses is a harder strategy. In this scenario, we actively learn from incoming data and try to make the best decisions. The goal is to minimize costs while avoiding large outbreaks. This is the mathematical equivalent of balancing a knife on its edge.

From what I can tell, America is trying it's best to follow this second strategy. As an algorithm designer myself, I can tell you that optimal decision making is hard. In my opinion, the demands of the American political environment make this task even harder.

The American public is very picky about the kinds of rules it's willing to follow. Here are a few "shoulds" and "shouldn'ts" I've picked up on...

The rules shouldn't be inconsistent from region to region. Why mask in Japan but not Florida? The rules shouldn't be radically inconsistent with things you've said in the past. Why weren't cloth masks good enough before but they'd good enough now?

The rules should be logical and simple to explain. The rules shouldn't change rapidly. The rules shouldn't be excessively cautious. The rules shouldn't fail catastrophically and result in a huge outbreak.

The rules should, as much as possible, be fair. They shouldn't ask more of some people than others because of who they are (race, gender, age, class, etc). The government shouldn't collect huge amounts of data on who people are, what they're doing and who they're doing it with.

The rules shouldn't be guesses based on prior knowledge about other diseases. They should be based on experiments and observations of covid itself. At this point, you might be wondering if these are reasonable things to ask for.

Fortunately, modern data science, statistics and AI have a lot to tell us about how reasonable these requests are. Unfortunately, what they have to say is all these things that annoy the public are very common features of good data-driven decision making.

The optimal set of rules isn't always easy to explain. They often change rapidly. Taken out of context, they can often seem contradictory. They're often customized to the exact situation and therefore ask different things of different people.

They often require lots of potentially invasive information. "Big data" is a catch phrase for a reason. On top of all that, not using prior knowledge is potentially a huge mistake. Your algorithm is forced to start from scratch.

In a theoretical context, this just means your algorithm requires more data and takes longer. In the context of the pandemic, it means losing lives in order to get more data in order to relearn things you already know.

Using data to make decisions is tough work. It's not as easy as they make it look on TV. It's not surprising that many of the organizations that use data best are billion dollar companies.

I'm not saying organizations like the CDC and the WHO are doing great. I'm saying you would probably like the experience of a magical AI that delivered perfectly personalized pandemic precautions to each citizen even less.

I don't want to leave you with the impression that the situation is hopeless. We can always do better. This thread is just a nerdy reminder that life is hard and people are doing their best.