Thread by @federicolois

1/n Epidemiological models are underdetermined. What @pjakma is doing is showing exactly that. You have to fight underdetermination with tooth and nails. I have wrote about this before, but as the video shows with that much variability it is pointless. Are there alternatives?

https://twitter.com/pjakma/status/1382264335624499203

2/n Yes. There are 2 types of models, exploratory and predictive. Usually your model with fit one category, it is very unlikely your will fit in both. Fitted models help you in the predictive realm, but the key point is to realize that under-determination is a thing.

3/n That forces you to have to diminish the amount of moving parts, and try to remove as many high-dimensional stochastic inputs as possible through averaging. That's the approach we took on our work.

4/n But again, you need to decide on viral parameters anyway. The key is that fitted models have to be able to 'fit' the behavior of several places at the same time. That forces the parameter space to become very constrained.

5/n If 2 different places with different stochastic noise have the same fit under a group of parameters, then those parameters are better candidates than before. When you can describe 6, it becomes stronger. It may not be right, but it will be better than the alternative.

6/n But even under those conditions (and with only 2 free parameters, and one implicit) you can still fit different behaviors. This is where fitting is not enough and you have to understand under-determination.

7/n This is where things become nasty and where most epidemiological models just fail. Let's use Madrid as an example. Any of those fittings are acceptable, but only 1 describe what happened. So what we have is not good enough.

8/n Adding extra outputs to your model that follow a different 'rule-set' is what makes possible to fight under-determination. For example in ours we found that age seroprevalence ratio allow us to constrain even more the behavior.

9/n Why this is important? Because it allows you to know for example that Madrid has an entirely different viral spread behavior than Stockholm. And the reason why is that, is because adding the constrain to match the measured value will fight under-determination.

10/n But then again, what does that mean? It mean that even if you do a forecast with unknown conditions with data from July 4th of 2020, you can still derive a very likely behavior which is resilient even under complicated conditions like viral mutation.

11/n What were those predictions you might think. Here there are. On the right, that was the deaths per million that the model has saw, then the predicted and the last week numbers.

12/n What I am saying is that: "All models are wrong, but some are useful". If you need to understand if a model is reasonable the first thing to look at is if it is under-determined as @pjakma did. That was a beautiful display of where 'experts' usually get it wrong.