Every Monday for the past 4 weeks, we have been sending our projections to the CDC.
So far, covid19-projections.com has been the most accurate model every week.
In last week's CDC projections, we projected 88,767 deaths on May 16. On May 16, the US reported 88,751 deaths
(1/7)
So far, covid19-projections.com has been the most accurate model every week.
In last week's CDC projections, we projected 88,767 deaths on May 16. On May 16, the US reported 88,751 deaths
(1/7)
I attached comparison charts of how our CDC projections stack up against the other CDC models.
I included both the absolute errors (above) and the percentage errors (below).
Note that we are the only model with <10% error for the past 3 weeks.
(2/7)
I included both the absolute errors (above) and the percentage errors (below).
Note that we are the only model with <10% error for the past 3 weeks.
(2/7)
Some additional notes:
- We are the only model to outperform the ensemble model, which is a combination of all the models
- Many models are unable to beat a simple baseline of using the previous week's average daily deaths to make all future projections
(3/7)
- We are the only model to outperform the ensemble model, which is a combination of all the models
- Many models are unable to beat a simple baseline of using the previous week's average daily deaths to make all future projections
(3/7)
The IHME model was the 8th or 9th best-performing model for each of the past four weeks, out of 9 eligible models.
Even after overhauling their methodology, IHME's projections from May 11 had a 40% error just 1 week out. We had a 0.2% error for the same week.
(4/7)
Even after overhauling their methodology, IHME's projections from May 11 had a 40% error just 1 week out. We had a 0.2% error for the same week.
(4/7)
I hope that the media can fact-check the models they cite as much as they fact-check our government officials.
Many people rely on the media for news, and I think it's important that the media uses accurate and reliable sources.
(5/7)
Many people rely on the media for news, and I think it's important that the media uses accurate and reliable sources.
(5/7)
I applaud all modelers for their hard work on this difficult task.
I hope we can hold modelers accountable for the projections they send to the CDC.
How can we expect our government to make accurate decisions when many of the models they rely on haven't been accurate?
(6/7)
I hope we can hold modelers accountable for the projections they send to the CDC.
How can we expect our government to make accurate decisions when many of the models they rely on haven't been accurate?
(6/7)
Lastly, this is my own evaluation. I welcome and encourage more independent evaluations. All of our projections are available online.
I will post evaluations of all state-by-state CDC projections in the next day or two.
All historical data from: github.com/reichlab/covid…
(7/7)
I will post evaluations of all state-by-state CDC projections in the next day or two.
All historical data from: github.com/reichlab/covid…
(7/7)
Below is a link to a visualization of all the projections. You can click on the "move forward" and "move backward" arrows to see our past projections as well. The numbers should match those in our evaluation.
reichlab.io/covid19-foreca…
reichlab.io/covid19-foreca…
