Terrific assessment of projections of demand for Swedish ICU beds. The first two panels are model-based projections by academics; the third is a simple extrapolation by the public-health authority; the fourth is the actual outcome /1 dn.se/nyheter/vetens… 
tl;dr Model-based projections drastically exaggerated the actual demand – sometimes by more than an order of magnitude. Today the number of patients in intensive care is about 450; it never exceeded 600 /2 icuregswe.org/data--resultat…
Around the same time, if I read their data file correctly, the IHME projected a demand of 4400, with a 95% uncertainty interval of 1400–11000. The real number is therefore way outside the interval /3 healthdata.org/covid/data-dow…
From a philosophy of science perspective, this should not be surprising. Models work well when the underlying data-generating process is known and stable and when there has been ample time to calibrate the model. These conditions do not obtain here. /4
In addition, some of these models apparently contain over 100 parameters, and would be difficult to calibrate under any conditions /5 nyteknik.se/opinion/forska…
From a sociology of science perspective, we should expect few modellers to admit having made mistakes: based on @PTetlock's research we should expect claims to the effect that they were "almost right." So far I haven't seen one saying "we were wrong." (But I could be wrong!) /6
We should be tolerant of mistaken projections. These are incredibly difficult prediction tasks. The modellers here were trying to be useful, and they were working under great time pressure /7
We should be less tolerant of overconfidence in particular and a lack of epistemic humility in general. A true expert would have known ahead of time just how much uncertainty was involved in their forecasts and expressed themselves accordingly /8 behavioralscientist.org/epistemic-humi…
The people involved in these forecasts expressed themselves with *way* more confidence than what was justified at the time. This was an unforced error on their behalf /9
Scientist overconfidence is a massive problem. In the short run, it undercuts efforts to use science to inform policy; in the long run, it reduces trust in science in general. We can and should do better /10
PS. People interested in learning more about overconfidence among scientist-experts might want to check this piece out: dx.doi.org/10.1080/135017…
Draft (ungated) version here: ssrn.com/abstract=877205
