Thread by @DavidSteadson

All datasets have strengths and weaknesses that as an analyst you need to understand and acknowledge. I'm regularly having to point out the methodological challenges with Excess Mortality data, especially as it relates to Sweden, so a brief explanation in this thread.
>

There are multiple ways to calculate excess mortality. The simplest is simply looking at the average over a period - commonly the last 5 years - and then comparing deaths now to that average, and looking at the difference. A key problem with that is mortality trends.
>

If we look at Sweden for example, prior to the pandemic mortality was projected to continue a more than three decade long trend of decreasing mortality. Fewer and fewer people were dying every year. Note: In the graph below, 2020 and up are projections, prior to that historical
>

If we look at Germany, mortality has been *increasing* since 2004 and this was projected to continue for the foreseeable future.

Source for both these graphs is Macrotrends, using UN Population projections from 2020.
>
macrotrends.net/countries/SWE/…

macrotrends.net/countries/DEU/…

If you ignore the trend and take a simple average of deaths from 2015-2019 and project this as your deaths for 2020 (and beyond), then you will *underestimate* the expected deaths in Germany and *overestimate* the deaths in Sweden.
>

If you then take actual deaths and subtract the projected deaths to come up with a measure of "excess mortality" then you will *overestimate* excess mortality in Germany and *underestimate* excess mortality in Sweden.
>

So German excess mortality will calculate higher than "reality" and Swedish excess mortality will calculate lower than "reality". If you then compare the two, well your "difference" will make it look like Germany has done relatively much worse than Sweden.
>

The (few) experts in this field understand this and there are various methods to account for it, such as incorporating the mortality projections, adjusting for changing population demographics etc.
>

Lately I've noticed yet another "trick". People are comparing including the pre-pandemic months of Jan-Mar 2020 in their calculations - when Swedish mortality was following the trend and at an all time low, artificially decreasing "pandemic excess mortality" even further.
>

Alas, there's been an epidemic of non-experts taking the raw data and *not* considering issues like this and confidentially publishing their "excess mortality" calculations - sadly, not just on Social Media, but also in media and occasionally scientific journals.
>

But this isn't the only issue when doing international comparisons. Not every country measures things the same way when they report the data, and unfortunately organisations like Eurostat - a common source for European mortality data - don't make this easy to find.
>

On Eurostat for example, you can find a great data browser with access to the raw data and even nice maps presenting European Excess Mortality by month.
>
ec.europa.eu/eurostat/datab…

But how was the data collected? Where was it from? Any quirks? You need to scroll to the top of that page and look at "Explanatory texts"
>

There you will find, in a long page of small text, the below. The data is based on their data collection of weekly deaths.
>

Scroll further down and you'll find a section on "Accuracy" where they note there may be under-reporting and they refer you to another file on the Weekly Death Statistics.
>

If we go there, we find for example this table that shows quite a large variance between countries on how quickly the data is updated. A lot of countries are not even close to updated for the most recent couple of weeks. Montengro (ME) takes months!
>
ec.europa.eu/eurostat/cache…

Scroll down further and we get to "Coherence and comparability" and we see there are some different definitions used, with UK and Armenia for example, reporting registration date rather than date of death.
>

And then there is Hungary, Latvia, and Sweden that have deaths with an "unknown" date **and this data is only available in the raw excel files**. It's *not* in the weekly data set which means it's not in the monthly data set. Since 2013 however, only Sweden continues to do this
>

If you access the Eurostat data via their database browser directly, you get a further warning about this - a note "important information".

The warnings are there ... but who (apart from me!) reads them?
>

ec.europa.eu/eurostat/data/…

If you follow the links to the raw data you can find out exactly how many deaths this is. I've graphed them by year below. As of wk 50 in 2022 there were 3500 deaths recorded without a date of death. These are *not* in the Eurostat Weekly and Monthly data.
>

If we head to the Swedish Bureau of Statistics, their preliminary 2022 mortality statistics to Dec 30 reports 90213 deaths with a date and 3500 with an unknown date. The total number is nearly *4% higher* than the number typically reported from Eurostat.
scb.se/en/finding-sta…
>

Now, this doesn't mean excess mortality calculations are out by 4%, because we're comparing to a 2015-2019 baseline - but as you can see in the graph above, 2022 unreported deaths are nearly double that of the baseline years.
>

So how serious a problem is it? Well, take this article below, that spread widely last year claiming Sweden and Denmark had similarly excess mortality in 2020 and 2021. Specifically they say Sweden had an excess of 6000 and Denmark 3000. Over two years.
>
dailysceptic.org/2022/07/09/exc…

So that's about the same, per-capita. But if you look at their graph, you'll notice they're looking at monthly data. Sweden had somewhere around 6000 deaths in that period with no date of death. Did they include them? If so, how? I don't know, they published no methodology.
>

The fact that, as best I can tell, neither of the two authors appear to have any expertise in the field, with no relevant published papers, makes it quite likely they aren't aware of this "hidden" nuance of the Swedish data. If so, they fully missed *half* of Swedish deaths.
>

But even if they *did* incorporate this somehow (I haven't down the calculations to check) buy for example distributing over the year based on trends with the registered death dates) it does show you how serious they problem is for those *who don't* account for this.
>

And I can guarantee you nearly everyone I've challenged on this *doesn't* incorporate it. I ask them how they handle the week 99 data and they don't know what I'm talking about.
>

https://twitter.com/DavidSteadson/status/1612742974365917185?s=20&t=ALD2eC5VOcF4-o9a9Eb-ug

And if you don't incorporate it? Depending on methodology you might be underestimating Swedish excess mortality by up to **100%**

So that's the end of this TED talk. Moral of the story - understand your data - and make sure the people you are listening to understand theirs.
/end

(edit: half of Swedish excess deaths)

(Edit: ugh, lots of typos in that one. done/by/the) Sorry! Editt button for all tweets please @elonmusk !

(Edit: confidently, not confidentially)