Thread by @MarkkuPeltonen

Recently there was a meta-analysis on the effects of masks conducted in Finland. A number of comments has been made about the quality of the piece, so I had a quick look at it. As the analysis was also mentioned at least in Sweden, few quick comments in English. 1/10

Background: the Finnish Ministry of Social Affairs and Health did a systematic review in May 2020 on the use of community face coverings to prevent the spread of Covid-19. There was no meta-analysis in the review, which focused on effectiveness. 2/10

The conclusion on that report was “very little research data available on the effectiveness of community face coverings in preventing the spread of COVID-19 in society.” and evidence “minor” or “non-existent”. 3/10

So, now then a formal meta-analysis, identifying the same 5 randomised controlled trials, showing an effect with relative risk estimate 0.61 (95% CI 0.39-0.96).
Few points: 4/10

The meta-analysis focuses on efficacy; what is achievable potentially when perfect conditions. They do something which they call “account of bias caused by non-compliance”; ie. if persons in the mask-group did not were masks they “adjust” for this. 5/10

To me, this sounds quite controversial: In my world we look at intention-to-treat first, and then perhaps maybe on the “per-protocol”/“as treated”.
Efficacy important, but this is now something different than what the original systematic review aimed at. 6/10

The problems of this accentuate in the Discussion, where the authors do not seem to understand the difference in efficacy and effectiveness, nor the fact that they are actually analysing something else than the original review, and making way too far-fetched conclusions. 7/10

There are other peculiarities, for example “Four of the analyzed studies evaluated the use of masks on respiratory infections directly, and in one the primary outcome was compliance with mask use.”. Hopefully an error, I don’t believe they actually mix the outcomes like this. 8/

. @jejkarppinen added the following comments after my initial post, which I agree with:
- The potential biases in the original papers were not covered.
- Quality of evidence was not evaluated at all.
- Dissemination of the results did not consider the potential problems. 9/10

Finally:
- I've not read the original 5 studies.
- I’m not an expert on systematic reviews/meta-analyses.
- I do think recommendation for masks is motivated, and the evidence is there (but not here..).
- I do think we should be objective when evaluating evidence. 10/10

The original systematic review the Finnish Ministry of Social Affairs and Health in Finnish is here (english abstract only):

julkaisut.valtioneuvosto.fi/bitstream/hand…

And the meta-analysis I’m commenting on is here (I don’t know if it is peer reviewed or not; does not change my point of view either way):

cdn.sanity.io/files/csynxgdb…

Ps. Somebody noted the lack of preregistered protocol, which reminded me that the PRISMA-guidelines helpful when reporting systematic reviews and meta-analyses.

Their checklist should be followed in reporting:

prisma-statement.org