Thread by @cremieuxrecueil

What happens when you provide students with subsidized or free meals?

Lots of studies have been published on this topic, but somehow the field hasn't reached a consensus.

Why?

Maybe because there's clearly publication bias. When accounting for it, effects fall towards zero:

If you just look at all the effect sizes in the literature, you might start seeing the issue.

Notice the long tail of positive results?

That tail shows up pretty much regardless of the details.

Universal or means-tested program? Outcome type? Meal type? Causal inference methodology?

Irrelevant: there's still a positivity bias.

That tail on its own is not evidence of publication bias.

But, when you plot it against the precision of the estimates, it becomes clear:

That tail is built off of problematic work.

Going beyond funnel-based evidence of bias, we can also see that there are excessively many studies just beyond common significance thresholds.

This isn't hugely important, but it does support the contention that the field has selectively published significant results.

Consistent with that as well, using p-value based corrections for publication bias in the form of selection models, the effect sizes with those are also minuscule.

Really, everything ends up small, and evidence for heterogeneity also came up limited.

Basically, the conclusion is that school meal programs don't do much in developed countries, at least for some of the harder outcomes used in the literature.

I noticed this a few months ago when I wrote about the benefits of running these programs.

My conclusion holds:

If you want to learn more, see sources, read my case for school lunches even though these outcome effects don't stick, etc., then just click through:

P.S. Effects are significant for some outcomes, but the size of the effects is really, really small.cremieux.xyz/p/tim-walz-goo…