Thread by @PWGTennant

Why do nutrition studies disagree about the effects of food on health?

New paper by @GeorgiaTomova in @AJCNutrition suggests it may be due to routine adjustment for total energy, which can produce misleading results if not used with care!
doi.org/10.1093/ajcn/n…
1/12 #EpiTwitter

Many nutrition studies are interested in substitution effects.

Substitution effects are the effect of SWAPPING a particular nutrient or food with one or more other nutrients or foods while keeping the total energy (or mass) the same.

doi.org/10.1093/ajcn/n…
2/12 #EpiTwitter

There are several approaches to estimating substitution effects.

To test their performance we simulated a compositional dataset with known causal effects (see figure).

We then examined how accurately each approach recovered the truth.

doi.org/10.1093/ajcn/n…
3/12 #EpiTwitter

Good news: all approaches were able to accurately estimate a well-defined substitution effect, i.e. the effect of replacing a single other nutrient or food with a single nutrient or food (e.g. sugar for protein or meat for fish).

doi.org/10.1093/ajcn/n…
4/12 #EpiTwitter

Not so good news: the most common approach* experienced bias with substitution effects involving several nutrients or foods (e.g. meat for cereal, dairy, and fish).

*the leave-one-out model, which controls for total energy intake

doi.org/10.1093/ajcn/n…
5/12 #EpiTwitter

Bad news: Many researchers INADVERTENTLY estimate substitution effects by controlling for total energy intake as a 'confounder' without realising that this can radically change what is being estimated.

doi.org/10.1093/ajcn/n…
6/12 #EpiTwitter

Suppose you are interested in estimating the 'effect' of meat on glucose.

You control for total energy, fruit & veg, alcohol, & other drinks.

You will now estimate the effect of meat INSTEAD OF all other uncontrolled sources of energy.

doi.org/10.1093/ajcn/n…
7/12 #EpiTwitter

Because most studies adjust for total energy and a range of other foods, most studies estimate different things.

This effect is not subtle. The sign can reverse depending on the substitution! This may partly explain the disagreement.

doi.org/10.1093/ajcn/n…
8/12 #EpiTwitter

Very bad news: Accurate substitution modelling assumes all variables are in the same units.

But it is most common for studies to examine food in grams/servings while adjusting for total energy in calories.

This. Breaks. Everything.

doi.org/10.1093/ajcn/n…
9/12 #EpiTwitter

Studies that examine foods in grams/servings while controlling for total energy in calories estimate VERY obscure quantities.

Our simulations showed these models could produce estimates with opposite signs to the truth 🤯😳

doi.org/10.1093/ajcn/n…
10/12 #EpiTwitter

In an accompanying editorial, @daniel_ibsen & @ChristinaDahm conduct a corroborating analysis in real data and confirm the same potential for these 'mixed unit' models to produce estimates with opposite signs to the truth!

doi.org/10.1093/ajcn/n…
11/12 #EpiTwitter

Devastating news: The use of these 'obscure' mixed unit models is extremely common in the literature.

What are the implications for our knowledge about the effects of food on health? I'll leave that for you, dear reader, to ponder!

doi.org/10.1093/ajcn/n…
12/12 #EpiTwitter

PS Some friendly coverage of the paper on the @UniversityLeeds website for those wanting a less technical description!

Thanks to @turinginst for funding @GeorgiaTomova!

leeds.ac.uk/news-science/n…