Thread by @GidMK

HAHAHAHAHA

No

This is a great example of problematic epidemiology reporting, so strap in and let's have some fun #epitwitter

https://twitter.com/MailOnline/status/1108284477128744960

First off, study is here. Tea and esophageal (throat) cancer: onlinelibrary.wiley.com/doi/abs/10.100…

Great read

It's a pretty cool study. Basically, researchers measured how hot people drank their tea, and then ~10 years later looked at the difference that hot tea made

They found that those who drank "very hot" tea - this was categorized as tea that was hotter than 60 degrees Celcius (140 Fahrenheit for US tweeps) - had an increased risk of cancer compared to those who drank <60 degree tea

How big was the risk?

Well, here's the first bit of fun

This is what the Daily Mail reported

A 90% increase! Sounds terrifying 😱😱😱😱

Looking at the study, this is broadly correct. The hazard ratio comparing very hot to low temp tea was ~1.9, meaning a 90% increased risk

But wait, let's think about this

Total population included in study = ~50,000

Total number of cancers = 317

Why might the 90% risk increase be misleading?

So the total number of cancers is 317, in a sample of 50,000 (or precisely, 49996)

That means that the crude rate of cancer in this sample is 0.6%

Now, we can't calculate the adjusted rates of cancer based on the info in the study. We'd have to run their regression model for that

But the crude rates will give us some information here

The crude rate in the cold tea group = 92/19,450 = 0.47%

In the hot tea group = 87/10,799 = 0.81%

The unadjusted relative risk ratio is ~1.70

But what about the ABSOLUTE risk difference?

Remember, relative risk ratio is p(1)/p(2) = 0.81/0.47 = 1.7 = 70% increased risk!

Absolute risk difference is p(1)-p(2) = 0.81-0.47 = 0.34 = 0.34% increased risk

THIS MAKES THE RESULTS MUCH LESS MEANINGFUL

(Yes, the answer above was that they used relative risk which is not a great measure when the risk of an event is small)

So, we've established that the increased risk of cancer was more like 0.34% from drinking very hot tea, every day, for 10 years

But that's not the only issue here!

Firstly, how did they collect tea heat preferences? It's not a variable you normally think about

The researchers got their subject to sit in a room, and gave them a cup of tea prepared at 75 degrees

They put another cup of tea in front of them with a thermometer in it, and at cooling intervals asked the participants to take a sip and see if it was their preferred temp

Now, I don't necessarily have a better way to do this - I mean, really, measuring how hot someone's tea is drunk? HARD - but this method has some clear elements of potential bias

However, this bias may not be that big an issue

Why is that?

This one is pretty easy

It's unlikely that the issues with tea heat measurement could've impacted group differences in cancer rates

This means that errors in grouping would've (probably) occurred equally across groups, and if anything biased the results towards the null

But there's another massive, glaring flaw that isn't touched on at all

Reverse causality

To think about this, let's rewind

The hot tea/cancer hypothesis goes like this:

hot tea = damage to throat = cancer

But these people were measured at adulthood, and then followed over time

The damage to their throat might have already happened

So rather than the above sequence, we might see

damage to throat = hot tea = cancer

Is this likely?

Well, the hot tea drinkers were also older, smoked more, used opium more often, less wealthy, and drank more at baseline

Given these confounders, is it likely that there was some reverse causality here?

(Forgot to answer the question above btw, it was all of the above - the effect applies equally to groups, probably a minimal impact, and biases towards the null anyway)

So, I'm just going to say it - yes, it's very hard in this situation to exclude reverse causality. There weren't any sensitivity analyses that really excluded it, so I think it's still pretty likely

This raises the last massive issue - residual confounding

Note how the authors describe their findings vs the Daily Mail

"Substantially strengthen the association" is NOT THE SAME AT ALL as "see their risk of gullet cancer rise by 90 percent"

(Also, lol, gullet cancer is a great name)

Why are the authors so cautious

Maybe it's because TEA DRINKING IS SUPER SOCIAL AND HARD TO UNPICK FROM CONFOUNDERS

It's in caps because GODDAM IT I SAY THIS EVERY WEEK

We can't exclude residual confounding. Maybe people drink tea differently in different social situations. Maybe hot tea is more often served with alcohol. Maybe a single measure of how hot people drink their tea isn't that great a predictor 10 years later when they get cancer

Maybe there are a million factors you can never measure, and never control for, and thus it's really hard to know if this is causal or not

Maybe

So the authors are cautious. They know that there's a decent chance that the heat of your tea has nothing to do with risk of esophageal cancer

But the newspaper needs a story

And so we get "hot tea causes cancer"

So, let's recap:

Large epi study showed 0.34% absolute increase in risk of cancer associated with drinking the hottest tea compared to cold tea, in large amounts, every day for 10 years

Reverse causality and residual confounding remain an issue

Do we trust these results?

Well, yes. The study was interesting, and well done

Do we think hot tea = cancer?