Thread by @ebtapper

How to read a cost-effectiveness paper

This is a powerful method. But poorly understood, often maligned. My goal is to improve critical appraisal and help good analyses get the appreciation they deserve

🧵#MedTwitter CEA: cost-effectiveness analysis

A decision must be made!

All CEA begins with a clinical decision where we are uncertain about the best path forward. Nevertheless, when we face patients we must do something, even if that something is nothing. CEA brings our dilemma to life. Helping us quantify trade offs

Usually we compare a fair description of usual care to an alternative - make sure you agree the choice is fair, realistic, and represents an actual clinical dilemma

Take for example whether a patient with NAFLD should be referred to hepatology or stay in primary care after testing

journals.plos.org/plosone/articl…

The if-then of decision making is laid out.

Q: does this look like a valid description of the dilemma?
If yes, proceed. If no, move on

Now, the die is cast

We are about to simulate 1000s of people being exposed to these decisions. How they are treated depends on diagnostic test characteristics or the safety/efficacy of the treatments. Positives, true and false and risks play out with costs and benefits

After the decision tree is drawn, people enter into health-states

Which state they start in is based on their epidemiology; how they are treated is based on what happened in the decision tree; and the future events depend on how we assume they/we respond to those decisions

Does the “state-transition model” look real to you? Does it look like a real patient experience?

If yes, continue. If not, stop

How realistic are these probabilities? Take a look at what should be table 1. Look good to you still?

Here’s a table 1, laying out disease prevalence and diagnostic test characteristics in a decision model about liver disease diagnosis

We are now at a point when the model can describe the way the population was classified. Was it correct? Sometimes this is in a table l, sometimes in the text

But what is cost-effectiveness?

CEA often compares the costs of care with a tradeoff, typically quality Adjusted life years QALYs

Costs are estimated from the costs of the interventions studied and the costs of care for each health state when the perspective is that of the system/payor over the patients life

QALYs reflect the life years gained after the decision multiplied by the utility of living in that health state

My utility is 0.9 when I wake up
1.0 after some caffeine
The utility of living with a disease should be taken DIRECTLY from a patient

E.g ncbi.nlm.nih.gov/pmc/articles/P…

Take a look at the table with the utilities in the paper. Does it look right? Are the supporting references from real patients? Or just “assumed”
Is the model tipping the scale favoring states related to the intervention

This is tough!
We get into it here ncbi.nlm.nih.gov/pmc/articles/P…

Now the costs

Are the costs of the interventions accurate? Are the costs of the health states reflecting all the care those patients would receive. Often these are in the supplement. It can be hard to judge. Which population were the costs derived? Is that like the modeled pop?

Then we need to analyze

We are looking to compare the costs and QALYs for each strategy

Let’s look at this table on surgical treatment of obesity for NASH

jamanetwork.com/journals/jaman… @KathleenCoreyMD

Each strategy results in costs and QALYs

The incremental costs and QALYs are tallied

Then you divide the incremental costs by the incremental QALYs for any intervention that adds QALYs to usual care

You get an ICER

Here, sleeve gastrectomy offers a 1+ QALY for $6563

Look at gastric bypass - it adds More QALYs than sleeve adds

Awesome!

But it has to be evaluated as incremental to the sleeve. Bypass can add 1+ QALY for >$200k

A couple of points

1️⃣the added QALYS are 4-5, accounts for age at model start, lower utilities, & “discounting”. A bird in the hand? Worth two in the bush. Same for QALYs. Future years are discounted
2️⃣incremental QALYs are 0.002-1. It the gain was massive, it wasnt a dilemma

So which intervention is cost-effective?

Depends how much we are WILLING TO PAY for an extra QALY

This is up to YOU

Acceptable answers range from ~30000💷 to 150000💵

There is much more I could say about cost effectiveness modeling. Should I? What questions do you have?

Summary:
1️⃣cost effectiveness models are powerful but have pitfalls
2️⃣use this framework to “gut check” the validity of a paper
3️⃣if you see a weak model, don’t dismiss the whole field. But it is ok to have high standards