Share this page!

Vinay Prasad, MD MPH 🎙️📷 Profile picture
Twitter logo
May 28 • 32 tweets • 11 min read
As you read the #ASCO22 abstracts, here are some tips I offer to separate true & useful from all the rest

PS: The Amgen ad on the banner of the abstract page is emblematic of what you have to contend with
🧵
I. Let me start with observational/ uncontrolled studies. These can be useful to describe prognosis, identify risk factors, track time trends, establish activity, etc. but there are many caveats
1. An abstract shows survival improves by decade in a cancer - author claims it is due to better drugs

Possibly yes, but also possible the definition of the cancer is different (AHEM myeloma), we look for it more often, our CT/PET scan is more sensitive (stage migration), etc.
The last point is under-appreciated. If you find someone who has a large colon mass and a tiny liver or lung met, that person might not have had the tiny met discovered in '99 (stage III) but in '22 they do & are stage IV

This shift will improve 5 year survival in BOTH groups!
It is called the Will Rogers phenomenon. (look up joke about Okies moving to Cali)

If you unpack these things, the truth is more complex, as we find with CRC. It is not all "better drugs"
jamanetwork.com/journals/jamao…
2. Observational study shows which cancer patients do worse when they have COVID19:

Let me guess?
Being older, on more medications, with more rapidly progressive cancer (aka being sicker/ more vulnerable)

Ask yourself if you are learning anything that isn't blatantly obvious
Next, think about the Table 2 fallacy

Most people doing this work are totally vulnerable to this b/c they are amateurs
3. New cancer drug does great in uncontrolled study
BRAVO!

Of course, consider that authors cherry picked their patients for the phase 1/2 trial!

Only the healthiest/ pan sensitive/ indolent tumor biology are included

It is not all comers!
3. An uncontrolled trial can establish that there is some response rate, which is a measure of drug ACTIVITY

But it cannot tell you EFFICACY--- that people live longer or better than alternatives
4. Drug has lousy single agent activity, but might be useful in combination....

Shut up! No it won't be.

These drugs do awful in drug development. The denominator is millions, the numerator is a dozen+, the ones that succeed are mediocre |1.4 mo os

drive.google.com/file/d/1yxamqR…
5. Among people who respond to a drug, survival is better than those who do not respond!

Also among those who have EFS events vs those who don't

Get out of here! This is an elementary error.

Analyzing outcomes by response is a flawed way to infer efficacy-- this has been known since 1983!

Stop it already!

(almost as old as me)
pubmed.ncbi.nlm.nih.gov/6668489/
6. We compared CAR-T patients to the database of....

GTFO

Comparing the ultra select patients in CAR-T trials to all comers.. pointless exercise

Let me guess? the paper is written by pharma?
7. Cost effectiveness analysis sponsored by pharma shows it is cost effective

Or CEA written by first and last author with conflicts & all middle authors work for consulting firm..

I am sure the first author did so much work

This is all bad
jamanetwork.com/journals/jaman…
II Lets turn to randomized trials

Just because it is randomized don't make it good!
1. Is the control arm what you would do otherwise?

17% of FDA registration studies use a control arm known to be inferior to best care

"A trial can only change your practice if the control arm is your practice!"
-vp

drive.google.com/file/d/1jNN1sW…
2. Did they improve PFS or did they actually improve OS?

PFS is a poor predictor of OS across most tumor types...

Even DFS has a lousy correlation, and for the class of drugs presented probably no correlation at all!
jamanetwork.com/journals/jamai…
3. IF they improved HR quality of life, did they measure it properly?

I doubt it!

jamanetwork.com/journals/jaman…
4. If you are testing a drug that has never established efficacy (sip T or vandetanib) you DON'T WANT crossover

IF you are trying to move a drug upfront, including to adjuvant you absolutely DO WANT IT

Read this to understand Xover

drive.google.com/file/d/1DQg0ny…
5. If the control was investigator choice was it a restricted choice?
or a free, unfettered choice?

This matters
6. Was post-protocol therapy up to the standard of your practice?

This will result in a similar bias as crossover. Bad post protocol care means downstream endpoints (PFS2/ OS) do not apply to your practice

More to come on this topic...

jamanetwork.com/journals/jaman…
7. Was drug dosing fair?

This is a technical point, best explained in our 2014 JCO paper

drive.google.com/file/d/1n4RDxM…
8. If it was a non-inferiority study, is the margin so big you could park a school bus in it?

Was the choice of non-inferiority justified in the first place?

The new drug has to be cheaper, more convenient or less toxic to even run such a trial.

drive.google.com/file/d/1dT3UFx…
III Stylistic things

Here are a few style things to look for
1. So many authors who didn't do anything
Don't meet authorship criteria
Enrolling pts doesn't make an author see ICMJE rules
2. Everyone is conflicted

More to come....

drive.google.com/file/d/1co0CAs…
3. Its a "virtuous topic"
Good people agree on the right answer
4. Speaker flashes the COI slide so fast, no human can read it

Happens 1/3 times as documented by @AaronBoothby2

jamanetwork.com/journals/jamao…
I left out my favorite one for RCTs

8. Lots of people dropping out is grounds for suspicion

See Vision trial where I explain this bias


Censorship games
Ok, you liked this thread and want to learn more. Here is what you can do

1. Check out Malignant, now with 100+ reviews
amazon.com/Malignant-Poli…
2. Watch the 3 (someday 10) part lecture series on cancer clinical trials on Youtube
3. Listen to & follow @Plenary_Session

podcasts.apple.com/us/podcast/ple…
4. Follow @vkprasadlab for updates on papers that probe these biases

• • •

Missing some Tweet in this thread? You can try to force a refresh
　

Keep Current with Vinay Prasad, MD MPH 🎙️📷

Vinay Prasad, MD MPH 🎙️📷 Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @VPrasadMDMPH

Vinay Prasad, MD MPH 🎙️📷 Profile picture
May 26
Over the 15 years I have closely read medical studies, I find this is true every single time you have a 'virtuous' issue.

Whenever you decide that good people believe x, you quickly find a proliferation of bad science.

In this case, good people believe medication adherence...
Is often suboptimal, and our efforts to improve it work magic, and as a result of those efforts, the patient is better off. But each of those propositions may not withstand scrutiny

Other issues where people have decided what is the right answer are similar...
The evidence supporting the claims is often threadbare, just absolutely incorrect, but the conclusion is the desired one. There is very little professional incentive to point this out, and a strong disincentive.

Ironically...
Read 7 tweets
Vinay Prasad, MD MPH 🎙️📷 Profile picture
May 24
I was just reading this superb toolkit from Urgency of Normal -- that's the group who thinks about kids & restrictions sensibly & numerically; trying to maximize kids' overall life outcomes/ health

Check out the highlights 🧵
static1.squarespace.com/static/61e5afd…
Puts it in perspective, doesn't it

We didn't cancel kids birthday parties (for years on end) for RSV
Wow, another fact
Read 7 tweets
Vinay Prasad, MD MPH 🎙️📷 Profile picture
May 20
#17 Our new paper @vkprasadlab is now out in @JAMANetworkOpen

SAME drug SAME cancer

Discontinuation rates are MUCH higher for toxcity in the adjuvant rather than metastatic setting

Why?

jamanetwork.com/journals/jaman…
First the effect we find is present for checkpoint inhibitors and targeted drugs, but less prominent for cytotoxics
Second it is across a wide range of drugs
For diverse indications
Check it out
Read 6 tweets
Vinay Prasad, MD MPH 🎙️📷 Profile picture
May 19
It's actually an insane precedent to recommend things for healthy kids with the lowest levels of evidence imaginable.

Over next 30 years regulatory science will enter a death spiral. FDA setting just enough barriers to market to favor large companies...
... but not enough barriers to actually ensure drug products improve outcomes people care about. The cost of the products r astronomical. The companies are bleeding us. Reviewers at the FDA most frequently go to work for Pharma afterwards...
... GDP spending on health care reaches record highs.

Never let a crisis go to waste is the old slogan. And Pharma took advantage of it with low regulatory standards for vaccine boosters.
Read 5 tweets
Vinay Prasad, MD MPH 🎙️📷 Profile picture
May 14
No RCT shows OS benefit to ASCT in CR1 in era of novel drugs
No RCT shows quadruplets better than sequential triplets (w/ proper access to dara)
No OS benefit in RCT for tandem ASCT
No data to support VRD maintenance
Myeloma is not an EBM field
Many trials serve pharma not pts
PS this trial has nothing to do with anything unless you were in the VCD to VMP business
Yes, there is. The reasoning is flawed. The burden is to show treating FLC >100 improves outcomes, not that response comparable. That burden has never been sated
Read 4 tweets
Vinay Prasad, MD MPH 🎙️📷 Profile picture
May 14
A key error during COVID was that TWO irrational extremes prevented us from having a sensible conversation about vaccinating the right people, based on good data.

1 Side was opposed irrationally
2 Proponents ignored myocarditis....
... allowed the widespread boosting of adolescents with no credible data.

And have constantly banged the drum for a child's vaccine for an old ancestral strain that failed the vaccine efficacy standard of the FDA.... sad!...
A well-intentioned effort to combat irrational anti-vax rhetoric created an irrational anti-antivax side.

And a sensible dialog by folks good at interpreting evidence could not be had....
Read 6 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

:(