This tweet got me thinking again about a topic that's been on my mind for the last several weeks and throughout the pandemic. In principle I fully agree with @flodebarre that people should evaluate arguments for logical soundness and consistency with facts, not who makes them.
But many people have asked me (most recently @AmyDMarcus) how thoughtful people should know whom to trust in getting information (science) and advice (for personal actions) and opinions (about policy) on a topic like COVID
Consistent with @flodebarre's tweet, my first response was you shouldn't trust anyone intrinsically, but should trust good arguments. As a scientist, that is how we are (or should be, there is still too much hero worship in our field) trained.
But for any particular topic, it is out of the expertise for 99.99...% of the world to evaluate the arguments. Ask me about COVID epidemiology, and I'll evaluate the arguments. Ask me about dengue epidemiology, and I'll try but not fully trust myself.
Ask me about diabetes epidemiology, same. Ask me about cardiac electrophysiology, probably can't. And those are all in my broad field of health science. On what our foreign policy should be in (pick a country) I'm at the mercy of the experts: whom to trust?
If I apply my nostrum of "evaluate the arguments not the speakers" to whether the Afghan withdrawal was a good idea, for example, I have a strong sense but also realize that I am too ignorant about the history, the current politics, & other considerations to really trust my views
So fair enough that a person (most of the world) who spends their time doing something other than studying COVID is looking for heuristics for whom to trust.
I think there is no perfect classifier of "reliable expert" vs "other." There are factors that probably make the prediction better than random. Here are the ones I use, as best I can unpack my mainly subconscious process.
Matters if it's someone I know or someone new to me. Lots more options available for someone I know. For someone new: 1. Expertise in one or more relevant fields. A degree is correlated with expertise, but is neither necessary nor sufficient.
The most insightful conversation I have had this week about COVID was with someone whose current highest degree is a master's degree. And a degree can signify broad expertise or very narrow knowledge.
But still, all else equal, a degree in a relevant field is reassuring. And that in itself requires judgment. An MD (alone) does not make one an expert on ventilation and virus transmission, any more than I'd ask @j_g_allen for how to treat lower back pain.
Nutritional epidemiology is different from infectious disease epidemiology. So it depends on what is being asked about.
2. Where someone works is again probably correlated with their likelihood of being knowledgeable when they speak out on a topic, but we all know exceptions in COVID of people from prestigious institutions saying nonsense or worse. Imperfect but not meaningless.
3. Does the statement show an awareness of considerations outside someone's narrow field, if it should? Most decisions about the pandemic involve epidemiology, other parts of public health, and also economic, social, and psychological considerations.
People who either a) limit their statements to their own field (an epidemiologist talking about how much distancing reduces transmission) or admit the relevance of considerations outside their field to a decision are more trustworthy than people who ...
start with "distancing reduces transmission x amount" and conclude "we must implement distancing" without at least admitting the implicit assumption that the costs thereof are worth it, for a certain time frame etc.
4. Does the person show in their statement that they have considered contrary evidence? And refute it in a way that is not ad-hominem and otherwise conforms to fair argumentation? Sometimes hard to tell, as the whole thought process may not be described, but sometimes it is.
5. Is it a person or an institution speaking? As far as one can understand, are there bureaucratic (cover-yourself, risk-averse) or self-interested motives that could explain the position they take? Again, imperfect, as a self-interested statement can also be true. But relevant.
6. (now getting into territory where "knowing" someone (personally, from long twitter following, or whatever in between) can help. Asking myself: can I guess what this person's position will be on a topic before I read it? If so,uninformative: they could be right and predictable.
but if not, their view is of greater interest and to me more credible.
7. (ht @HelenBranswell) do they ever say "I don't know" or "that's beyond my expertise" or do they have a view on everything? I trust the former more.
8. Do they distinguish between fact, educated speculation, and opinion?… Here @BillHanage and I wrote about this distinction in the context of the current pandemic
Probably will think of more. Bottom line, figuring out whom to trust is hard. The most trustworthy people are sometimes wrong. (ah that reminds me, 9. do they correct errors publicly when they make them? Good sign).
No heuristic or combination of them is perfect, and ultimately the more one can evaluate arguments and not speakers, the better. And an informed opinion is not one downloaded from the MOST TRUSTED SOURCE but one informed by broad reading, discussion, etc.
A corrollary is that for most of us who work for a living and/or have other demands on our time, having a truly informed opinion on most matters is beyond our time constraints.
Thanks to someone who brought up peer review. This is whole other ball of wax but yes peer review increases confidence, but is not infallible. In short my view is that peer review is the penultimate safety net / slice of swiss cheese in science.
Science works because we have multiple filters: scientists being trained well and working carefully; scientists wanting to be the ones to see the flaws in own work before someone else (reviewer) does; peer review; replication
when all that's in place, science does well; when one or more layers of net are frayed, caveat emptor

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with Marc Lipsitch

Marc Lipsitch Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @mlipsitch

13 May
In this clip @SenRandPaul FALSELY claims… that the @cambridgeWG has characterized work at the Wuhan Institute of Virology as gain-of-function.
I and many other @cambridgeWG support proper investigation of SARS-CoV-2 origins including the lab leak hypothesis and continue to oppose many forms of GOF research but it is just fabrication to say we have made any statement as a group about work in Wuhan.
What we called for was a moratorium on GOF research until proper risk-benefit calculations can be done. Just as this pandemic was starting, two of us were strongly critical of how @NIH and @HHSGov evaluate GOF proposals…, calling for much more transparency.
Read 5 tweets
12 May
Really important @nytimes article on how easy access to vaccines remains a key issue not just hesitancy…. I’d add two points less explicit in article
1. People say that seeing others get vax without incident reduces their hesitancy. If so then each vaccine administered to those where access is the main problem can have a multiplier effect in overcoming hesitancy in others.
2. If we think of getting vaccinated like any other choice then hesitancy and ease of obtaining are two sides of same coin. If hard to get, a little hesitancy will stop. If easy to get, only the very hesitant won’t.
Read 4 tweets
30 Apr
Great that attention is remaining focused on #AMR a big problem.… . But the relentless claims that this is primarily or largely a problem of agriculture are not evidence-based.
Some years ago @BillHanage and I and @G_RegevY and our students pointed out the absence of strong evidence (while acknowledging challenges of generating it)…
Recent excellent genomic work from Prof Sharon Peacock's group, @MarcBonten's group among others has failed to find evidence of strong links between resistance in animals and resistance in clinical isolates. @BillHanage wrote about this…
Read 4 tweets
19 Apr
Is there some bizarre legal notion of causality at play here that the force used would have had to be such that it would have killed anyone, not just the actual person it did kill?
I'm willing to believe that the law traditionally has such a notion (or not -- hope some lawyers will help me understand) but if so it seems truly indefensible. We all have the preexisting condition of being mortal. We each have a different breakpoint for how much...
of a certain kind of abuse we can survive. It seems incoherent to have a standard that it's not murder if a different person would not have died under the same horrific abuse -- and I'm not saying that is true, only that this seems to be the defense's standard.
Read 4 tweets
22 Mar
.@alisonannyoung has been one among the most active, persistent, and fair-minded reporters covering lab accidents in the US @USATODAY. This long piece makes the serious case for investigating poss lab origin for #SARSCoV2.….
This is not about conspiracy theories or China-bashing. This is about the basic principle, at least as old as Rev. Bayes and Sherlock Holmes, that when something unusual happens, you have to consider explanations that are also individually unlikely.
To be explicit, the @WHO mission's conclusion that a lab accident was unlikely was unjustified (as @alisonannyoung notes they are frequent) but even if true, global pandemics are also uncommon, so by definition the sequence of events leading to one do not happen all the time.
Read 11 tweets
15 Mar
CONGRATULATIONS DR. @AccorsiEmma defending her PhD with flying colors today. Thanks especially to @chuttenh who supervised much of the work linking microbiome and #epidemiology #EpiTwitter and to committee members @SHaneuse and @BillHanage @HarvardEpi @CCDD_HSPH @HarvardChanSPH
Two of her thesis papers have been published.…
Read 4 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!