One of the issues @alexa_tullett, @siminevazire, and I talked about in this episode is a super important and kinda counterintuitive question (b/c superficially it feels unscientific): When should you believe research you do not understand? (thread)
This question was posed really well by @NaomiOreskes, who pointed out that even scientists do not have the subject-matter and technical expertise to evaluate all of the science they consume and rely on ted.com/talks/naomi_or…
Oreskes proposes that we should accept scientific consensus as a kind of argument from authority - but a special one because the "authority" is a community of experts
In the podcast, we dig into that a little more. Scientists are humans too and groups of them can share faults and biases. So not all scientific consensuses (consensi? lol) are the same. When should you trust them?
1. Look at the form of the arguments. As @ArthurLupia has argued out, the public value of science is that instead of saying "trust this conclusion because of who I am (e.g., a prestigious expert)"... docs.wixstatic.com/ugd/fa8393_8f8…
...scientists formulate their arguments as, "I'll show you the reasoning and evidence that led me to the conclusions so you can see it for yourself." We make our arguments in a form that others can evaluate and critique
So even if you do not have the expertise to critique it yourself, you can often recognize if someone is making an argument in that form. Are they making their case through reasoning? Are they being transparent about the methods and data that they used?
2. Look at who is permitted to engage in the critical process. Does the consensus comes out of a scientific community that will engage with anybody who makes a good-faith critique based on reasoning and evidence? Or are some people shut out for invalid reasons?
Again, you can often see clues to how scientists are engaging even if you don't understand the arguments themselves
Although to do this well, you have to be able to detect bad-faith criticisms and sealioning. Scientists can justifiably ignore people who haven't done their homework and are just slowing things down
3. Over and above who is permitted, who is actually participating in the critical conversation? So the diversity of perspectives in the critical community (and in the wider circle of who is actively participating from outside of it) is another signal.
This isn't necessary in every case. But ideas that have survived actual criticism, including from people who come with different assumptions and goals, are stronger
4. As a negative sign, look for when scientists are using invalid bases -- like their status, prestige, credentials, etc. -- to substitute for settling questions with reasoning and evidence
A key word is "substitute." Scientists are humans, and like all humans many of us are nice and some of us are jerks
"Do you know who I am???" followed by actual engagement with a critic is a jerk move but doesn't undermine credibility. "But "do you know who I am???" as a replacement for actual engagement should be a warning sign
Of course all of these are heuristics. The best-case scenario is when you are enough of an expert (or can acquire enough expertise) to evaluate work yourself. But if you only accept scientific conclusions when you can do that, you'll miss out on a lot
So in conclusion, to decide whether to trust a scientific consensus, the public should look at whether the scientists are organized as a healthily critical community. And if scientists want to public to take us seriously, that's what we need to do for ourselves /end
P.S. I hope I made this clear at the top of the thread but just in case - this thread is a summary of a conversation with @alexa_tullett and @siminevazire, the ideas in it reflect all three of our contributions
