Thread by @pauli_alin

Experts have been wrong about many things during this pandemic. It happens; it's normal; it's expected. But what happens after it it turns out that experts have been wrong? The standard model of #expertise suggests that experts change their opinion or become non-experts. 1/

However, evidence from this pandemic suggests that after experts have been found to be wrong about something (say, role of aerosols), they don't change their opinion overnight, if they change it at all. And they can get away with it, still remaining 'experts'. How? 2/

To begin to explain this, let's borrow a framework from physics. According to @skdh, a physicist, physicists have been wrong a lot, so this is a useful field to borrow a framework from. The framework allows us to explain and predict how pandemic expertise develops. 3/

In her book "Lost in Math", p. 40, @skdh writes: "It is rarely possible to actually falsify an idea, since ideas can always be modified or extended to match incoming evidence." According to this, experts can claim that being wrong was justified before more evidence came in. 4/

This claim of theirs is easy to evaluate in theory but hard to evaluate in practice; the evaluator would have to study what evidence was available to the expert when they said something that turned out to be false, and then compare that to the evidence the expert actually used.5/

Such a comparison would allow the evaluator to base the evaluation of the expert's competence in empirical evidence: e.g., if an expert in fact had access to evidence contradicting their claim when making the claim, then the evaluator could deem the expert incompetent. 6/

Such evaluations via comparisons are rarely made though. Why? One possible reason is that societies are not interested in evaluating the competence of their experts. This is implausible as such evaluations are conducted all the time (e.g. job interviews, peer evaluations). 7/

Another, more plausible reason is that making such comparisons requires resources (e.g. time to comb through papers & journals). As such comparisons would need to be made for each of the expert's claims, it's easy to see why the evaluator would rather just believe the expert. 8/

We are arriving at a resource-based explanation of expertise: individuals considered 'experts' can continue being considered 'experts' regardless of the quality of their claims when evaluating the quality of their claims requires more resources than is available. 9/

This explanation suggests that individuals initially considered 'experts' can remain considered 'experts' indefinitely when the society in which they operate does no allocate sufficient resources to evaluate the quality of their claims. 10/10 #SociologyOfExpertise

@pauli_alin (Collins & Evans, 2002, p.236) Collins, H. M., & Evans, R. (2002). The Third Wave of Science Studies: Studies of Expertise and Experience. Social Studies of Science, 32(2), 235–296. doi.org/10.1177/030631…

@threadreaderapp unroll