Thread by @ewanbirney

A meta-thread on my take of how to "read" science as a scientist. This is to arm non-scientists about how to navigate a world where one sees the "leading edge" of science develop as we do now in COVID.

(Context: I am an expert in human genetics and computational biology - data science in biology. As Deputy Director General of @embl I have the pleasure of being involved in a lot of science in a strategic way both inside @embl and internationally).

The first point is that most scientists have sets of observations about the real world which are solid - they have been measured multiple times; multiple groups found the same thing; ideally measured in different ways.

Any understanding of the world has to be consistent - or at least maximally consistent - with these solid observations. (when one set of solid observations does not fit it feels very very uncomfortable to a scientist)

"Solid observation" might be experiments; might be observation of complex phenomena in the world; might be observation + theory. In each case there is a sub-discipline like art/standard practice to be "solid".

For most scientists we're only lead practitioners in a handful areas - ie, we could have done the analysis or experiment ourselves. But we also usually have a broad appreciation of our surrounding penumbra of science, knowing the obvious "gotchas" and going out for here...

There is a semi instinctive of what solid observations look like in science. One hallmark is that it is the stuff experts agree on so quickly they don't even argue about it - it is solid enough that the experts move quickly onto areas of more productive disagreement.

this gives the first slightly counter intuitive thing about reading leading edge science - you need to read from the outside what is *not* being said so much, because thats the solid ground. It's often quite hard to work out in fact.

*Occasionally* the (previous) solid ground shifts significantly. It is rare that it turns into pure quicksand rather more like a tectonic plates moving (we thought this fitted here, but actually... it fits like this). This is tremendously exciting when it happens but it is rare

Far more common in science is this progressive "solidification" of observations from a solid core. That's pretty much where we are with SARS_CoV_2 - progressive growth of observations, solidifying more and more ground around us.

In my head, on top of this solid ground of observations is a theory/model of how it all fits together - a sort of intellectual super-structure. This super-structure is personal to each scientist (though often the core bits agree).

Let's call this superstructure "understanding" and from it scientists predict useful experiments or observations to do next - on the "marshy edge" of the solid ground - science is not a random walk in experiments or observations, rather it has a structure.

Unlike solid observations there is much debate between scientists at some point of their respective personal super-structures; indeed, scientists will often rush their conversations straight to the points of disagreement which can get quite heated!

So - it looks like scientists disagree alot often. This is ... the nature of science and exploring the world. When done productively the argument resolves down to "well if I am right, this experiment or observation will say X, but if you are right, it will say Y".

(sometimes it is not productive; sometimes arguments are stomped on too much or sometimes arguments get so heated and personal it messes up the progression. These are all sort of meta-failure modes of a region of science)

To "read" science from the outside therefore you need to find that common core of understanding standing on the common ground of agreed solid observations. Over time the common ground grows, and the edifice of common understanding grows with it.

It is a category mistake to blindly trust one scientist over another (though we all carefully burnish our track records of being right) and any good scientist will change his or her mind in an instant when presented with clear data.

(scientists who cling onto their own opinions in the face of data either have to be so conservative they can never progress or just become pseudo-scientists that endless discard things that don't fit. All good scientists get things wrong regularly - they just move on quickly!).

So - look for "missing space" often of agreement between scientists rather than focusing on the disagreements (and scientists when you are communicating for a broad audience, I think it's best to a quick sketch of key solid ground things and why it is so solid)