ONS, breaches, + teachers: THREAD

Today I reported an ONS breach to the UK Statistics Authority.

Two breaches in fact: one regarding an “ad hoc” analysis on education staff in part 9 of the 6 Nov ONS covid survey pilot, and one for a related 3 Nov government leak. 1/
On 3 Nov, the Department for Education tweeted a video of Chris Whitty saying:

“All the data, including ONS data, do not imply that teachers are a high risk occupation, unlike, for example social care workers, and medical staff like myself.” 2/
According to the UK Statistics Office “Code of Practice for Statistics,”

this was a breach of trustworthiness principle T3: Orderly release. 3/
T3.4 states that “ahead of their publication [...] no indication of the statistics should be made public.”

T3.8 says “Policy, press or ministerial statements referring to regular or ad hoc official statistics should [...] contain a prominent link to the source statistics.” 4/
This 3 Nov announcement led to confusion, as before 6 Nov, the only published occupation-differentiated ONS covid data was on deaths during spring lockdown.

And although Whitty said “all the data, including ONS,” he never ended up identifying non-ONS data on this topic. 5/
The more important breach, though, has to do with the 6 Nov ad hoc analysis itself, which defied the “Code of Practice for Statistics” in so many ways as to raise serious concerns about independence. 6/
Though the other analyses in this 6 Nov report used data through *31 Oct* & included time dependence, the ad hoc teacher analysis only used 2 Sep-*16 Oct* data, & suppressed time dependence.

This matters: 17 Oct is about when 2-11-yr-olds overtook 35-49-yr-olds in infection. 7/
Why omit the 17-31 Oct occupation data—especially when this omission was so likely to bias results?

Did they feel they had run out of time to analyse already-acquired data? Code of Practice says that’s an unacceptable reason, especially considering the 3-day-early leak. 8/
OTOH, with teacher training, many schools started Mon 7 Sep instead of Thu 2 Sep. Infection takes time to incubate before testing +.

So up to 10 days worth of their early data predated the option for school-acquired infection, diluting total positivity results for teachers. 9/
Wait, it gets worse: just LOOK at the error bars!

Data was disaggregated in such a way as to make the study totally underpowered.

Error bar width is
>2 times the positivity value for most edu categories,
>33% higher than positivity for ALL BUT ONE edu category.

But there was one exception: “teacher of unknown type,” with error bar 35% SMALLER than the positivity—meaning most respondents simply answered “teacher.”

THIS teacher category had
28% higher positivity than key workers,
16% higher positivity than other professions. 11/
So really, the chart should have just looked like the one below, with more sensible error bars,

and with teachers in the highest positivity category by far, even without the 17-31 Oct data.
It was only the disaggregation of teachers into underpowered buckets that made it plausible for ONS to claim:

“Data [...] show no evidence of differences in the positivity rate...” in the text and “there is no evidence of difference in the positivity rate” in the caption. 13/
These “no evidence of differences” comments don’t mean what the audience is intended to think they mean.

Rather, they mean, “we disaggregated and restricted the data in such a way as to make the analysis too underpowered to generate significant evidence.” 14/
This badly violates the Code for Practice.

And saying “no evidence of difference” vs “evidence of no difference” isn’t a get-out-jail-free card.

Context matters. Readers expect that if you publish an ONS analysis, the verdict is not: this was underpowered by design. 15/
The ONS are also obligated to provide rationale for substantial deviations from standard methodology, such as why they omitted 17-31 Oct data and suppressed time dependence for this one analysis, in a report that consistently provided time-dependent data through 31 October. 16/
On a side note, it contradicts Swedish findings.

In spring, Swedish teachers of upper sec schools (fully remote) had median infection rates.

Teachers of (fully open) lower sec schools had nearly twice the % of confirmed cases, and were among the most infected professions. 17/
It is not the role of the ONS to manufacture false evidence for purposes of state propaganda. This message of this one analysis was

-announced by Chris Whitty on 3 Nov,
-tweeted by the Department for Education on 3 Nov, ...
-invoked by SAGE in the 4 Nov (published 13 Nov) report “TFC: Children and transmission 4 Nov, 2020,” where it was the only included “evidence” on teacher risk—in fact they (predictably) changed “no evidence of difference” to “show no difference”,...
-tweeted on 13 Nov by the Department for Education (this time by invoking the 4 Nov SAGE report) as new evidence of school safety, even though much of the actual child-related data in the report involved strong evidence of substantial transmission,...
-and invoked 18 Nov by Scottish Government as part of their evidence of school safety from covid,...

(Again, “no evidence of difference” became “found there is no difference.”) 21/
As an American-born, this all brings back too many bad memories of George W Bush’s “weapon’s of mass destruction.”

If you push hard enough for the manufacture of evidence, evidence will be produced, whether it’s valid or not.
That’s why ONS independence is so incredibly important.

We can’t let this happen again.

Since this kind of blew up, I wrote a follow-up thread. Wanted to clarify several things. Also, sorry for instances of inflammatory language in the above thread. Trying to be a good Twitter communicator, but still learning. 24/

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with Sarah Rasmussen

Sarah Rasmussen Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @SarahDRasmussen

5 Nov
School + CEV households: THREAD

New update yesterday for UK Guidance on shielding. Guess what stayed the same:

“Children who live with someone who is clinically extremely vulnerable, but who are not clinically extremely vulnerable themselves, should still attend school.” 1/25
“Should” is a euphemism for “must.”

If parents deregister, they lose their school place, must educate their child on their own from scratch, and risk denial of re-admission at their local school.

But trying to stay at school but work from home means fines and prosecution. 2/25
Just what risk is on the table? It’s hard to estimate, but let’s try. 3 Q’s:

a) What’s the current risk of a UK child catching COVID in school?

b) What’s the accumulated effect of repeating these risks over time?

c) What’s the risk of the child transmitting at home? 3/25
Read 27 tweets
24 Oct
Schools + comments on some sources (THREAD)

This recent schools article makes some great points, and @apoorva_nyc is one of the top science journos on covid out there, but a few of the sources cited in this one have some issues, especially a few pages from govt websites. 1/22
-UK link is outdated. For late Oct, random sampling estimates all 3 child age-groups more infected than any adult group. 2-11s are 2nd highest!

-The Netherlands govt website badly mischaracterises its study.

-Pediatrics childcare-worker study and NYC article need comment. 2/22
UK schools opened at start of September. No remote school, even for families with medical vulnerability. No temporary home schooling either.

The article’s linked UK govt webpage shows Sept infection ⬆️ for secondary-school and ⬇️ for primary-school and a few other groups. 3/22
Read 22 tweets
19 Oct
@apsmunro @interpolated I have no problem with the article itself. It’s with the summary written for it, which you amplified without explaining any corrective context, not even the fact that those positive results were from random sampling, whereas the ordinary assumption would be targeted testing.
@apsmunro @interpolated In the meantime, you cheer on 0-follower 21-tweet trolls who attack me when I try to have a nuanced discussion where I simply ask questions about the level of evidence we should demand before advocating for a certain policy—a question relevant to your own advocacy. Image
@apsmunro @interpolated But in case it’s more helpful if I explain what I mean when I speak of recent cherry picking of headlines, articles and/or evidence standards... 0/n
Read 21 tweets
9 Oct
Schools and personally-acceptable risk (THREAD)

This thread is an attempt to address questions certain UK parents and school staff have raised to me.

I’m not an epidemiologist, but as a mathematician I’m weighing in on certain general notions of probability and risk. 1/12
No one knows how bad this 2nd wave will get. It depends on what hard and soft measures are implemented and when, and how they are supported.

But I think it is ethically and scientifically unjustified to trivialise the concerns of parents and school staff about school risk. 2/12
Yes, there’s now good evidence that the covid mortality rate for healthy under-19-year-olds is likely no worse than flu *for*that*demographic.*

But that is only one type of risk. 3/12
Read 13 tweets
3 Oct
@SmutClyde @michaelroston @ThePlanetaryGuy Unfortunately, neither my World Scientific institutional access through U Cambridge nor that through IAS Princeton includes IJGMMP—I guess limited demand.

But there’s a retracted Mac J Med Sci pub by the same authors (et al) w/ “topoisomerase-like waves.”
@SmutClyde @michaelroston @ThePlanetaryGuy The retracted article’s argument seems to go:

1) topoisomerase unwinds DNA,
2) um, waves can be kind of wound up looking (?),
3) ergo, waves could unwind DNA like topoisomerase.

Thing is, that doesn’t make sense topologically.
(And I’m a topologist for my day job.)
@SmutClyde @michaelroston @ThePlanetaryGuy Topoisomerase doesn’t unwind DNA like a ball of yarn; it untangles by *crossing*changes*—temporarily snipping DNA for it to pass through itself, thereby changing the embedded topology of the DNA as a tangle/knot.

Simply “pushing DNA around” with a wave would NOT change topology.
Read 9 tweets
27 Sep
@ingridjohanna66 @threadreaderapp Thanks!

Initially stumbled on all this by accident.

I’m new to Twitter. Have mostly tried the academic route on this. The letter of corr + systematic review I sent to LC&AH on this were rejected, and now my univ’s Research Gov Office is working with UKRIO to organise an audit.
@ingridjohanna66 @threadreaderapp I originally worked alone on this, since didn’t want to disrupt med researchers at a time like this.

I know journals are doing the best they can with an avalanche of submitted articles that could influence policy that saves/jeopardises lives.

Difficult to know how hard to push.
@ingridjohanna66 @threadreaderapp (To clarify, what I sent was rejected by the LC&AH editor without ever being sent to peer review.)
Read 4 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!