Thread by @MarriottNigel

If this goes viral, I'm claiming the credit for the hashtag!

The #stats behind "Moving to a world beyond p<0.05" bit.ly/2we28Ox should be called #Pexit.

Yes there are parallels between #Pexit & #Brexit as I will explain! /1

First look at the timing.

March 2016 - ASA statement on misuse of p-values bit.ly/2ONg5sX

June 2016 - The UK votes to leave the EU bit.ly/2OMy2be

They were preceded by decades of debate on the use & misuse of P-values & the UK's relationship with the EU.

/2

In September 2016, I attended the @RoyalStatSoc conference in Manchester. #Pexit was a keynote session where @AmstatNews gave its reasons for its statement.

See youtube.com/watch?v=B7mvbO…. I am in the front row taking many notes!

9 mins into that clip is a slide explaining ...

/3

... how @AmstatNews arrived at its statement. It started in 2014 & led to much debate & discussion.

#Brexit referendum was also preceded by a renegotiation put on the table by @David_Cameron Bloomberg speech in Jan 2013 and similarly there was much debate leading up to 2016.

/4

March 2019 - #Pexit is made official with ASA publication "Moving to a world beyond p<0.05" bit.ly/2we28Ox

Dec 2019 - #Brexit is made official when Conservatives win an 80-seat majority in #GE2019 & withdrawal agreement is passed bit.ly/2HkeUNJ

Between ...

/5

...2016 & 2019, both #pexit & #brexit saw much debate over what the events of 2016 meant. 2016 told us what we didn't want but what did it mean for the future?

So much for timing. I think you have to agree that the coincidence is uncanny though.

But what finally connected…

/6

… #Brexit & #Pexit for me was paragraph 3.1 in the 2019 ASA statement bit.ly/2we28Ox on Accepting Uncertainty. I would urge you to read that in full in you read nothing else.

I still remember sitting in my hotel room in Bruges 2 days before the referendum & ...

/7

… suddenly realising that the entire #Brexit debate was a classic Risk V Uncertainty question.

Remain was a known Risk. The EU is a flawed institution but it has definitely done good as well as bad. Whether the EU would become more federal or return powers to the states ...

/8

… is a question that could be answered by looking at what had happened in the past. The bottom line is that Remain was a known quantity.

Leave on the other hand was pure Uncertainty. It wasn't possible to estimate the odds of various outcomes. Not all outcomes would be ...

/9

... bad for the UK, some could be good but the key point is that nobody could possibly know whether the outcome for Leave would be good or bad. Now that #Brexit has happened we still don't know what the outcome will be.

#Pexit is the same. The widespread adoption of ...

/10

... P-values & hypothesis testing has undoubtedly benefited mankind & it is a process that can be taught widely and easily calculated. Yet p-values have been harmful as well with issues of p-hacking and the reproducibility crisis. Is there a better way?

The 2019 ASA ...

/11

... statement is a start but we still don't know where #Pexit will take us. To quote from paragraph 1 in the paper -

"Yet the voices in the 43 papers in this issue do not sing as one. At times in this editorial and the papers you’ll hear deep dissonance, the echoes ..."

/12

"...of “statistics wars” still simmering today (Mayo 2018).

At other times you’ll hear melodies wrapping in a rich counterpoint that may herald an increasingly harmonious new era of statistics. To us, these are all the sounds of statistical inference in the 21st century..."

/13

"...the sounds of a world learning to venture beyond “p < 0.05.” This is a world where researchers are free to treat “p = 0.051” and “p = 0.049” as not being categorically different, where authors no longer find themselves constrained to selectively publish their results..."

/14

"... based on a single magic number.

In this world, where studies with “p < 0.05” and studies with “p > 0.05” are not automatically in conflict, researchers will see their results more easily replicated and, even when not, they will better understand why. As we venture..."

/15

"... down this path, we will begin to see fewer false alarms, fewer overlooked discoveries, and the development of more customized statistical strategies. Researchers will be free to communicate all their findings in all their glorious uncertainty, knowing their work is ..."

/16

"... to be judged by the quality and effective communication of their science, and not by their p-values. As “statistical significance” is used less, statistical thinking will be used more."

Replace some of the words and that could have been @BorisJohnson in full flow!

/17

The point is though that #Pexit is not a done deal, we still have a way to go & we should not forget that P-values still have something to offer.

Let's have a referendum if you've got this far! Are you a Premainer or Pexiteer? Do you favour soft or hard #Pexit?

/18

Why is #Pexit on my mind though? Since 2016 (yes really!) I have been working with the contaminated land industry claire.co.uk to help them revise their statistical guidance for interpreting soil contamination data. The task is to decide if land previously ...

/19

... used for commercial purposes e.g. chemical factory, is safe for redevelopment as residential land or whether the soil needs to be replaced first.

The existing guidance is classic 1-way hypothesis testing & comes straight out of a Stats 101 university textbook.

/20

The issue is how it is being used. Either it is not being used at all & decisions are being made without statistical input or it is being used too precisely leading to p<0.05 style decisions. It is important to bear in mind that the ultimate decision maker is a ...

/21

... local authority planning manager who takes advice from experts in the field who are mainly scientists and engineers.

It took 3 years (yes really) to agree the new guidance & the transition from Premain certainty to #Pexit uncertainty was at the heart of the debate.

/22

We may not have the answer but we are certainly on the front line of #Pexit!

On 6th March, I will be explaining how I have made the industry a guinea pig for #Pexit at the SILC Annual Forum bit.ly/2ORH4DY.

The link to the new guidance should be available soon.

/23

A long tweet simply to advertise my talk but it was only today I twigged the parallels between #Pexit & #Brexit and I wanted to share my thoughts for comment.

/end

may be of interest to @d_spiegel @TomChivers @anthonybmasters @graemearcher @Ron_Wasserstein @HetanShah