Thread by @anthonyjwells

I mentioned this voodoo poll last week when it was being touted about social media. It looks like the Mail on Sunday was the only paper to fall for it. Well done to all those other newspapers who did not. It's tosh of course. I'll explain why below (1/20)

I should add that the poll won't "send shockwaves through Downing Street", as people working in Downing Street will know the difference between a properly conducted poll and a self-selecting propaganda exercise. I do wish journalists did (2/20)

Let's start at the start. The poll was not conducted by a proper independent market research company that paid attention to using unbiased wording and ensuring a representative sample. It was conducted by a pressure group, sent to its own mailing list and on social media (3/20)

What does that matter? A poll is only meaningful *if it is representative of the population it is trying to represent*. That is, if the population is 51% female and 49% male, the sample should be 51% female and 49% male, and so on (4/20)

Proper market research companies go to great lengths through quotas & weights to ensure their samples are representative (and publish tables to demonstrate that they are!). Pressure groups doing open-access surveys of their own mailing lists? Well... (5/20)

More importantly, market research companies recruit samples using methods that aim to minimise bias. This poll was primarily sent to a mailing list of people who had signed up to a pressure group campaigning for pro-motorist policies and lower fuel taxes. (6/20)

By definition, the Fair Fuel mailing list is going to be heavily biased towards the sort of people who care enough about pro-motorist policies & cutting fuel duty to join an organisation campaigning for those ends (7/20)

That the link was also circulated on social media by pro-cycling activists cannot hope to correct that. It is the direct equivalent of a voting intention poll done by sending an email to the Tory party's own mailing list. I *hope" no newspaper would publish that (8/20)

Secondly, a primary function of polling companies is to ensure that questions in polls are fair and balanced. To say the Fair Fuel questions did not meet that standard is something of an understatement (9/20)

For example, their question on whether people wanted an increase or decrease in fuel tax was prefaced by a table claiming Britain had the highest tax in Europe & a lengthy quote from an economic consultancy calling for a tax cut (10/20)

Their question about haulier's fuel duty was prefaced by a statement saying a tax reduction would lower inflation and the cost of products. These are obviously not unbiased questions (11/20)

The question the claim that only 2% of cyclists think they should have to adhere to the laws of the road is based on this beauty of a question - the only way to say yes was to endorse a road tax for cyclists (12/20)

Finally, even if they are asked using a proper sample, and not after a million leading questions, polls asking "would X make you more likely to vote Y" are still bollocks. I've written about this at length before - for example here ukpollingreport.co.uk/blog/archives/… (13/20)

and a chapter in @robfordmancs and @philipjcowley's Sex Lies and Politics (available amazon.co.uk/Sex-Lies-Polit…
... though sadly the Amazon preview cuts out half way through my chapter, so you'll have to buy it) (14/

In short, a sample of people who joined a pro-motorist mailing list, having just answered a lot of questions about policies for motorists, and then asked whether it will impact their vote are obviously going to say yes. I amazed it was only 27% (15/20)

If you actually ask a representative sample what issues are important to them transport is picked by 2% of people. Policies are not necessarily very important drivers of voting, but if they are, it's things like the NHS, Brexit, economy & Covid that matter yougov.co.uk/topics/politic…

Meanwhile, if you are a journalist who actually cares about making sure that polls and surveys you support are robust and reliable before writing them up in your newspaper, there is some good summary advice from the BPC here
britishpollingcouncil.org/wordpress/wp-c…
(17/20)

In particular, pay attention to this paragraph

And these questions at the end... (19/20)

If the answers are

"A pressure group did it themselves"
"Their own supporters, and activists on social media"
"They just sent it out into the world, and let it get passed about"
"A pressure group"
"Ones a pressure group wrote themselves"

Then maybe ask if it's worth reporting?