Thread by @KidTempo

Welcome back to the popular series of "Misinterpreting survey results" (though this time I think without an attempt to deceive).

Again, a cautionary tale of how misreading survey results can lead to very misleading conclusions...

Previous thread:

https://twitter.com/KidTempo/status/1149655215538204673

https://twitter.com/Survation/status/1161591081756233728

Survation has just published the results of a survey showing preferences of whether people wish to a) Remain in the EU, b) Leave with a Deal, or c) Crash out of the EU with no deal.

In addition to which choice voters would prefer, the survey also asked which would be the second preferred outcome i.e. the second preference - which is a good question to ask BUT its results must be treated with care.

No. The 2nd preference cannot just be added to the 1st preference - especially in a question like this where there are only three options (not to mention that two of them favour Leave).

With only three options (A, B, and C), and where people who voted for the extremes (A and C) are totally opposed to the opposite options (C and A), the choice of 2nd preference would almost always be the middle option (B) - it does not mean that B is in any way a popular option!

I mean, take this exaggerated example: Option B is clearly not the most popular 1st choice, nobody really likes it but when taking the 2nd preference into account looks like it has overwhelming support. This is just wrong.

So why did it score so highly? Probably because the Option A people hate Option C, and the Option C people hate Option A. With no other choice they go for Option B.

Would everybody be happy with Option B? No - at best it could be said that the most number of people would be equally unhappy with Option B. Mutually assured misery for all - hardly optimum.

If we know that the opposite sides are entrenched (and when it comes to #Brexit we definitely know this is the case) then the best we can do with this data is take away the middle option and try to determine what those voters would choose instead.

This example is very simple, so it should be clear from the data* that of the 5% that voted B, 4% would choose A as 2nd preference, and only 1% choose C.

That would mean that it's most likely that A would get 84% and C only 16%. The optimum result is therefore A.

*I'm assuming that all A and C voters would choose B as a 2nd preference (because its a simple example). In reality this would not be the case - there would be a minority of people that would choose the opposite option instead of the compromise option.

Attempting to do the same with the Survation results, taking away the 2nd preference option of "Leave with a deal" gives the following results:

Remain in the EU: 56.2%
Crash out with no deal: 43.8%

So, not unlike polling elsewhere...

Survation could have included data linking the 1st and 2nd preferences, but unfortunately they didn't, so the best we can do is speculate as to how the "Leave the EU with a deal" vote would be divided up in a binary choice between "Remain" and "Crash out with no deal"

Having said that, we can try to read the tea leaves based on, for example, the 2019 EP vote (there was a lot of tactical voting in the 2017 GE so it's not reliable, and the original 2016 referendum is equally problematic).

It's a safe bet that BXP voters aren't keen on remaining in the EU (though it's surprising how many actually want to leave with a deal - more than with no deal!)

Based on the numbers, Conservatives are also pretty Leavey (side note: it's surprising how few answered the survey)

Again based on the numbers, the other parties are overwhelmingly on the side of Remaining in the EU (even Labour). Amongst those parties Remaining is clearly the 1st preference.

Based on the above, it could be said with some degree of confidence that the binary support is (very roughly) 56% Remain, and 44% NoDeal.

But there are caveats - big ones - with using this data to try to interpret how a vote would turn out in the even of a binary choice.

It's difficult to determine how likely someone is to actually vote just based on how they voted in a past election (Survation should have "likelihood to vote" data that they can directly link, but that data cannot be accurately determined from the tables)

BXP voters are over-represented in the survey - as are, arguably, the Greens. It could also be argued that the quiet conservative Conservatives are underrepresented (many chose not to vote in the EP elections)

The balance of people who voted Leave or Remain in the 2016 referendum is also problematic. On the one hand there are more Remain voters who participated in the survey than Leave voters - contrary to the referendum result - on the other, that may be for *ahem* demographic reasons

There's quite a high number of "Don't know's" in the 2nd preference table - this could be legitimate undecided voters, or also people who chose Remain/NoDeal as their 1st preference and were unwilling to choose "Leave with a deal" as their 2nd preference.

As with "Don't know's" above, it seems that quite a lot of people declined to answer the 2nd preference question. I expect this is for the same reason - they refuse to select a 2nd preference because they are committed to their 1st choice (which is most likely Remain or NoDeal)

Finally, the biggest factor that surveys cannot hope to capture are the views of the unengaged. These people don't participate in surveys; they have little interest in political social media, they may or may not vote - turnout may vary wildly depending on whether it rains or not.

If and how the enengaged vote may make a huge difference and is difficult to predict.

But based on this Survation survey, it looks like #Remain is ahead by around 12% and I'll take it.

Err, this is the specific table from the Survation thread that I am talking about, not the voting intention...