This will probably be interesting to absolutely nobody who isn’t me, but here is a long-ish thread on how we run our Labour members polling:
Step 1 is identifying Labour party members to be surveyed. We do this by running short “mega-polls” to the YouGov panel (which is why we don’t need the permission of the Labour party, or the access to their membership lists).
The scale of this is a little bit crazy, we have probably asked at least a million people if they are members of the Labour party since 2015, and hundreds of thousands since the January cut-off date.
The next step, which is a bit trickier, is making the survey representative. Unlike surveys of the whole population there are no publicly available targets to weight to, so we don’t really know how they breakdown by age/gender/region etc.
Thankfully, the party has a habit of leaking this information to journalists from time to time. E.g. this from @PippaCrerar has been very useful: mirror.co.uk/news/politics/…
In addition, we can weight those original “mega-polls” to be representative of the overall population, and then look at how Labour members break down inside that. This helps to beat out some biases that might otherwise exist.
Finally, I have gone back and re-surveyed some of the respondents from our polling in the 2016 leadership campaign. This allows us to look at the churn among the membership (how many people have joined / left since the party since then, and what kind of people are they?).
Yesterday’s poll for @SamCoatesSky was weighted by age, gender, region, and membership length for full members. The affiliates section was weighted by region and Trade Union.
We also used turnout figures from 2015/16 to work out how big each part of the “selectorate” should be, adjusting for the fact that the number of full party members has grown enormously since then.
How accurate is this method? It has worked really well in the past, and was pretty spot on in 2015 and 2016. However, as with all polling, there is still a lot of room for error, and the lack of official weighting targets does make it a lot trickier.
On the other hand, having biases towards groups (old people, graduates etc.) will matter less with members because they are a fairly homogeneous group. You also get around the issue of polling being skewed towards the politically engaged – because they are all politically engaged
Despite this (and previous performance), over the long run I would still expect membership polling to be slightly less accurate than generic polling. Thankfully, it doesn’t matter this time around because the results are so decisive.
