Thread by @Nate_Cohn

Well now that I'll have everyone's attention, this is the right time to talk about some slight weighting changes we'll be making in (primarily) Sun Belt districts, which were announced yesterday in this piece and will be implemented today nytimes.com/2018/09/20/ups…

Basically, most of our polls are weighted pretty heavily by fairly granular categories.
In the Sun Belt, that's been an issue.
The added diversity of those CDs = more categories.
The samples have been particularly tough, so more weighting needed too

If you've been following closely over the last few weeks, you might have noticed something called our "design effect," a measure of how much we're weighting. We average about a 1.2. Over 1.4 is bad, and generates warnings on the page.

In TX07/FL26, we spent a lot of time with a design effect far over 1.4 and ultimately got back under 1.4 in the end. In TX32, we're stuck at an atrocious 1.56 at n=406, and more sampling probably won't fix it as easily

The changes announced yesterday deal with this problem. We're going to weight by just slightly broader categories of age and education. We're also moving to stratify the sample by more categories than we have in the past.

All of this is really quite minor stuff. But we're going to implement these changes in TX32 today, midstream, in part because it's been the toughest district yet of these polls. We're dropping from 4 categories of age to 3 categories of age, and 4 categories of education to 3

We're also redefining the way we calculate age, by taking the self-reported age of our respondents, not voter file age.
Basically: our response rate among young voters is so bad in these few CDs that our young voters are contaminated by parents taking polls for their kids

In our summer testing, we found that >98% of our respondents are the people who we want.
But if we get a response rate of 1/250 among young people, that 2% measurement error starts to be a significant portion of the category

Anyway, these changes were already planned based on prior tough Sun Belt CDs. They're good moves, based on the underlying data not effect on the results (which I didn't even simulate in our prior CDs bc I don't want my judgment about method to be effected by this)

We're not going to change anything about the display. Even if we weight by 18-39 year olds, we'll still show you our terrible response among 18-29 year olds in these districts.

Anyway, we're going to push these changes in TX32. The results will move.
The main thing I'm watching: the design effect. We're at ~1.6. The goal is to be less than 1.4, which is the 'warning' criteria
How much closer will we get by going from 4 to 3 categories of edu and age?