There are two ways to handle party ID in a poll
Last edited Mon Sep 24, 2012, 04:22 PM - Edit history (1)
There are two ways to handle party ID in a poll, and neither one of them is quite right, or wrong.
Who picks up the phone and talks to pollsters is not randomly distributed so you can call people for a century and still not have a representative sample.
Pollsters improve their accuracy by "weighting" certain factors. We know from censuses and public records and thousands of other polls that the electorate is 51-52% female (or whatever the exact % is). If our sample happens to be only 48% female we weight that 48% as counting for 51% of the result, and reduce the weight of male sample accordingly. We don't throw out any body's answers. We just count them a little more or less to better simulate a perfect sample.
Weighted factors can include age, gender, race, and recently cell-phones. Weighted factors tend to be fairly stable things. The percentage of the American population that self-identifies as African-American changes over years... 12%, 13%, 14%, 13%. But it will never go from 9% to 15% to 11% in three monthly polls. Our sample of AA might do that, though, so we adjust those numbers so that whatever AA opinion we capture counts as 13% of the total. It's not perfect, but it is more accurate. Cell phones, latinos and 65 year-olds are all on the rise and we look to the totality of current information to see how much to weight them. They change, but they do not jump around wildly.
Party identification is one of the key demographics in any political poll but, unlike gender and race, self-reported Party ID often jumps around as much as whatever the poll is measuring. It's a self-reported polling answer and formal party-registration is a lesser and lesser concept. Not all states have it. And some registered Dems have voted straight Republican since 1980 and visa-versa... what do they think of themselves as?
So how do we handle Party ID?
We can go with whatever people say in the poll (unweighted) or we can say, "In the last election the exit polls said Dems were X% and Pugs were Y%" and weight our results to match that. But the exit poll is still a poll. And maybe the last election isn't representative of this election. So we come up with the most sophisticated estimate of Party ID we can and norm to that.
But what if Party ID is changing? During Watergate it seems likely that fewer people were identifying themselves as Republicans. The people still telling pollsters they were Republican were die-hards. Weighting those die-hards up to the "true" Republican number would create results surprisingly favorable to Nixon.
But if you do not weight the Party sample then people would look at the poll and say, "Geez... why did you only poll Democrats?"
And what if you did simply have a bad sample that did include too many Democrats? How would you know? If you had a bad sample that was 65% female you wouldn't just shrug and say, "the sample is the sample. Go with it."
Weighting Party ID is like using a yardstick to measure a yardstick. It's circular, but also sensible... as long as the yardstick is still three feet long.
So what is the right thing to do? Weight Party ID or not?
The answer is, both. It is best for some polls to do it one way and other polls to do it the other way.
No one poll will be perfect. Over time, if the weighted and unweighted polls all diverge in a certain direction, the weighting formulas have to change.
Just one more reasons averages of polls out-perform the average poll.
(Throughout this OP, "we" means we as a society in 2012. I am not a pollster.)
= new reply since forum marked as read
