One more Polling Primer. There is a good chance that NY polls are off

[Godhumor]

I haven't done one of these in a long time, so for the newer posters, I'm a statistical analyst in real life and have done various primers on understanding the ins and outs of polls over the course of the campaign. I am a Clinton supporter and don't shy away from it, but I try to do these posts as non-partisan as possible.

Let's get the elephant in the room out of the way, first. Polling has not been very good this cycle. I know a lot of people believe that based on what they've seen, but recently 538 quantified it as, on average, polling has been off by an absolute 11 points (Absolute being how far off reality either + or -). That is just not good. At all. As a statistical analyst, and one who believes firmly in scientific polling, this has not been a pleasant cycle. There will be a reviewing of methodology at quite a few firms when all is said and done.

On top of the just general mediocrity in the industry this year, NY polling presents some additional challenges. Primarily the very stringent registration requirements that, frankly, not all voters are that well versed in. And that presents issues for polling outfits.

Most pollsters ask respondents to self-identify a party preference and likelihood to vote in a primary (Generally combined with past voting behavior if applicable). Hopefully, the issue with this method is already obvious--there will be people who self identify as Democrats and believe they will vote who are not actually able to do so. Depending on the sample size and methodology used, this may actually be a substantial portion of the respondents.

A good example of this is actually from the NBC poll yesterday. Among likely voters Clinton is up by 14. Among potential electorate her percentage drops to 7. The difference between these numbers can be described as "Likely voters are those who we know are able to vote in the primary and are most likely going to do so" while potential electorate is "All people who say they can and may vote, but we can't confirm they're all able to do so or will turn out."

So how do firms identify those they are sure can turn out? There are many different ways, but the primary method used is calling from registration rolls for the party. In other words, polling people who are verified by the party that they are registered.

Polls generally don't use registration rolls for 100% of their contacts for many reasons (Including that it is expensive), but, for closed primaries like NY, the more respondents who are verified as registered, the more likely the poll is to be closer to the eventual reality. Polls that are primarily relying on self-id for verification will have ineligible voters responding.

Right now, polling shows the race around a 10-15 point lead for Clinton, which may hold up, obviously, but it would not be a shock, considering the poor polling cycle and the intricacies of the NY system, if the final margin is different from the polls. My biggest hope is that, if polling is off, some firms make adjustments before the following week's closed primary onslaught.

And, yes, I finally get to vote here in NY. Can't wait to do so.

[hrmjustin]

1. I think it could be a 20% win for Hillary here.

I would say more than half of the Bernie people i know are indys and can't vote here.

Of course we will find out on Tuesday.

But remember that NY law says absentee ballots don't get counted until 9 days after election day and the absentees will favor Hillary so the final margin will take awhile to find out.

[Yurovsky]

6. Wall Street is certainly hoping so...

they don't want to wake up the day after and realize they pissed away millions of dollars on a front-runner who pulled up lame in crunch time.

[geek tragedy]

2. I thought Marist screened for registration but maybe not.

Monmouth does and they produced a 12 point margin.

Query whether this is poll herding.

I think it's Clinton +18 here when all is said and done--Sanders running slightly lower than Obama did.

[Godhumor]

3. Yes they screen for registration

I believe at around 50-60% of respondents.

Why I said the more screened the more likely to be accurate. The question I have is whether the traditional 50-60% registration screen will be enough to capture a real sample of likely voters. I have my doubts, but we'll see how next week plays out.

[geek tragedy]

4. it's tough to peg likely voters here because we have low turnout

even amongst eligible voters and because there's still machine politics here.

In 2008 he had smaller turnout than Ohio.

[firebrand80]

5. It's also my understanding

That it's difficult to poll states that don't have competitive primaries very often, because there aren't any good turnout models to work from.

I think that's a big reason why MI was screwed up so badly, we had that weird thing happen in the Dem primary in '08, so there really wasn't a race there.

[LonePirate]

7. Do you know if any analysis has been performed related to primary v. caucus or open v. closed?

I suspect closed primaries are easier to poll or predict than open primaries and primaries are easier to poll than caucuses.

[Godhumor]

9. You've got the order correct with one caveat

Generally, for polling primaries on order of difficulty are ranked as follows:

Caucuses of all kinds - nearly impossible (Why most firms no longer even bother with them. Not worth the time or expense to end up still being wrong)

Open Primaries

Closed Primaries

The caveat is that the accuracy of closed primary polling is highly correlated on how good the voter screen is. If the screen is wrong, the poll is off. For a state like NY with two different due dates to consider for registration, there will be some outfits that half-ass or just screw up their verification screen.

[1StrongBlackMan]

8. Excellent primer ...

that will be read by approximately 1 in 22.7, and read and understood by approximately 1 in 70.2.

[Lucinda]

10. Thankie for the info!