Polling NH: What Happened ? (Zogby analysis)

[TwilightZone]

Polling the New Hampshire Primaries: What Happened? - John Zogby

There was no shortage of polls going into the New Hampshire primary in 2008 and it looks like we all missed the mark on the Democratic side. This will require a lot of scrutiny in the coming days and weeks, but here are some initial thoughts on what has been happening:

1. According to the exit polls, 18% of the voters said that they made up their minds on primary day. That is just an unprecedented number. I have polled many races, especially close ones, where 4% to 8% have said they finally decided on their vote the day of the election and that can wreak havoc on those of us who are in the business of capturing pre-election movements and trends. But nearly one in five this time?

2. It looks like the always feisty voters in both Iowa and New Hampshire have rejected pre-election coronations. In the case of Iowa, Democratic voters said that Mrs. Clinton is not inevitable, while in New Hampshire they were not ready to endorse the Obama train without checking the engine.

3. The compressed schedule of the two events may have had an impact. Normally the winning candidate gets an initial big bounce out of Iowa, and then plateaus. Then the next primary race begins. With less than five full days, Obama got his bounce in New Hampshire, then the settling down period began on the last day -- under the radar screen.

4. My polling showed Clinton doing well on the late Sunday night and all day Monday -- she was in a 2-point race in that portion of the polling. But since our methods call for a three-day rolling average, we had to legitimately factor the huge Obama numbers on Friday and Saturday -- thus his 12 point average lead. Unfortunately, one day or a day-and-a-half does not make a trend and we ran out of time.

More: http://www.huffingtonpost.com/john-zogby/polling-the-new-hampshire_b_80657.html

[Skip Intro]

1. You got it wrong big time, zomby, that's what happened.

[Renew Deal]

6. He says it was a 2 point race on the last day of polling

It was the rolling averages that blew up the lead.

[TwilightZone]

9. That's a huge swing, in the 15-point range.

To average 12% over three days, the total of the lead for the three days would need to be 36. Subtract Obama's two-point lead on the last day (according to Zogby), and Obama's average for the prior two days would have been a 17-point lead (the remaining 34 divided by 2).

That's a huge swing. If indeed the polls were showing that, somebody probably should have mentioned it, even if protocol indicates otherwise.

[Renew Deal]

11. They were using a 5 day rolling average

Which I thought was a bad idea, considering there was only a 5 day window.

[TwilightZone]

12. Zogby says that his was only three days:

But since our methods call for a three-day rolling average, we had to legitimately factor the huge Obama numbers on Friday and Saturday -- thus his 12 point average lead. Unfortunately, one day or a day-and-a-half does not make a trend and we ran out of time.

[Renew Deal]

13. He did say that.

I was thinking of the CSPAN poll.

[TwilightZone]

15. You're right, though. A five-day average in context isn't much better.

It's just not enough time to gauge the post-Iowa bump and the subsequent leveling off.

Assuming all of this is correct, Zogby is probably right about one thing - they ran out of time to get an accurate picture. (Big assumption, I know!)

[Anouka]

21. which means? n/t

[TwilightZone]

26. I'm sorry...which part?

My point was essentially that if we take what Zogby claims at face value (a big assumption, I'll admit), there was an indication that the race was swinging significantly in Hillary's favor at the very end of the polling periods.

Had anyone bothered to mention that beforehand, the outcome wouldn't have been quite as unexpected.

It's irrelevant, of course, but if it's true, it would partly explain the huge discrepancy between the rolling average polling data and the result.

[protect our future]

25. TwilightZone, I do political polling too but

I don't know how the margin of error is computed. Can you enlighten me?

OF COURSE beyond a shadow of a doubt the latest poll results should have been given a serious look, but I suppose using a three-day average is the way it has been done before so that's the way it continues to be done. Dumb, in this particular situation and you were right to question it.

The three-day average makes little sense when you realize that so many New Hampshire voters were suddenly besieged with three superb viable Dem candidates and besieged with -- how many pollsters from how many organizations? Six phone calls a day? Ten? And how ANNOYING that can be when you need a chance to assess the candidates one more time and make a final decision! Making that final decision had to be hard cuz, as I said, three great candidates. All viable!

Betcha pollsters got a multitude of hang-ups and *none of your businesses.* Which you get in any political poll, but I'm guessing even more than usual in New Hampshire (and who can blame them!). How can these people possibly be accounted for? What about voters that maybe said they were leaning toward one candidate or another but not completely sure? Are they listed as undecided or as definite? And what about people who never answer their phone or all you get is an answering machine no matter how many times you try to call?

Thanks for any answer. I posted my take on how and why things went down as they did in another thread but I didn't talk about the polling.

on edit: I forgot to ask you something else. What about new voters who have recently registered? Where I live they often fail to be added to the lists of voters in time to be polled, so it's as if they're invisible. Could that have happened too?

[TwilightZone]

29. Those are some very good points.

MOE is a statistical calculation that depends on sample size. I don't remember enough of my stats classes to recall the formulas, but you're correct in that it is certainly a factor and needs to be taken into account.

Margin of error is rarely considered by the media or anyone else, for that matter. We all tend to just ignore it and just look at the baseline numbers as if they represented exact numbers. If a poll has a 5% MOE, that's an enormous potential swing between prediction and reality.

That being said, the MOE in most of the NH polls wasn't enough on its own to account for the swing.

Regarding the three-day averages, it's a catch-22 from the polling perspective. Rolling averages are used to blunt the effects of a single-day aberration, but at the same time, they can miss momentum shifts, as seems to be the case here. In NH, something like 40% of the voters didn't firm up their selections until the last three days. There's just no way that a three-day rolling average can account for that. Not even a final, single-day poll can account for 20% of the people deciding on primary day.

And, just like you said, pollsters are only getting data from people who actually answer the phone and agree to take the poll. They're not getting data from all sources or new voters or people who never answer the phone, etc.

Someone in another thread said that polls aren't predictors, they're just snapshots of a moment in time. I agree with that and I think that the NH results confirmed that. The polls were probably more accurate than it would appear, but only for the time period that they reflected. They obviously didn't pick up the last-minute changes or brand-new voters, and they could have oversampled independents or undersampled women.

The exit polls and actual results are the only real data we have, and according to reports, they match pretty closely.

[protect our future]

33. "Shapshots in time." What a perfect description.

Thanks, you have been very helpful in satisfying my curiosity. But do the voters who are leaning one way or another yet not absolutely positive get counted though? Sometimes voters who say they're undecided will tell you who they think they'll vote for but it's not etched in stone. Or do they even get asked if they're *leaning*? Again, just curiosity. I may do the polling but have nothing to do with the interpretation of the results.

[TwilightZone]

34. Some polls do ask about "leaners".

The Marist poll, for example, asked people for a definite choice, but also included questions intended to include the people leaning toward a candidate, but not firmly dedicated to him/her yet. They published "with leaner" and "without leaner" numbers.

The polls also generally have categories for undecided and refused. Most don't include refused unless a respondent answers some questions but refuses to answer a specific one. For example, they may answer all of the questions about specific candidates, but decline to answer a question about their religious affiliation.

About the best way to learn more about the polls is to read the accompanying info. Here's a link to the Marist poll I mentioned above: http://www.maristpoll.marist.edu/NH/NHPZ080107.pdf

Here's a link to some other recent polls: http://www.realclearpolitics.com/epolls/2008/latestpolls/index.html

Enjoy, and welcome to DU, by the way. You picked a pretty wild and crazy time to join. Don't let the, uh, passion scare you off. :)

[protect our future]

35. Thanks for the welcome!

I've been lurking since '04, then finally joined but mostly continued to lurk passively. Lurking here is a great way to get all the latest news that's fit, or decidedly unfit, to print! And thanks for more good info on polling.

[TwilightZone]

36. In between all the craziness, DU is definitely a good place to keep on top of things.

If it's news, especially political news, it will end up here almost before it even happens.

Maybe that's why we keep coming back.

Or, maybe it's the craziness. It's like a train wreck. Can't...look...away. Hehe...

Take care.

[protect our future]

38. And amidst all the entertainment here ...

... there are posts by some of the most brilliant people I've ever had the privilege of lurking around and rampant humor equalling whatever those striking writers could dream up.

[TwilightZone]

39. Agreed.

There is definitely a full spectrum of just about everything here.

[DemocratSinceBirth]

19. I Am Glad That The Zomby Tagline Is Catching On

That and Republimussen...

[rinsd]

27. You trend setter you!

:hi:

[rinsd]

2. Kick for my fellow poll junkies

[Renew Deal]

7. You must have seen the Limbaugh thread,

Oh wait :evilgrin:

[rinsd]

16. !!

:spray:

[bleever]

3. K&R for the discussion.

(There's still no excuse for Diebold servers being employed.)

[BootinUp]

4. He couldn't forsee any of that from his polling data? nt

[TwilightZone]

5. If we take what he says at face value, he did see it to some extent.

Particularly item #4. His polling showed a big swing in Hillary's direction at the last minute, but the three-day averaging minimized its appearance.

[BootinUp]

8. So he should have reported it with the results

if he was worth his salt.

[TwilightZone]

10. That was my thought:

http://www.democraticunderground.com/discuss/duboard.php?az=show_mesg&forum=132&topic_id=4011969&mesg_id=4012175

[indimuse]

14. Did they even poll after THE HILLARY ...so called

BREAKDOWN???? NO!!!!!!!!!!!!!!!!!!

[TwilightZone]

17. I hate to beg, but can I get one more recommendation for this thread, please?

Shameless, I know. But, with all of the talk about the polls, I think that this is information that should be seen, if not necessarily believed.

[BootinUp]

18. K&R

[TwilightZone]

20. ...

Thanks! :)

[killbotfactory]

22. Clinton was ahead before the Iowa bump

and It looks like things were returning to normal after the weekend, but all the pollsters extrapolated the opposite way.

[adapa]

23. As a NH voter, I reserve the right to make up my mind when I walk into the booth

My decision will be personal,
it will be informed
it will be reasoned and
with the best interest of our country at heart

Personal means just that-personal as in private if I so choose.

[TwilightZone]

30. That's a very important point, and in NH, it seems to be a common one.

Poll data is only accurate if it truly represents the base that it is sampling. If most people choose not to participate or provide misleading information to protect privacy, the data just isn't going to be representative of reality.

[SIMPLYB1980]

24. Hillary won.

[DangerDave921]

28. excellent

I like the point about the condensed time of these primaries. The system is really out of whack, and we make these tiny states disproportionately important.

NH prides itself on being 1st (even making it a matter of state law to keep that "honor") even if it means having to back up the primary to god knows when. All the states end up piling together, the race gets distorted, the media gets all flumoxed, and our choices get screwy.

So we race like hell to pick the candidate, then have 9 months of primary election season. I'd prefer more time in the primaries to really see the candidates with actual voting going on.

[TwilightZone]

31. In a way, the outcome in NH made the IA/NH dynamic less important.

At least less important than it looked like it was going to be. Not that the media didn't still overreact itself into a frenzy.

I like the idea of spreading out the primaries, but what the heck would we do with Super Tuesday?

[DangerDave921]

42. Or

Or we could do all the primaries on the same date, just like we do the national election. Everyone votes at the same time. That way, no one would claim undue influence or media coverage as a result of an early win. Because the early wins are really just important for the media exposure and the increased donations.

OR: We could ban all freaking polls. They are useless. Worse than useless; they actually affect the way people vote. Who the F--K cares about a poll done in October 2007 about who would win a fight between Rudy and Hillary? What possible role can that serve other than to give the media something to talk about?

[TwilightZone]

43. The problem with same-date primaries is that the small states would disappear.

The candidates would focus on NY, CA, FL, TX, OH, MI, etc., and the small state primaries would be meaningless.

It's a catch-22, of course, and I'm not sure what the answer is. Maybe we could have three primaries a week, one small, one mid-sized, and one large state, and just spread it out over 17 weeks. It would still be over in about four months, and that would seem to eliminate some problems from each scenario. Probably makes too much sense.

Polls aren't predictors, and NH was the best example yet. They're just snapshots, and they simply cannot account for the late surge in voters, new voters, last-minute mind-changing, etc. I don't know what the solution is there.

[adapa]

46. Plus a candidate will have to have massive amounts of money-

One of the major reasons having the first primary's in small states is the relatively low cash buy in for lesser known candidates.
If if was a 50 state primary the Bidens&Dodds would be out.

[TwilightZone]

32. ...

:)

[TwilightZone]

37. ...

[RUMMYisFROSTED]

40. :tfh:

[Lucinda]

41. Thanks. I cant beleive they didn't post the poll without some sort caveat.

The polls of the two - three day period around the election also factor in. The registered Dem numbers were very close.

[Rydz777]

44. What a jerk! Zomby gets paid for explaining his failures.

[mhatrw]

45. Response.

1. According to the exit polls, 18% of the voters said that they made up their minds on primary day. That is just an unprecedented number. I have polled many races, especially close ones, where 4% to 8% have said they finally decided on their vote the day of the election and that can wreak havoc on those of us who are in the business of capturing pre-election movements and trends. But nearly one in five this time?

Yet of all of these "last day decision" voters, slightly more said they decided for Obama than Clinton. So why do you believe the percentage, but not who the percentage said they decided to vote for?

2. It looks like the always feisty voters in both Iowa and New Hampshire have rejected pre-election coronations. In the case of Iowa, Democratic voters said that Mrs. Clinton is not inevitable, while in New Hampshire they were not ready to endorse the Obama train without checking the engine.

So they were just being contrary to the pollsters as well?

3. The compressed schedule of the two events may have had an impact. Normally the winning candidate gets an initial big bounce out of Iowa, and then plateaus. Then the next primary race begins. With less than five full days, Obama got his bounce in New Hampshire, then the settling down period began on the last day -- under the radar screen.

What a difference 24 hours makes!

4. My polling showed Clinton doing well on the late Sunday night and all day Monday -- she was in a 2-point race in that portion of the polling. But since our methods call for a three-day rolling average, we had to legitimately factor the huge Obama numbers on Friday and Saturday -- thus his 12 point average lead. Unfortunately, one day or a day-and-a-half does not make a trend and we ran out of time.

How many people did you poll during that time, Zogby? My guess is 400 at the very most.

[adapa]

47. Analysis from men=Racist voting, Analysis from Women=empathey

I've been listening to the talking heads & NPR way too much lately but an odd trend is surfacing.

When the guy's talk about the polling mess up in NH, they by & large attribute it to NH residents being raciest.

When women talk, they attribute Clinton's win to a last min surge.
Hillary not only said why she wanted to be president but also said she cared about our problems. No more the cold Ice woman, She hears us. The guys just don't get how much that spoke to all the caregivers out here. By caregivers, I'm talking all the women who take care of other people, family, parents, friends etc all.

The exit polls says
Hillary cares about people 41% to Obama's 19%
Hillary won married mom's 42% to Obama's 34%
Hillary won unmaried & w/out kids women 49% to Obama's 31%
Hillary won white women 46% to obama 33%
Hillary won non-white women 50% to Obama's 38%

You can laugh all you want, guys, but when Hillary explained why she wanted to be Prez in that heartfelt moment in the diner, the race changed.