The Iowa results confirm that the methodology used by PPP, Gravis, and Emerson is deeply flawed

[Attorney in Texas]

Last edited Wed Feb 3, 2016, 09:18 PM - Edit history (2)

Now that we have some actual results to grade the pollsters, it is worth considering which polls were close and which polls were ridiculously off the mark so we can discount the junk polls as we go forward.

Which polls were closest to the target according to the data at Pollster and Real Clear Politics?

CBS/YouGov predicted a 1% gap between Sanders and Clinton within the margin of error.

Quinnipiac predicted a 3% gap between Sanders and Clinton within the margin of error.

Iowa State predicted a 2% gap between Sanders and Clinton within the margin of error (but the data was gathered over an unusually long 17 day period).

Des Moines Register predicted a 3% gap between Sanders and Clinton within the margin of error.

ARG predicted a 3% gap between Sanders and Clinton within the margin of error.

NBC/WSJ/Marist College predicted a 3% gap between Sanders and Clinton within the margin of error.

Fox predicted a 3% gap between Sanders and Clinton within the margin of error.

Which poll missed the mark, but was not too far outside its margin of error:

Monmouth predicted a 5% gap between Sanders and Clinton outside its 4.4% margin of error.

Which pollsters were full of shit?

Gravis predicted an 11% Clinton lead with a 3% margin of error.

PPP predicted an 8% Clinton lead with a 3.4% margin of error.

Emerson predicted an 8% Clinton lead with a 5.6% margin of error.

What separates the bullshit polls from the most accurate polls?

All of the bullshit polls use a dubious robo-call polling method.

The more accurate polls use a live landline and call phone polling method (except CBS/YouGov, which uses a on-line model).

What does this tell us going forward?

Here is the race in New Hampshire according to all polls:



Here is the race in New Hampshire according to the robo-call polls:



Here is the race in New Hampshire according to all polls except the robo-call polls:



The robo-call polling is untrustworthy and when it is included in a poll aggregation, it skews the aggregation.

BEWARE THE COMING ONSLAUGHT OF NEVADA ROBO-CALL POLLS!

[72DejaVu]

1. Mostly it tells us that you can't really poll caucuses.

[Attorney in Texas]

2. No, that's not it because all of the robo-call polling erred in the same direction (for some reason

robo-call polls err consistently in favor of Trump on the Republican side and in favor of Clinton on the Democratic side). This has been a consistent pattern through the election cycle. It holds true in caucus states and primary states. If you look at the robo-call versus non-robo-call polling in New Hampshire, for example, the gap is 4 times bigger in the non-robo call polls. This puts the robo-call polls outside of the margin of error for the other polls and vice versa -- i.e., they cannot both be right. Iowa tells us that it is the robo-call polls which are wrong and the other polling which is correct.

[OhZone]

10. Maybe, I do think it's largely the problem with polling caucuses -

and not being able to predict what the turn out will be. And the polls swung wildly.

[Chitown Kev]

13. I don't think that the polls swung all that wildly

I think that it pretty much was a simple case of Sanders piking up O'Malley's support (which was ~4% in most polls)...that pretty much accounts for the difference.

[firebrand80]

3. I get the sense

That the polls really weren't that far from each other, the various results come down to the turnout that is being projected.

I got the overall impression from IA polling that Hillary would win narrowly, and that's what happened.

In that sense I think the polling, taken as a whole, gave me a pretty good idea about where the race was at.

[Attorney in Texas]

4. Not really. The robo-polls said she's win by a huge margin. The non-robo-polls said it was a virtual

tie with Clinton ahead by an almost immeasurably small fraction.

The end result was within the margin of error for the non-robo-call polls and the end result was well outside of the margin of error for the robo-call polls (and this is true whether you look at Gravis, PPP, etc. -- all the robo-call polls had Clinton ahead by considerable margins).

[firebrand80]

5. I see what you're saying

Some of those polls were way off.

I was talking more about how I look at the polling data. I take a step back and look at the big picture; those late polls with huge margins were pretty clearly outliers.

[Attorney in Texas]

8. Exactly. They were outliers and they had one thing in common -- they were all robo-call polls

[uponit7771]

6. Good to know, all robo call polls skewed in same direction too... very good data and a reason to

...devalue robo calling polls

[Attorney in Texas]

7. Nevada has been underpolled. It is easy to pay Gravis or PPP or someone to do a cheap and dirty

robo-poll in Nevada. I will be shocked if we don't see a bunch of poor quality polling in Nevada in the next two weeks. I'd like to get ahead of that polling and call bullshit now before the results are in so there can be no question that the complaint is not based on who comes out ahead but is based on the crappy methodology.

[Attorney in Texas]

9. title changed

[Vote2016]

11. kick & rec

[reformist2]

12. Exactly. If Gravis and PPP are more than 8 off again in NH, they should be dropped from consideratio

[AtomicKitten]

14. This crap polling is meant to screw with the aggregate.

Mark Penn is doing polling for her behind the scenes "throwing polls out there" to make it appear she's doing better than she is.
http://www.politico.com/story/2015/10/clinton-mark-penn-polling-214637