We Might Be Seeing Some Herding From Pollsters

[DemocratSinceBirth]

The accuracy of my election forecasts depend on the accuracy of the presidential polls. As such, a major concern heading into Election Day is the possibility that polling firms, out of fear of being wrong, are looking at the results of other published surveys and weighting or adjusting their own results to match. If pollsters are engaging in this sort of herding behavior – and, as a consequence, converging on the wrong estimates of public opinion – then there is danger of the polls becoming collectively biased.

To see whether this is happening, I’ll plot the absolute value of the state polls’ error, over time. (The error is the difference between a poll’s reported proportion supporting Obama, and my model’s estimate of the “true” population proportion.) Herding would be indicated by a decline in the average survey error towards zero – representing no difference from the consensus mean – over the course of the campaign. This is exactly what we find. Although there has always been a large amount of variation in the polls, the underlying trend – as shown by the lowess smoother line, in blue – reveals that the average error in the polls started at 1.5% in early May, but is now down to 0.9%.

http://votamatic.org/

[democrat_patriot]

1. Translation?

[DemocratSinceBirth]

2. Pollsters Are Looking At Each Other's Work And Adjusting Their Surveys Accordingly

Silver said he saw the same thing in 08.

[geek tragedy]

4. This would give hope to the Republicans, no?

They could think that the state polls have been herding all along.

[smorkingapple]

3. Then this throws off a lot of the aggregators and nerds right?

I don't like to read that. Maybe it doesn't mean much in the end but if it means all polling in recent week plus is biased in same direction we COULD be in for a surprise tomorrow. I wonder how Silver and Wang etc account for this at all.

[DemocratSinceBirth]

5. I Think Nate Addressed It

It is what it is. It's something to be concerned about but not overly concerned.

But the agggegators aren't getting worked up it so neither am I...

And just a cursory review of the data suggests the herding effect isn't that strong at the state level.


I do think the nat'l pollsters are afraid to be wrong.

[DollarBillHines]

6. I heard him talking about that on NPR

I'm going with the odds-makers.

I'll take money over fear, anytime.

[DemocratSinceBirth]

7. Me Too

Discussing the herding effect is a nice topic for a Poli Sci. paper but for almost all the polls to be wrong in one direction defies credulity.