Polling MOE versus probability that the difference is greater than zero

[papau]

From a pure statistical standpoint, a lead is a lead and it's always better to be ahead than behind - so why does the media do a mythical "statistical tie"?

http://www.washingtonmonthly.com/

Polling MOE versus probability that the difference is greater than zero


<snip>In fact, what we're really interested in is the probability that the difference is greater than zero — in other words, that one candidate is genuinely ahead of the other. But this probability isn't a cutoff, it's a continuum: the bigger the lead, the more likely that someone is ahead and that the result isn't just a polling fluke. So instead of lazily reporting any result within the MOE as a "tie," which is statistically wrong anyway, it would be more informative to just go ahead and tell us how probable it is that a candidate is really ahead. As a service to humanity, here's a table that tells you:



So in the poll quoted above, how probable is it that Kerry is really ahead? The MOE of the poll is 3%, so go to the top row. Kerry's lead is 2%, which means there's a 75% probability that he's genuinely ahead of Bush (i.e., that his lead in the poll isn't just due to sampling error).

Generally speaking, national polls use sample sizes of about 1,100, which translates to an MOE of 3%. State polls often use a sample of 600, which produces an MOE of 4%. Subsets of polls sometimes have MOEs of 5% or higher.

Now, there are plenty of reasons other than sampling error to take polls with a grain of salt: they're just snapshots in time, the results are often sensitive to question wording or question ordering, it's increasingly hard to get representative samples these days, etc. etc. <snip>