Any statistics genius here?

[historian]

We are being bombarded all day long with poll this, poll that etc... Do any of this have ANY value as a predictor of events?

[LuLu550]

1. Sorry,

I'm a mathmaticaly "moran"

[dubae524]

2. what do you think?

:)

[historian]

3. havent a clue

thats why im asking

[Dookus]

4. I think most statisticians will tell you

"it depends"

It depends on the poll itself, the wording, who's being polled, the size of the pool, and a gajillion other factors.

Polls also don't really predict future events - they take a snapshot of current thinking. If the event is in the very near future, it's reasonable to extrapolate from that snapshot.

But a poll of people's electoral preferences taken today will have little bearing on what happens in November. A poll taken the day before the election will be a lot more useful. Exit polls are VERY reliable.

[immoderate]

5. I'll try to simplify

If a sample is representative of a population, it takes about 1500 to predict how the population will respond, to within about 99% of confidence.

--IMM

[slackmaster]

6. Any poll where respondents select themselves has no value

Utterly worthless. All you get is a contest to see how many zealots on one side or another of an issue can rally themselves to "freep" or "un-freep" a poll.

Even most scientifically designed and executed polls have an underlying agenda. Follow the money.

[TahitiNut]

15. I guess that'd include an election, huh?

:eyes:

[Bucky]

7. No. Polls taken on DU only measure opinions on DU (and aren't scientific)

For a poll to mean anything, the people who respond to it must be randomly selected, not self-selected volunteers. About the only thing you can say about them is that they measure opinions of people who like to take surveys on this one website board--not a very important statistic.

If you're talking specifically about true public polls (that we only talk about on this board) saying who people plan to vote for for president this year, the answer is still the same. They mean very little because most voters aren't paying attention yet.

[David__77]

8. They're fairly accurate close to an election.

It seems that they have some validity because often the results are similar to pre-election polls. But I question the validity of, say, a presidential poll right now. It's hard to say "if the election were held today" when the conditions are not remotely similar today to those of a pre-election final stretch.

They also seem to underestimate "momentum." That is, they don't effectively capture emotional motivation. In CA in 1998, they underestimated Democratic turnout related to the impeachment. And in 2002, (barring those who claim electoral fraud) Republican turnout related to "war on terror" was underestimated. The neo-Confederate turnout was underestimated in Georgia that year, leading to surprise when Cleland and Barnes were defeated. Again, all this rests on the assumption of valid elections.

[LeahMira]

9. Only marginally...

The polls represent the opinions of those who self-selected to participate. With internet polls, you get responses only from those who have internet access, which, if you look at the Pew Internet research, is a specific group of people (generally middle to upper income, better educated, younger, and urban). With phone polls, you get responses from those who agree to participate. For any number of reasons, some people refuse (could be that the baby is crying, or could be that they just don't like phone polls).

There are so many other variables, like the way the questions are phrased and whether or not the survey basically asks the same question in a slightly different way, how the sample of participants is selected (by telephone area codes is the usual way), and even by the sound of the poll-taker's voice!

The internet surveys, I wouldn't give you ten cents for. Polls taken by some of the organizations that do that professionally, or polls set up through a university have a better chance of being accurate reflections of how the people are really thinking.

[alarcojon]

10. Mathematician here (close enough?)

Two things to consider:

1) all polls have a statistical margin of error, typically three or four percent (the exact value depends on the size of the sample).This is rarely commented on in the media, and most journalists ignore it completely. Suppose a poll says Kerry 47%, Bush 45% with a 3% margin of error; It's just saying that Kerry will be in the 44%-50% range 95% of the time, Bush in the 42%-48% range. In other words, Kerry could have an 8% lead, or Bush could have a 4% lead, or anything in between. When pundits give significance to day-to-day fluctuations of a percentage point or two, they are misunderstanding what the poll actually claims.

2) the mathematics behind polling is sound, but requires a truly representative sample of the population being polled. That is much easier said than done. For example, many people will hang up on a telephone pollster. If the type of people who hang up are not representative of the population, this will skew the results. Along these lines, I have even heard somewhere on DU that pollsters do not use cell phone numbers - that's GOT to mess up the sample.

[Stuckinthebush]

11. Educational Statistician...

It depends....

Polls are a way to collect survey information on a sample taken from a population. Sampling theory says that if the sample is truly random and representative of the population, then the opinions expressed in that sample will be representative of the population as a whole. Since we can never control for all error, then we are only confident that the results are representative of the population at a certain level. A sample has to be of a certain size (based on the population), and the sampling procedures must ensure that the sample is random in order to state that we are X% sure that the results are not due to chance alone. Also, there is another variable in sampling for polls - the margin of error. While we are X% confident (usually 95%) that the results aren't due to chance alone, then we have to determine how accurate we think the opinions are reflected in the sample. So, based on sample size and confidence level, we can then get a margin of error. If the sample size is large enough, and the procedures ensure randomness, then our results will fall in a margin of error band - say 3.5%. Therefore, we can then say that we are 95% confident that John Kerry has a population likability rating anywhere from 64.5% to 58.5% given a sample result of 61% likability.

All of this to say that polling is another form of social science research that is dependent upon random sampling. Keep in mind that true random sampling will never happen with a telephone poll because you can't have an equal chance of getting anyone in the population to respond to your poll.

Take polls with a grain of salt, but know that they can be very good at pointing us in the direction of public sentiment if good procedures are followed.

[central scrutinizer]

12. when the subjects are people, bias is almost impossible to eliminate

I teach mathematics, including statistics and the above posters are correct in their statements about the mathematical foundation of sampling. Besides the non-response bias (people hanging up) and the cell phone problem, other issues can introduce bias. You must see the exact wording of the question and look for loaded words - it is very hard to write a neutral question on a sensitive issue. On telephone polls, the voice of the questioner may influence the answer - suppose the questioner had a sultry feminine voice and the subject was a man - he might try to be engaging with her and tell her what he thought she wanted to hear rather than his true opinion. Would he answer differently if a man called? What if the man had a southern accent? or a Boston accent? or it sounded like a black woman and he was white? increasingly call centers are being outsourced to India - so how would he respond if he felt the questioner wasn't even a US citizen?

[depakid]

13. Ask yourself a few questions

First of all (not to reiterate what the above posters have said):

Who are they getting their data from?

Sampling errors have been the bane of political polls since the literary digest poll predicted Alf Landon would beat FDR in 1936. They also brought you Dewey defeats Truman and a host of other misadventures far too numerous to list. Even very slight, seemingly insignificant deviations from true random sampling introduces bias- which, by the way, can be predictable in the direction it skews the results- and so not always "bad" from a pollster's point of view, particularly if the client want to produce a certain set of results to support their agenda.

The points about respondents with a predisposition to hang up or surveyors not calling cell phones are well taken. A poll can be reliable within some confidence interval and still not give valid results. Happens all the time.

Where do they get there data from?

National political polls use some form of multistage clustering to draw their "random samples" from. Since political attitudes are correlated with regions (and districts within regions), getting a good "national" sample is tricky, to say the least. To get a feel for how this works, imagine a map of the US divided into regions and counties. Now, imagine that you're designing a national opinion poll on some contentious attitude or another. You have to draw your results from scattered counties all across the country to get what you hope will be a representative sample. It's not easy to do- if you inject your knowledge about the counties into the equation, you'll end up with bias. On the other hand, if you choose the wrong counties in the wrong regions, you don't get a representative population.

National polls are always suspect for this reason (and frequently worthless anyway, considering the nature of the electoral system).

As another poster pointed out, polls are very poor predictors of future behavior. One only has to look at this year's primary elections to see that. The closer they are to an election, they better they predict, but even then they are sometimes off by considerably more than their margin of error.

Given that many pollsters have an agenda, and that even when the best statistical methods are in place, the results can be manipulated by something so simple as the form or the order of the questions, you should always question what they're really telling you, why they're trying to tell it to you and whether what they're asserting accurately reflects reality. As often as not it doesn't.

One final point- even skewed polls can be useful for some things. They can be good indicators of trends, provided that you read the results with care and an ample dose of skepticism.

[damnyankee2601]

14. 5 years of Statistical Process Control

Only true random samples can be representative of a general population. You need a minimum of 20 samples and a maximum of 1000 for large populations.

No internet poll is in any way random, because it only polls those with computers who are alerted to the existence of the poll. This a very narrow and skewed representation of the American public since it leaves out the elderly, the poor, and those who don't watch TV news.

Professional pollsters take great pains to avoid this, and their stuff is as accurate as you can get. Their only skew factor is that they only get data from people who don't hang up on them. To mitigate this, they call people in smaller sub-groups until they get enough complete responses from each group.

There is also a real art to creating a poll that does not accidentally become a "push" poll. That is, questions that force certain answers. So once again, the professional poll companies employ some very good pshycologists.

Don't trust any poll from any group with an agenda. The only data with any validity is the data people PAY for.

[TahitiNut]

16. The myth of "random" sampling.

While a "random" sampling methodology is (more than) sufficient, it's not necessary for the survey of the sample to validly represent the population from which it's sampled within some stochastic margin of error. All that's necessary is that there be no dependence on the characteristics being surveyed inherent in the sampling methodology.

Virtually all sampling is anything but "random". All computer algorithms, for example, are deterministic and not really 'random'. It's merely that they're deterministic based on parameters that don't have a relationship (we hope) to the characteristics being surveyed.


Telephone polls, no matter how "random" the selection of number, are inherently biased with respect to surveying anything economic. Affluent households are more likely to have one or more telephones and call-waiting, and are (arguably) more likely to have someone at home. Telephone polls (due to refusal rates exceeding 50%) are also inherently biased with respect to political issues associated with privacy -- a bias that's somewhat indeterminate. It's speculated/asserted that professional polling organizations have methods for normalizing the bias inherent in this polling method and sampling methodology, but because the results of virtually identical polls vary beyond their margins of error, it's arguable whether such normalization techniques themselves embody biases.