What my Ph.D. professor of research methods says about Gallup

[kstewart33]

Another poster said this morning that Gallup on NPR said that they do their random calls, and FIRST used the questions that determine likely voters, and THEN they ask the voter who they're going to vote for, and THEN they ask the party. They pointed out that people planning on voting for GWB are more likely to identify themselves as Republican.

My husband is a professor of marketing and statistics (Ph.D.) who teaches survey methodology.

He says: A random telephone survey is never actually random. When you make the calls, some people are more likely to be home and participate in the survey (e.g., unemployed people, the elderly, etc.). So already your sample of respondents is biased. Zogby uses the known percent of voters from past elections to counteract this bias. Gallup is essentially saying that they ignore this bias.

It is not a random sample. For some reason, more people who are Republicans are participating in the survey. Given that the percentage of Republicans is high in the survey, the sample in no way reflects actual voter participation, and is simply biased. The interesting question is how or why is this the case? Gallup's answer is lame. They are pretending that their methodology is objective when it is not. Gallup hasn't answered the key question of why this is happening. It's a bit like shrugging your shoulders and saying, oh things like this just happen. That is inexcusable for a polling organization that should be experts in how to achieve a representative sample for a survey of tremendous importance.

[stellanoir]

1. thanks

for the clarification.

Might your husband consider e-mailing NPR's ombudsman about this ?

They're sometimes very good at responding or reading criticisms on air.

[gandalf]

2. Bias of polled sample?

"Some people are more likely to be home and participate in the survey (e.g., unemployed people, the elderly, etc.). So already your sample of respondents is biased."

1) Don't they do this type of call in the evening when people are more likely to be at home?

2) Is there a reason to assume that the percentage of people who will vote for, say, Kerry, depends on the fact that they are at the moment unemployed? Is the resulting bias (if they do the calls during the day) relevant for the question for whom they would vote?

[cheshire]

3. Interesting that they sound like this administration. I have always felt

that certain people where home more than others. Those who have to work will always be Dems. Thank you both.

[libertad]

4. That makes sense. A lot of Democrats are probably working when

they make the calls. My mom, who is a housewife, has been polled. She's always at home. I've never been polled and I'm a working person.

[snoogins]

5. ...

Gallup is run by moonies and fundies, look at moveon.org

they just want the apocalypse bringing refucklitans to keep bringing us to the brink of hell and push us into it.

[muriel_volestrangler]

6. A Democratic pollster's take on this

http://mysterypollster.typepad.com/main/2004/09/moveon_vs_gallu.html

They say Gallup “refuses to fix a longstanding problem with their likely voter methodology” and imply that weighting by party is the fix, never mind that most of the “other publicly available national likely voter polls” they tout to counter Gallup do no such thing.

And an earlier comment of his:
http://mysterypollster.typepad.com/main/2004/09/my_post_on_weig.html

All of which suggests that the slight increase in Republican party ID evidenced on polls last month is entirely plausible, but may also be quite short lived. Pollsters weight by long term averages in Party ID at their own risk.

In fact, his whole blog is worth reading, if you want to think about polling.

[Wapsie B]

7. Thank you for this.

After what went on in '00 my suspicions on Gallup were raised. This confirms it. Btw, they do a crappy workplace satisfaction survey as well.

[serryjw]

8. "Correct" polling is cross sample of Demographics.

I have posted this before. I worked P/T in 1983 at the Harris Poll in NY during Ronnie's Term. We spent 2 days looking for a black woman whom made over $100K/yearly!

If you call 303 861 or 303 863 you are calling liberal gay Capitol Hill (Denver,CO). It is so easy to slant the results. At Harris we had mods making sure WE didn't READ the questions in a bias way to effect the response
Today with caller ID, Call intercept and cell phones exclusively, all the VM...You don't get to a cross section of the country.I have dial up and am on the net 15 hours ad ay. NO ONE can get to me! I got a call from a RW organization 6 months ago. They wanted to * KNOW* my opinions on abortion. When I told them I was 100% pro-choice they thanked me and hung up! They REALLY didn't want to hear from me!

A Black woman making over $100K VS a black woman making $25K is probably going to respond differently. NONE of these polls are worth anything. We all get crazy when the polling results come out. I don't, I know they don't reflect the temperature of the country

AT BEST! They are ONLY a *snap shot* for that moment in time. There is way to much happening in the world to take these seriously.

[Tansy_Gold]

9. Thank you, serryjw

It's the demographics.

As I've said before, the problem is not with the METHOD but with the METHODOLOGY.

If Gallup uses a demographic of 60% repukes, their poll will be skewed. If they use a demographic that ignores 18-24 year olds who have never voted before but are scared shitless of a draft, their poll will be skewed. If they use a demographic that results in more calls made to high-income area codes, their results will be skewed.

The only value I can see to any of the current polls is as an indicator of a trend. If one candidate begins to go up steadily based on the same polling methodology, maybe there's something going on to boost his support, or decrease his rival's. But what we're seeing is so much seesawing that no one is doing anything particularly strong, no one is screwing up big-time, and the bases are holding steady -- BASED ON THE STANDARD MODELS.

The problem is that the model is changing, and the question is, are the polling demographics changing to match it? If 18-24 year olds comprised 15% of the 2000 voters (a hypothetical number; I don't have the actual stats), are they still 15% of expected voters in 2004? What if they're 20%? If African Americans comprised 10% of voters in 2004 and voted 85% Democratic, what happens if they become 12% of 2004 voters and vote 95% Democratic?

TruthIsAll has done a long-running daily analysis of the polls here on DU, with a daily prediction of Kerry's electoral vote collection. Even that prediction has gone up and down -- because the polls are only a snapshot of "if the election were held today." Well, folks, it ain't bein' held today; it won't be held until November 2, and that the ONLY poll that counts.

Reports keep showing increased voter registration all over the country -- and even among Americans living abroad. Reports indicate that the overwhelming majority of these new voters are registering as Democrats and many have said they are angry at boosh and want to boot him out. We all have stories of former boosh voters who have turned to the light and will vote Kerry -- in the military, in the high ranks of repukes, in business, in the families directly affected by 9/11. Where are all the stories of dedicated new boosh voters flocking to register to keep the chimperor on his throne?

Why all the stupid crap about the set-up for the "debate"? Does anyone besides me remember the months and months it took to determine the size and shape of the table for the Paris Peace Talks during the Vietnam War? This circus of the absurd that's going under the title "Presidential Debate" echoes that long-ago farce.

The GOP is terrified. They are shitting their collective pants because they know, THEY KNOW, boosh does not have the support to win legitimately any more than he had it four years ago. They are resorting to EXACTLY the same kind of tricks Hitler's minions used to put and then keep that evil creature in power. As the poster said, same shit, different asshole.

Tansy Gold

[serryjw]

10. The POLLTERS are BIG business!

They love churning the pot. They beleive it's a self fullfilling prophecy. IF they tell us Bushit is winning people will either agree or stay home. After 3 years of dong this Bushit I know that this country is very angry. People who never voted are now registered. they want Bushit OUT!
Don't under estimate the powerof Oprah! After her show yesterday millions are going to register.


The tried and true is no longer TRUE! I'm not even sure that a cross section of demographics will even work any more.We never had this LARGE % of Independents that DON'T vote party line. Most of my friends are registered as Independents BUT want Bushit OUT!

[Eurobabe]

11. Good. I don't have Ph.D. but I do polling and surveying

and I know how easy it is to lead and skew. (I am very careful and subscribe to CASRO ethics myself)

These polls are useless pieces of shiite that the media uses to influence and sway.

Ironic that the PhD guy who does Ohio poll for Uof Cincinnati never answered my question on his methodologies, ie cell phones included, first time voters?