Zogby 4 point lead (49-45) rises to 9 points (51-42) using Rasmussen party ID weights

[Chasing Dreams]

Don't worry or be misled by Zogby's hype and idiocy. A little deeper dive into his numbers reveals this:

McCain wins 87% of the Republican support, and Obama 84% of the Democratic support, and each candidate wins 11% of the opposing party’s support. Obama continues to lead among independent voters – his advantage now stands at 16 points, 51% to 35%.

With independents at a breathtaking 16 point lead even in Zogby he must have higher proportions of Republicans than Democrats in his poll. But even the Republican leaning pollster Rasmussen believes Democratic turnout will swamp Republican turnout this year:

For polling data released during the week of October 26-November 1, 2008, the partisan weighting targets will be 40.0% Democratic, 32.8% Republican, and 27.2% unaffiliated.

Now, if we combine the Zogby results from 10/28 with the Rasmussen party weights we get a more realistic view of what is happening -- even in Zogby!:

McCain Obama Weight Party
11% 84% 40% DEM
87% 11% 33% REP
35% 51% 27% IND

42% 51% Total

[On the Road]

1. Zogby Does Not Believe in Weighting on Self-Reported Party ID

because it is fluid and not able to be confirmed. I agree with him in principle. It remains to be seen whose polling will be the more accurate.

[msallied]

2. Zogby's track record has pretty much proven him to be

an ineffectual fraud.

[On the Road]

8. Well, He Showed Himself Biased Toward Kerry Last Time

One of the most fair and balanced turned out to have been Fox. For what that's worth. You just don't like him because he's not confirming your belief.

[msallied]

11. No, I don't like him because he's a push-polling fraud

who was WRONG in 2004 and who earlier this year predicted a "McCain landslide." Don't presume to know why I do or do not "like" someone. I have enough brains in my skull to evaluate the methodologies of the pollsters, and Zogby's methods are highly flawed.

[On the Road]

24. I Didn't Follow All the Details of the "Push-Polling" Controversy

but I find it difficult to believe that Zogby is doing anything like what he's been accused of doing here.

[msallied]

26. Someone screen-captured the poll itself.

Oh and nevermind the online "interactive" thing is probably more subject to error than the Rasmussen self-reporting party identity model.

[On the Road]

28. Yes, I Saw That Screen Shot

and the assumption was:

--That all these leading questions were being asked before asking for voting preference,
--That those responses were incorporated into his national polling estimates,
--That it was done in order to improve McCain's numbers, and
--That that accounts for the difference between Zogby and other polls.

It is easier to believe that partisans are making a leap of faith based on not understanding what the purpose of those questions was or what conclusions were drawn from them. I promise that was not the basis of the national polling numbers everyone sees.

[Chasing Dreams]

12. No, I like him as a person,

but he is a really crappy pollster. But if you don't believe me, go over to fivethirtyeight.com and check out what the great statistician Nate Silver has to say about weighting by party ID in the year 2008.

[On the Road]

30. Since Nate Silver's Name is So Ubiquitous on That Site,

I was not able to google any discussions on weighting by party ID. Would appreciate any links. Here's a rationale I posted above for NOT weighting by party:

Consider the effects of large numbers of registered Republicans calling themselves independents, which may be happening this year. The polling group of self-reported independents will include many more registered Republicans than normal. This has the following effects:

--True independents (for lack of a better term) are mixed with registered Republicans, and the average is probably more pro-McCain.
--The group of self-reported independents is larger, and is weighted down. This underrepresents true independents.
--The remaining self-reported Republicans are more right-wing.
--This more pro-McCain group of self-reported Republicans are weighted up, and overestimates McCain votes from Republicans.

I like fivethirtyeight.com, but do not take it as gospel. Silver has an interesting background in sabermetrics, but politics is not always analagous. I like Football Outsiders' modeling, too, but their model recently showed Philadelphia as the strongest team in the NFL. Silver's models are open to as well.

[MadBadger]

21. We dont like him cuz his polls have sucked, he push polls, releases Internet polls leaks to Drudge

[Chasing Dreams]

4. Fine. But that is the difference between the widening and narrowing poll stories in the MSM.

Zogby is getting fewer Democratic responses because millions of people are involved in this campaign now. Canvassing, phone banking, helping anyway they can. They are not home to answer the phone. Indeed, many don't even have land lines!

I predict Zogby and the others ignoring new registration figures and self reported affiliation, as well as self reported likely voting behavior vs. 2004 likely voter models, well they will be discredited in a week.

At least Gallup has the decency to show both types of models. Obama is up 5 in one, 10 in the other. They recently put out the latter set of figures because they don't want to go down in flames with Zogby.

[On the Road]

19. What Gallup is Doing

as I understand it is not related at all to what Zogby is doing in avoiding a party ID weight.

Your comment shows that you're judging the results by the narrative they create, and interpreting it in light on your own anecdotal experience in volunteering for Obama. That is the essence of what a pollster should NOT do.

Pollsters do not think "how can I adjust my model to make the result fit my preconceptions." While it shouldn't make a difference, Zogby is a Democrat. He went out on a limb for Kerry last election despite his own poll numbers. He is very open in discussing methodology, and knows what he is doing. He may not be right this election either, but none of the other pollsters may be, either.

[MadBadger]

25. What kind of Democrat leaks their numbers early to Drudge and Push Polls for Newsmaxx?

[pnwmom]

32. Zogby actually does weight by party.

http://www.zogby.com/methodology/readmeth.dbm?ID=1366

Survey Methodology 10/24/08 thru 10/27/08

Zogby International was commissioned by to conduct a telephone survey of .

The sample is <1202 likely voters> interviews with approximately <25> questions asked. Samples are randomly drawn from telephone cd’s of national listed sample. Zogby International surveys employ sampling strategies in which selection probabilities are proportional to population size within area codes and exchanges. Up to six calls are made to reach a sampled phone number. Cooperation rates are calculated using one of AAPOR’s approved methodologies<1> and are comparable to other professional public-opinion surveys conducted using similar sampling strategies.<2> Weighting by is used to adjust for non-response. The margin of error is +/- 2.9 percentage points. Margins of error are higher in sub-groups.

[democrattotheend]

35. Honestly, I doubt that has much effect

I doubt there are so many people out volunteering enough of the time to actually make a difference in polling. People who volunteer for campaigns are still a fairly small portion of the population.

[BlooInBloo]

7. "Principle" has exacly ZERO to do with modeling.

[On the Road]

13. Weighting on Self-Identification

means adjusting on a variable (party ID) that's correlated with the answer. (For example, Republicans voting for Obama are likely to report they are independent.) It doesn't take much understanding of the process to see why that is not desirable and not likely to lead to an accurate result.

[Chasing Dreams]

20. That's ridiculous. Women have leaned toward the Democratic candidate for the last four cycles.

Are you suggesting that we shouldn't control for gender? All of these factors are "correlated" with the answer; that's why it is necessary to adjust in a manner consistent with the population as a whole. It is simply not credible to publish a poll and claim McCain is catching up when you polled more Republicans. I call that poll rigging.

[On the Road]

23. No, Not at All

Gender is objective. So are age, income, geography, and other demographics. Party ID is not.

Respondents often change their party ID based on who they have decided to vote for. Not necessarily from Republican to Democratic, but between those two and "independent."

It is the same reason you don't a regression with two variables that are correlated with each other.

Perhaps Zogby should be weighting his samples differently. But the weightings should be changed in other areas.

Now, it is possible that there is a positive correlation between voting Democratic and not being reachable for a poll that is not captured by any of the demographics. That creates a real problem for pollsters.

[Chasing Dreams]

27. I'm going to quibble a bit more, but at least you are apparently agreeing that Zogby's weights may

be off somehow given the high number of Republicans. But I have two additional points:

1. People often lie about their income -- both higher and lower than actual -- on surveys. I wouldn't call income self reports objective.

2. Even if party id is highly correlated with candidate preferences, regression modeling using instrumental variables such as two-stage least squares will work just fine. Indeed, in regression that is what you are supposed to do. Completely ignoring the party ID correlation by leaving it out gets you an F on the exam.

[On the Road]

29. You are Correct That Income is Self Reported

but it has not been shown to be a cause of distortion, or something that varies between election cycles.

Think about what weighting for party ID does. Consider the effects of large numbers of registered Republicans calling themselves independents, which may be happening this year. The polling group of self-reported independents will include many more registered Republicans than normal. This has the following effects:

--True independents (for lack of a better term) are mixed with registered Republicans, and the average is probably more pro-McCain.
--The group of self-reported independents is larger, and is weighted down. This underrepresents true independents.
--The remaining self-reported Republicans are more right-wing.
--This more pro-McCain group of self-reported Republicans are weighted up, and overestimates McCain votes from Republicans.

It is very possible that weighting by party identification THIS YEAR is skewing the results toward McCain. It will not have the same effect every election.

As far as what Zogby is doing, I have no idea. I just think this particular part of this methodology is sound.

[Awsi Dooger]

9. I agree with him in principle also

Some of the Party ID margins this year have been unrealistically high, with voters embarrassed to identify as Republicans. But on election day I expect it to narrow.

Zogby uses Party ID from the previous election, which helped him in 2000 but it burned him in 2004 and this year it makes no sense. I would guess +5.

[On the Road]

14. I Didn't Know That's How He Did It

It makes a certain amount of sense. I don't know that any pollster this year. has a handle any longer on how to get a representative sample. We will soon find out.

[pnwmom]

31. That is no longer his position, although it is the position of the national

association of pollsters, and I agree with it: weighting by party shouldn't be done, because party affiliation is fluid.


But Zogby now weights by party. This is from the Zogby site:

http://www.zogby.com/methodology/readmeth.dbm?ID=1366

Survey Methodology 10/24/08 thru 10/27/08

Zogby International was commissioned by to conduct a telephone survey of .

The sample is <1202 likely voters> interviews with approximately <25> questions asked. Samples are randomly drawn from telephone cd’s of national listed sample. Zogby International surveys employ sampling strategies in which selection probabilities are proportional to population size within area codes and exchanges. Up to six calls are made to reach a sampled phone number. Cooperation rates are calculated using one of AAPOR’s approved methodologies<1> and are comparable to other professional public-opinion surveys conducted using similar sampling strategies.<2> Weighting by is used to adjust for non-response. The margin of error is +/- 2.9 percentage points. Margins of error are higher in sub-groups.

[On the Road]

36. Thank You.

I was basing my comments on a radio interview in 2004.

Although, on the other hand, it says that party is used to weight for non-response. This is not the same as weighting the respondents, which I would think is the bigger source of error.

It says the poll commissioned by Reuters and C-Span. Does the pollster or the customer determine the methodology?

[pnwmom]

37. The pollster determines the methodology. But indirectly, the customer does,

by choosing that pollster in the first place.

The point of weighting is to account for sampling errors, and the primary source of sampling error is non-response. (If 10% of people don't answer their phones, then the sample of phone answerers might not accurately reflect the make-up of the larger sample. Maybe fewer old people, or men, or Republicans answer their phone. )

[On the Road]

38. Is That True?

I don't follow polling in this detail. I had always thought that the weighting was mostly designed to correct for skewed samples rather than response rates.

Do you know how high non-response rates are this year? Is it up from previous elections? I was not able to find it by googling.

I did find this on weighting by party ID. Very succinct and knowledgable.

It seems strange that normal polling procedures call for asking the respondent "Do you consider yourself a Republican, Democrat, or independent?" It would seem more accurate to weight based on an objective criterion, such as party registration. Not perfect, but it would be an improvement.

[pnwmom]

40. Thanks for the article. Here's one you might be interested in,

from the National Council on Public Polls. (They have a lot of other useful info on their site, too.)

http://www.ncpp.org/?q=node/39

One of the reasons polls may differ is the way the results are weighted. It is common practice to weight to demographic characteristics. Most times this reduces the sampling error. That is the point of weighting. Sometimes weighting makes things worse. Here's how it works.

Good Weighting: If people in the sample are selected with different probabilities sampling theory requires that those probabilities be used in the weights. If they are not used the results will be biased. Unequal probabilities occur in telephone polls when a person can be reached at more than one phone number. We also have unequal probabilities of selection if one person is selected from households with different numbers of adults. If you live in a one-adult household your chance of selection is 1 in 1. If you live in a three-adult household your chance of selection is 1 in 3. In both these circumstances weights must be applied to keep from distorting the results.

There is another type of weighting that may be useful. It requires two things to be true. First we must know some characteristic for all people in the population and all people in the sample. Second, that characteristic must be correlated in an election poll with the vote. For example, men and women vote differently. Gender is correlated with vote. If we weight the sample to reflect the correct proportions of men and women in the population we will improve the results. In non-election polls there must be a correlation between the weighting characteristics and the most important items being estimated.

Bad Weighting: The most common bad weighting in political polls is weighting just the likely voters for the number of Democrats, Republicans and others. Party identification is correlated with voting, but what is missing are meaningful numbers of party members for the whole population. Without that, the weighting is a guessing game rather than good theory. Some pollsters use the numbers from an exit poll from a past election, but the numbers of people who consider themselves members of a party changes from month to month and year to year.

Consider the change in party identification from the Pew Research Center polls throughout 1996. In the beginning of the year the Republican-Democrat split was 30%-30%. On Election Day it was 26%-36%. The number who considered themselves Republicans went down steadily the closer the survey was to the election. Using party identification to weight just the likely voters in a political poll is little better than a guessing game where the pollster is substituting his or her judgement for scientific method.

SNIP

[Amonester]

3. 11% DEM??

11% joe's-the-soon-to-be-kicked-out-senator... (?)

[Chasing Dreams]

5. Not sure where your getting at, but the table is a little hard to read

because the headings are shifted a little bit. Take a closer look and it should make sense.

[Amonester]

15. I was referring to the 11% registered DEM (?) who said they'll...

vote for mcShame/pale-in

who would vote for such a bunch of (qualifier withheld)?

(joe LIEberman)

alright I know he's indy (but...)

[Thrill]

6. Zogby is full of shit

[Drunken Irishman]

10. So does this mean Rasmussen's numbers will be more favorable for Obama tomorrow?

[Chasing Dreams]

16. Doubtful. I seem to recall that he has independents going only slightly toward Obama.

And part of the "problem" is that Obama's army is now out in force, unable to answer the pollster's call...

[Drunken Irishman]

17. Well just as long as it doesn't tighten again.

[Amonester]

18. And, to answer the pollster's call, they have to call them in the first place.

But since they only own a cell, as they dumped their old home line a few months ago...

[WCGreen]

22. It makes no sense to me that Zogby or anyone else would rely

on self identification in this day and age with everything concerning voters available at a nominal cost or even free.

It's ridiculous.

[pnwmom]

33. It makes no sense that anyone would use party weighting at all,

when party identification is so fluid.

The national association of pollsters advises against it, but Zogby weights by party now. I'm assuming they use 2004 exit poll numbers, which would explain why their results show a narrow gap between Obama and McCain. (Republicans are being over-weighted in the poll.)

[DemocratSinceBirth]

34. Zo(m)by Is A Pundit First And Then A Pollster

He waited to 1:00 PM EST on the day of the election to release his results for the 04 race and was still wrong!

He got burned because he peeked at the raw exit polls which proved to be misleading...

Ha ha ha...

[budkin]

39. Zogby has never been right.