The other Citizens United decision— whether to include their poll

[cthulu2016]

Most controversy about polling aggregators gets down to which polls they include, and how.

The problem is that you cannot make it up as you go along. You must have rules and you must follow them. The second you start cherry-picking, making individual judgment calls, you are just another guy on a barstool with an opinion.

Some folks do not like that Nate Silver includes Gravis polling, as if his inclusion of them means he is saying they are legit. This misses the point... any rigorous selection method will include some instances that an agregator, personally, does not think are legit but the risk of excluding them is higher than the risk of including them.

Say we have ten polls—3,3,4,4,5,5,5,6,6,9. The average is 5. The 9 appears to be an outlier. Should we throw it out? Well, if so then why are we keeping the two 3's? Because they are "wrong" but not as wrong? If the two 6's are right then the 9 is no further off than the 3's are, and there are twice as many threes. And if the 6's cannot be right then why are they in the sample anyway? And so on.

I like Democratic pollster PPP. I do not like Republican pollster Rasmussen. I am convinced that robocalling is an inferior polling method that favors Republicans by 2-4% because you are not allowed to robocall cell phones.

But if I threw out Rasmussen how could I keep PPP? They also robocall. Their results do not appear as biased to me as Rasmussen's, but if I included PPP and threw out Rasmussen because PPP looks better to me then I might as well just decide what I think the numbers should be.

Throwing out outliers presents a similar problem. If you discard outliers then you are distilling results, favoring 'in-line' polls simply for agreeing with each other. A polling analyst can say, "This is such an outlier I am not taking it seriously until it is confirmed by other polls." But a poll agregator cannot do that. The whole idea of averaging polls is to smooth out the nose and deviations. If you presume to keep the noise and deviations out of the average in the first place then you are making an assumption about what the real number should be.

The first time you toss a poll other than for reasons you pre-defined and apply to all polls then the enterprise is corrupted by personal bias.

Which brings us to Citizens United. They commissioned a poll of Ohio that was released today. Romney +3. As a one-off Ohio poll commissioned by a propaganda organization for release in the last week of a campaign it appears to be a stunt to skew poll agregators.

But even while making that observation, that doesn't mean it is correct to discard it. A lot of one-off state polls come out this time of year from all sorts of organizations. To maintain the integrity of the process, if the Citizens United poll is tossed as intentional bias then you must be sure that ALL biased polls are tossed, and you cannot be sure of that.

If you use your judgment then there will be a bias in which biased polls you catch. Without objective selection criteria your aggregation is just a fancy version of your opinion.

(This is how I feel about felons voting. Of course they must be able to vote, even while in prison IMO, because voting is seeking an average of the populace, not a predetermined result. A ban on felon voting must ban everyone who has ever committed a felony, not just those who got caught and convicted. Also, it must include everyone who would have committed a felony in certain circumstances they never happened to face. Otherwise the ban is a ban on unlucky people, or on people who get convicted, not on people who commit crimes, and it distills and perpetuates all flaws in the criminal justice system. Similarly, if you toss one bad poll but miss another you are distilling your own prejudices.)

Unfortunaely, with objective selection criteria your method can be gamed. So you probably do want to refine your method (in the way evolution keeps animals one step ahead of parasites by making small changes each generation)... but after elections, not on the fly.

So you do what Talking Points Memo did with the Citizens United poll. They included the poll in their Ohio average, because it met their pre-determined criteria, while publishing a comment on the poll saying that Romney's favorability in the poll seems absurdly high so it should be viewed with caution.

[jjones217]

1. These polls are absurd

I think you are correct in saying that they should be including in the poll aggregates, but there needs to be a very obvious disclaimer placed next to them. In the same Citizens United Poll that showed Romney +3, Josh Mandel was shown as having a +5 over Sherrod Brown. That's just absurd. Brown has an aggregated lead of +5.5 in most models, and even the extremely GOP-biased Rasmussen reports poll shows Brown as +1.

I've read recently, and I actually think there's some truth to it, that the GOP is delegating these fringe groups and biased polling agencies to push a last-hour Pro-Romney polling skew so that if they manage to steal the election on Tuesday they can just point to the "polls" that show a Romney lead.

On a side note, I would be amused if the GOP actually did manage to steal the election in Ohio, only to fall short in VA and Colorado, and end up losing the electoral college vote anyway.