Nate Silver Calls Out RealClearPolitics for Repug-Bias

[jefferson_dem]

Real Credibility Problems

Last week, I discussed Real Clear Politics' decision to exclude the Research 2000 daily tracking poll from their national averages. After a lengthy discussion with RCP founder John McIntyre, I decided to defend them, pointing out that while RCP may have a debatable framework for deciding which polls they do or do not include, they at least appeared to have applied this framework consistently.

My gut-level takeaway from my conversation with McIntyre was somewhat different from what I represented, however. My gut-level feeling was that RCP was in fact cherry-picking those results that were to its liking, and then coming up with post-facto rationalizations to justify its decisions.

RCP made a decision yesterday that convinces me I should have trusted my gut. Specifically, it was their decision to re-include polls from American Research Group (ARG) in their state-by-state averages.

ARG had been effectively "banned" from RCP for several months now, going back to the Democratic primaries. Sometimes RCP listed ARG polls with an asterisk and did not include them in their averages; more often they didn't list them at all.

Why didn't RCP include ARG's polls in its averages? Presumably because they had an exceptionally erratic performance during this year's primaries, a performance that -- when coupled with ARG's relative lack of disclosure about its methodology -- had led RCP to conclude that their polls were not credible.

As you may know, I am not much of a fan of ARG myself. However, while ARG ranks toward the back end of our pollster ratings, so do a lot of other pollsters that RCP has no problems with.

What I do know, however, is that whatever RCP's reasons were for excluding ARG from its averages, none of those reasons should have changed over the past 48 hours. Did ARG suddenly improve its level of disclosure? No. Did their polls suddenly become more accurate? How could they, when we haven't had any elections to evaluate them.

No. The only thing that changed is that ARG released a set of three polls yesterday that had considerably more favorable numbers for John McCain than other contemporaneous polls of those states. In each of those states -- North Carolina, Virginia, Nevada -- ARG is presently the outlier on the John McCain side (tied with Mason-Dixon in the case of Virginia). RCP did not feel any compulsion to include ARG's numbers when ARG cycled through all 50 states a couple of weeks ago (including many where there has been very little polling). Only when ARG released this gravity-defying set of polling in Virginia and North Carolina and Nevada did RCP suddenly have a change of heart.

<SNIP>

http://www.fivethirtyeight.com/2008/09/real-credibility-problems.html

[jberryhill]

1. Meh... I'd rather not suffer from overconfidence

At bottom, all polls are dependent on their models, and at electoral-vote, pollster, 538, and so forth, there is also going to be an irreducible kernel of subjectivity.

The "young voters and new voters are going to have a significant effect" is as shopworn a political cliche as is "energy independence". I would love to be surprised.

What you can do at RCP that you can't do at 538 is play with the map on your own.

Maybe it's my preference for pessimism as a motivator, but if RCP represents a "worst case" against my preferences, then all the better.