Damn it, I guess I have to do a "Warn Off" for Overtime Politics (Ignore them, entirely)
So, Overtime Politics is showing up all over DU these days because they're producing numbers way different than anyone else and that can be fun to see... especially when it benefits your candidate. However, Overtime Politics is not a real polling outfit. Not even close.
They really first showed up on the DU radar when they posted a poll about Bernie being within twenty of Clinton in South Carolina. The no longer with us poster who started that thread vehemently defended Overtime because.. well, I'm not really sure, to be honest.
But Overtime had two attractions going for it:
- They're producing contrarian results
- They assign an MOE to their polls, which make them look valid.
So what is Overtime?
First of all, they're brand new. I mean really new. Too new to even have an operation in place to produce valid polls. The site was registered on October 20 of this year, out just over two months ago.
However, they're planning on hiring a second phone operator to be able to release 3 or 4 polls per week per race. That, well actually that is scientifically impossible to do. The pace at which they release polls should be a huge warning flag.
But maybe they've figured all this out. Maybe they've managed to create a streamlined process that will be the envy of actual pollsters. Well, they have streamlined something...their methodology:
"We call both landlines and cell phones at about a 3:1 ratio, however, telephone polling is a very time consuming process, the overwhelming majority of people do not even answer their phones when we call. In order to speed up the process, and get what we feel is the most important metric (“Who are you going to vote for?”), we only ask our audience if they are planning on voting in their state’s primary, and if they are, who they are going to vote for."
There is a lot wrong in that paragraph, enough to make the statical analyst in me to just weep softly into a beer. But the highlighted area is of greatest concern. Every poll is only two questions: are you voting in the primary? Who are you voting for?
The reason no one can see demographics in their polls is because they don't collect them. They don't weigh responses, they don't establish population parameters, they don't establish location densities...they don't do any of the very, very basic things real pollsters do in order to establish validity of results.
So what do they do? They tally up the total "votes" for each candidates, divide by the overall total and call it a representative poll for a total population. Since it is missing literally every single piece of work that goes into a poll to make it valid, I can state emphatically that the only thing it is representative of is that the owner of Overtime Politics doesn't know how to poll.
The owner's comment about the lack of methodology and controls doesn't help:
"The site has been getting lots of emails saying our methodology is flawed, but no poll is going to be perfect. There are certainly polls out there with larger sample sizes, smaller margins of error, and weighted demographic averages, and I can assure you that CNN or the Wall Street Journal have more resources than Overtime Politics to fund these polls. We feel as though our strength will be in polling states which do not get much polling attention until right before they vote. Although the polls may not be as thorough as some of the others, we will at least be trying to get some data for you."
And as for the MOE people like to bang on about showing it is scientific? You can calculate one right online. So if he calls, say 400 people and the state has a population of X. Guess what, he can establish a margin of error. It is too bad he literally never established his population parameters so that the MOE actually references something.
And if you still think I have an ax to grind, even though I have always tried to be nonpartisan about analysis, well, please consider that neither RCP nor Pollster have bothered to report any of their findings.
They're not real pollsters, not even remotely close.
You can find the Overtime Politics excerpts at http://overtimepolitics.com/
Be warned, going there may result in a sudden case of the stupids.
= new reply since forum marked as read
Just because some polling outfit the OP likes doesn't count Vanderbilt University in their lists doesn't mean it was a bad poll.
Besides, the OP likes some betting aggregate site for judging people. 
Response to Godhumor (Reply #7)
Using Nietzsche to counter a poll interpretation you don't like is up there in the logic spectrum with using Bible quotes to defend creationist beliefs. 
And in any case, he was not arguing against practical facts, just pointing out that confirmation bias exists.
key-master of pollsters on DU, and, as DU's polling overlord, it's up to you to "warn off" or "accept" whatever poll maker appears.
And, of course, I always take whatever you have to say about polls and poll makers as straight from the poll god's mouth. Because you yourself have no dog in this race, and no personal bias involving primary candidates at all.
Just in case you needed it: 
and, edited to add, for the disingenous: I've never seen a poll by the organization you are patronizing us with, and probably never will.
through different lenses, from different points of view. I'll stand by mine. 
