A statistical tool to calculate exactly how reliable the analysis of an "expert analyst" on TV is:

[DetlefK]

Statistics has this nice mathematical tool for calculating how reliable your predictions are: the chi-square-value.
http://en.wikipedia.org/wiki/Chi-squared_distribution

Long complicated math put short: We take your prediction and we take the actual outcome and transform the deviation between those two into a single convenient number, a variable named "chi-square". (The calculation itself is fairly simply once you understood the basics.) A perfect match between your prediction and reality means a chi-square of 0, and the larger the deviation, the larger chi-square gets.

Imagine a world where the professional bullshitters on TV are actually graded based on their accuracy. Whether it's election-results or real-world developments, one glance and you could tell how much the opinion of this guy on TV is really worth.




Would you like this or not?

[underpants]

1. Fox News is already chocked full of statistical tools

[GreatGazoo]

2. Studies show people seek out "news" sources they agree with.

Which means that they aren't as interested in learning anything new as they are in having their belief system confirmed.

It is a bit like listening to the cheerleaders instead of looking at the scoreboard. And the chi-square on cheerleaders would be nearly meaningless.

[FSogol]

3. Predictions are always just that. You could predict that HRC was the front runner in 2008 and you'd

correct. Then Obama arrived and won. Your prediction is now wrong, but enough things changed that your original prediction no longer held. That doesn't make it bad analysis when it was made.

[ProfessorGAC]

4. Beat Me To It, Sogol

Chi^2 works great if the variables are known. I could predict that only 60% of people believe in extraterrestrial life, but if a ship showed up in Washington tomorrow, the number would go up. But, the variable of a ship actually showing up is a complete unknown today.
Not only do conditions change but the variables themselves change in a prediction.