Math Guys: Are the ODDS 1 in a TRILLION or 1 in 10 MILLION?

[TruthIsAll]

I would like some help here.
 
I calculated the ODDS of the exit poll deviations to Bush as 1
in a trillion.

In their latest paper, US Count Votes (USCV) calculates 1 in
10 million.

HOWEVER, THE ANALYSES ARE IN ESSENTIAL AGREEMENT. 
THE ODDS ARE "ASTRONOMICAL".

My probability could very well be too low. But based on the
data and calculated state MOE's, the probability calculation
is correct. I calculate the odds for 15 states exceeding the
MOE for Bush. In fact, 4 additional states were just under the
MOE (within 2 to 5%). If just three of them are added to the
15, the odds of 18 exceeding the MOE literally vanishes beyond
the capacity of Excel to display (see the table below).  

The fantastic USCV paper does not provide the actual exit
poll and final vote deviations used in the analysis (only the
results).

This post seeks to determine possible reasons for the
discrepancy in probabilities. In fact, they are much closer
than they might appear, as shown below.  

If anyone with statistical expertise is reading this, please
jump in with comments. MathGuy, Papau, others?

USCV (Freeman)  and I use slightly different methods (MOE
deviations vs. t-statistcs):
-I calculated that 15 states exceeded the MOE in favor of
Bush, each with 2.5% probability.
-USCV calculated that 7 of 50 states have t-values below
-2.7, each with less than 1% probability.

We both use the Binomial probability function. 


How did USCV calculate the exit poll deviations? We do not
know the detail data used to calculate the exit poll vs. vote
deviations.

My calculations are based on exit poll data downloaded by
Simon (2-party adjusted) vs. the final vote. I use the
deviations and the MOE, calculated from the exit poll sample
size.

If the USCV deviation data differs from mine, that could
account for the discrepancy between their calculated odds (1in
10 million) and mine (1 in 1 trillion).

From the bottom of Page 2 in the USCV report:
"Seven of fifty states have t values less than -2.7,
meaning that each of them has LESS than a 1% probability of
having the reported difference between exit polls and
election results occurring by chance. The binomial
probability that 7 of 50 should be so skewed is LESS than 1
in 10 million. A full comparison of the exit polls with the
null distribution via a Shapiro-Wilk test yields a
probability that is astronomically small"	

The seven (7) individual probabilities are not given. 	
We only know that EACH is less than 1%.	

USCV states that "a full comparison of the exit polls
with the NULL distribution (blue curve) via the Shapiro-Wilk
test yields a probability that is ASTRONOMICALLY small".
DAre they saying that the probability is astronomically small
in comparison to the 1 in 10 million odds? If so, would the
test in fact yield probabilities closer to 1 in a trillion?

Assuming (conservatively) a 1% probability for each of the
seven (7)states (we know it is less), then the probability is
given by:	

Prob = 1 - BINOMDIST(6,50,0.01,TRUE)	
Probability  =	6.85284E-07
so the odds are 1 in 1,459,249.

Since the probabilities are LESS than 1%, that could account
for the difference between the 1 in 1.45 million odds and the
"less than 1 in 10 million" they estimate.

In fact, if just ONE (1) state were added to the seven, the
odds would be 1 in 27 million. TWO (2) additional states (a
total of nine) would lower the odds to 1 in 577 MILLION.

Here is a probability table for various N states, using the
USCV criteria based on t values less than -2.7:

St      Prob.	        Odds
N	  T <-2.7       1 in
5	0.000145689	 6,864
6	1.08968E-05	 91,770
7	6.85284E-07	 1,459,249

The probability rapidly declines as N increases…		

8	3.69316E-08	 27,077,101
9	1.73063E-09	 577,823,565
10	7.13284E-11	 14,019,663,602
11	2.6098E-12	 383,170,938,646
12	8.57092E-14	 11,667,356,547,592
13	2.88658E-15	 346,430,740,566,961
14	        0	           #DIV/0!
		
.......................................................

My analysis is based on the Exit Poll margin of error.		
N = 15 of 50 states exceeded the MOE (each with .025
probability) in favor of Bush. NONE did so for Kerry.		

the probability for at least 15 states to exceed the MOE is:
Prob = 1-BINOMDIST(14,50,0.025,TRUE)= 9.166E-13	
The odds are 1 in 1,090,988,281,824	

     Probability	  Odds
N	N > MOE        1 in 
5	0.008132778	 123
6	0.001510773	 662
7	0.000237311	 4,214
8	3.2064E-05	 31,188
9	3.7768E-06	 264,775
10	3.92003E-07	 2,551,003
11	3.61654E-08	 27,650,776
12	2.98705E-09	 334,778,890
13	2.22187E-10	 4,500,703,425
14	1.49595E-11	 66,847,251,840
15	9.166E-13	 1,090,988,281,824

And going as far as we can:
16	5.24025E-14	19,083,049,268,519
17	3.88578E-15	257,348,550,135,457
18	      0	           #DIV/0!




 
 

[Warpy]

1. When odds get that high,

"as likely as the Pope on a pogo stick" about covers it.

[NationalEnquirer]

20. It doesnt matter at that point.

Statistically, either way it was virtually impossible!

[Al-CIAda]

2. This only confirms that bu$h has a miraculous mandate from God! -eom

[k8conant]

3. It looks as if you're basing probability on different things...

but in any case the report says the 7 of 50 skew is < 1 in 10 million and the full comparison (via a Shapiro-Wilk test--don't ask me what that is) "yields a probability that is astronomically small." That would put your 1 in a trillion in between.

:kick:

[L. Coyote]

4. Why compare when your numbers are different? n/t

[TruthIsAll]

5. You do not appear to understand the point of my post.

"Why compare when your numbers are different?"
We don't know what numbers they used.

Do you disagree with my desire to determine the reason for the disrepancy? If so, why?

Must you use this as an opportunity to criticize my work?
When all I seek is the truth?

Or are you just the last naysayer standing?

[Blue_State_Elitist]

16. I don't want to criticize you

but I remember reading your statistics during the campaign that stated Kerry had a 98 percent probability of winning... it lifted me up certainly, but reflecting back, might have given me false hope.

Even with widespread fraud, your stats gave too much of an advantage to Kerry.

[TruthIsAll]

17. And I was right. It was actually closer to 100%.

You fail to appreciate the mathematical win probabilities.

They were derived using the normal distribution using the
final average of 18 national polls (MOE of 0.73%) and a 5000
trial Monte Carlo simulation based on the latest state polls.

I projected that Kerry would win 51.80% of the two-party vote.
According to the national exit poll, Kerry won by 51-48%.

Let's review the vote projections and probabilities:

Final poll averages:
      Nat9	Nat18	State
Kerry	47.44	47.33	47.88
Bush	47.00	47.17	46.89
			
Projected%			
Kerry	51.61	51.46	51.80
Bush	48.39	48.54	48.20
			
Prob%	 
Kerry	99.89	100.0	99.76
Bush	0.11	0.00	0.24

			
			

[TruthIsAll]

18. This was a response to your post on my "wild probabilities"

in the original thread where this post first appeared. I decided to start a new one here.

I may have overreacted, but it was due to my short fuse. I get upset when my work is demeaned by someone who has no analytic rationale in doing so. I have been assaulted by FreeperTrollNaysayers without letup here for months.

Now that Freeman and 10 PhD's have spoken very clearly on the exit poll issue, I hope these assaults will subside.

In any case, I would like to take back my prior post. I protested too much.

[TruthIsAll]

7. Coyote, if I misinterpreted your question, I apologize for my reply.

I should not have questioned your motivation and referred to you as a naysayer based on the one line title.

I guess it's my short fuse, tempered by the trollfreeps.

[Bill Bored]

6. I'm still trying to figure out the probability that

the head of Diebold Election Systems and the VP of USCountVotes would have the same surname (O'Dell)!

That said, does anyone have actual turnout numbers for Dems and Repubs yet? That would be helpful in getting to the bottom of this. The unadjusted poll claimed it was Dems 38, Repubs 35. They changed it to 37/37 at the last minute. And we should look at these in the swing states of Ohio, FL and PA, to see if the adjustment there is consistent with the national poll.

[TruthIsAll]

8. We are way past that. Mitofsky has been hoisted on his own petard.

Obviously, the state polls and the original NEP (13,047) poll of 38/35 are in agreement: Kerry won by 51-48. And that's not including those who were disenfranchised and never voted.

As far as I am aware, in every election the national exit polls have always had Dems Party ID ahead by at least 2-3 points.

[Bill Bored]

12. TIA, please elaborate

It's important that we understand which polls you're referring to.

Are you saying that the ratio of Dems to Repubs in answer to the party affiliation question on the final state poll reports was 38/35? Surely it wasn't the same in every state, and I'm not sure that question was even asked on the state polls -- was it?

Some may disagree, but I think this is a KEY piece of data because it proves that in order to give the bush the win, or anything close to it, the turnout numbers had to be adjusted in favor of the Republicans. Also, unlike the secret ballot, there should be a way of checking the party turnouts INDEPENDENTLY of both the exit polls AND the election results. This is what you want if you're going debunk one or the other.

So far, all the efforts I've seen on either side rely only on self-consistency, i.e., the exit polls are right because they just are, or the election results are right because they just are.

We need some independent means to verify that one or the other is wrong. I think the turnout numbers are the way to do that. And of course, they can also be affected by such things as registration fraud and voter suppression, both of which we know actually took place.

If you sign up 1 million new Repukes and 1 million new Dems have there registration thrown in the trash, or are prevented from reaching the polls, you might end up with an equal turnout which, combined with other factors, gives bush the win. I don't know why this idea is given such short shrift on this board when the "proof" is right there in front of us in the very same exit polls AND on the ground.

[TruthIsAll]

24. I am referring to the Pristine NEP National poll (13,047).

In every national exit poll that I have seen, the Dems are represented by 38 or 39%.

Check Gore and Clinton for the actuals. I forget.

[davidgmills]

14. O'dell

Some number in six or so billion depending on whether you are statistically counting the world population or actually counting it.

[thanatonautos]

9. Clearly you're in basic agreement with these authors.

The Shapiro-Wilk test they mention is a test which
can be used to try to determine whether it's unreasonable
to believe that a series of random samples was
drawn from a normally distributed population.

The test yields a result called W, which is
straighforwardly calculated from the sample values,
the means, and the variances and covariances of the
of the sample. Using a Monte-Carlo simulation
one can convert the W-statistic into a probability.
Small values of W are associated with rejecting the
null hypothesis that the population is normally
distributed.

In this case the test was of all the 50 state exit poll
deviations from the official results, and the aim was to
see whether these can be interpreted as being random
samples drawn from a normal distribution centered on the
official election results (apparently with a fixed standard
deviation computed according to the margins of error of the
state polls, although these details are left unstated in
the paper).

Such a test clearly gives an extremely low W value if
you insist that the center of the normal distribution
is at the official result. A shift of at least one standard
deviation of the mean value of the sample from the
mean of the blue curve is very evident in the graph
of the data.

The authors claim that if you introduce this shift into the
null curve, then the null hypothesis is not rejected
(p=0.4) as long as the three worst outliers (the
states deviating the most toward Kerry) are dropped.

The authors further state:

A full comparison of the exit polls with the null
(blue) curve via a Shapiro-Wilk test yields a probability
that is astronomically small.

It's clear from the context that the authors mean
astronomical in comparison to the 10,000,000 they
estimated just from the seven states with the lowest
t-values (largest left deviations on the graph).

The reason for the difference with the binomial probability
you calculated for 7 states to deviate in this way is
certainly that the authors are making use of the probabilities
of the actual deviations estimated from their actual
t-values instead of using the conservative estimate that
they all have t's of -2.7 or probabilities of 0.01, for
example they may have used an average probability over
all seven states and put that into the binomial formula.

You could try to do better from this paper by estimating by
eye the deviations of the seven points from the mean value,
then converting those into t-values. The t's can
be converted to probabilities.

It seems clear the result will be less than what you
estimated from the binomial distribution. To me the
deviations look pretty large: there are at least two
states deviating at about four sigma. Whether the
1 in 10,000,000 the authors state is actually correct is
not clear to me. It seems to me that they probably
should have a smaller value than this. But all they
claimed was an upper limit, of course.

I'ld like to have the numbers they used.

[TruthIsAll]

10. Thanks for the clarification. Some may scoff at the odds, but..

The difference between 1 in 10 million and 1 in 10 trillion odds (or higher) is very significant, if only to further prove the impossibility of it all.

[thanatonautos]

11. Bottom line ... whichever estimate of probability you wish to use

no reasonable person can possibly justify the belief
that the results were due to random statistical error.

If you use the 50 state value based on the normality
test, which they don't quote, I bet the chance may be
quite a bit less than 1 in 10 trillion.

So you're maybe understating the case.

:)

[thanatonautos]

22. You could also try using a multinomial distribution to estimate the

probability of seven (or more) states deviating.

If you divided up the possible outcomes into ranges
of one standard deviation like so:

(0) less than -4 sigma
(1) from -4 to -3 sigma
(2) from -3, -2 sigma

and so on, you'd get 10 bins. You can calculate
a probability p_i for getting a result in each bin
pretty easily using a table of the cumulative
normal distribution.

Then the probability of getting exactly k_i exit poll
deviations falling into the i'th bin, for every i from 1
to 10, would be given by the multinomial distribution:

P(k1, k2, ..., k10) = C * p1^k1 * p2^k2 * ... * p10^k10

C = N!/(k1!k2!...k10!)

This kind of analysis should be pretty easy to carry out
using the graph in the paper, either for N=7 or for N=50
states.

I'll try to do it later on and see how it compares to the
much simpler binomial estimate.

What BlueEyedSon said is also a good idea ... you could
definitely write to the authors of the paper and ask
for the data that they used. I would think they'ld
likely be very willing to provide them.

[ottozen]

13. W?

[davidgmills]

15. I got lost after basic agreement

[thanatonautos]

19. Yup ... I guess it might be better to call it S.

On the other hand, maybe a small value of W is something
we'ld all like to see.

[BlueEyedSon]

21. You could correspond directly with the USCV folks.

PM me if your are interested.

[Nederland]

23. Doesn't matter

Without knowing if the samples were representative or not, applying statistical formulas is waste of time. Garbage in, garbage out.