(A Hopeful Tale by Lindeman & Stanislevic) H.R. 811: Fact & Friction -- Part III

[Wilms]

A methodology by which an election audit's size can be calibrated with a specified confidence level (like 99%) using the race margin and number of votes cast.

Holt, take note.


In a nutshell, we believe – and will show below – that the basic problem with the H.R.811 audit is simply the misallocation of resources. By making smarter choices, the country can achieve high confidence in the outcomes of almost all federal races with about the same amount of count-auditing effort. -Lindeman/Stanislevic

H.R.811: Fact & Friction – Part III

By Mark Lindeman, Ph.D. and Howard Stanislevic, Research Consultant

Wednesday, March 28, 2007

In Part I and Part II of this series, we pointed out that the post-election audits provided for in H.R.811 are too small to achieve a high level of statistical confidence that close races for US House seats will be correctly decided. While this observation is actually in agreement with a letter being circulated by Rep. Holt’s office signed by nine prominent experts, we also pointed out that this letter actually overstates the confidence level of Holt’s audits by not taking variations in precinct size into account.

snip

First, we decided to examine all federal elections in the last three cycles (2002 through 2006) – the presidential race, elections for all 100 Senate seats, and almost 1300 contested House races (about 575,000,000 total votes cast) – to explore the consequences of H.R.811’s quirky allocation of audit resources. For each race, we estimated the size in hand-counted votes of an H.R.811 audit, and the probability that this audit would detect hypothetical outcome-altering vote miscount, using the precinct size adjustment and the other assumptions we made in Part II of this series. We chose to measure the audit size in votes, instead of the number of precincts, because precincts come in many different shapes and sizes. Even using Holt’s methodology, some precincts will require only one race on their ballots to be audited, while others may require audits of all three federal contests. The cost of doing the audit is therefore not necessarily proportional to the number of precincts audited, but is much more closely related to the number of accurately hand-counted votes.

We then figured the size of conducting what we call a 95% SecureAudit in each race: an audit large enough to achieve 95% confidence of detecting outcome-altering miscount under the same assumptions as we have used in evaluating the H.R.811 audits. The closer the race, the larger the necessary audit size. (Some folks call this idea a “probability-based audit,” as in, “based on yielding at least an ‘X’% probability of detecting outcome-altering miscount.” We don’t care much what the audits are called, as long as the idea is clear.) We did the same calculations for a 99% SecureAudit.

Over these elections, a 95% SecureAudit would support much greater confidence in election outcomes than H.R.811 audits, at somewhat lower cost. Across all three election cycles, H.R.811 audits would mandate manually auditing about 20.3 million votes.

snip

http://e-voter.blogspot.com/2007/03/hr811-fact-friction-part-iii.html


ER Discussion on Part I:

(A Sad Tale by Stanislevic) H.R. 811: Fact & Friction -- Part I
http://www.democraticunderground.com/discuss/duboard.php?az=view_all&address=203x468830


ER Discussion on Part II:
(A Sobering Tale by Lindeman) H.R. 811: Fact & Friction -- Part II

http://www.democraticunderground.com/discuss/duboard.php?az=view_all&address=203x469036

[Einsteinia]

1. 10% at the PRECINCT is better than 100% hcpb at the County

10% at the PRECINCT is better than 100% hcpb at the County

10% at the PRECINCT confers a 99% statistical accuracy
BUT not if it's tabulated at after leaving the precinct.

Because once the paper is verified at the precinct. it is under citizen-controlled purview.

BUT once it leaves the precinct into the anonymous stream, into that dark plumbing system, there's really no way to verifiably know that the ballots cast are indeed the ballots tabulated.

[Stevepol]

2. Isn't this the same thing Kathy Dopp has been talking about for several months?

She and her crew of PhD statisticians at Election Archive have got several PDF files with all this worked out, at least their version of it. How is this one different?

[OnTheOtherHand]

4. sure, similar philosophies

Stanislevic and Dopp have worked independently, perhaps because Dopp is fairly difficult to work with. But there are a host of people calling for audits that achieve a given level of confidence in the outcome. For those who will not give Dopp the time of day, perhaps they will listen to someone else.

[Einsteinia]

6. Well, it's . . .

Well, it's the evolution of
first the Gold Star, then the Gold Standard, and now the Titanium Standard, which has been worked on over the past 3 years.

And you critiqued the Gold Star way back when, do you remember?

[OnTheOtherHand]

8. you may have gotten confused about who was replying to whom

Kathy Dopp was not endorsing the Titanium Standard audit in the document that stevepol linked to.

As far as I can remember, my comments on the Gold Star audit were mostly positive -- maybe you meant "critique" in that sense. I personally prefer to audit some precincts completely, but it can be argued either way.

[Einsteinia]

16. Clarification

Yes, I meant "critique" to mean "comments"

And Yes Kathy Dopp is not endorsing the Titanium because, of course, it is not her audit protocol. (But for the record, she did like the very similar Gold Standard version for reasons I can't recall now).

My point is that this Titanium Standard is an overall security plan and the specifics of the "best" random sample of ballots formula can be determined by each state.

[Einsteinia]

5. No, it's very different & here's how:

Here it is: >>>> http://www.califelectprotect.net/Titanium.pdf

The Titanium has throughly thought out all the checks and balances, because any formula is only as strong as its structural context. The Titanium is incorruptible. One of Dopp's was asking for a council of expert oversight.

These checks & balances paradigms do NOT work:

1. Expert regulatory agencies: EAC, FDA, FCC, EPA
(the "experts" get bought)

2. Government and Congress
(both get fed by the same lobbyists)

3. Enron and Arthur Andersen
(the auditor can't be hired by the auditee)

Here's some paradigms that do work:

1. The Manhattan Project:
No one person has all the pieces until the whole thing comes together at the end

2. Security without obscurity: Everyone has all the information to analyze at the same time

3. The Titanium Standard (read it above)

[Wilms]

9. Has "The Titanium" been peer reviewed?

Yeah. I asked this before.

[Einsteinia]

10. Yes, it has been peer reviewed

The concept of checks & balances is very basic and logical. No less than 200 people have seen it and about half of those have offered comments.

See the author affiliations on the bottom of the front page. We chose to NOT include names so to avoid it turning it into a battle of the egos. We just want it adopted.

And as for the audit formula it contains, it calls for is the R-SOB (Random Sample of Ballots), which only would require 5% of ballots at each precinct; which, of course, is unlike Jonathan Simon's call for 10% of Congressional races, it looks at all races.

But with that said, if people are going to quibble over 5 or 10%, I'm now just saying 10%.

Because in California, it would be quite do-able either way. Precincts in California cannot exceed 1,000. But almost never do more than 600 people show up. So 10% of 600 is a mere 60 ballots. I say use some high school students to come in and to the late shift as a field trip part of their high school curriculum, with supervision for the entire process.

It is currently legal for high school students to do particpate. Bowen last year had a bill to ask that the schools not get docked money if high school students worked as poll workers, but that got killed. However, they can still come in and do the late shift--if elder poll workers don't feel up to the task.

The main thing is: We need to count 10% of those ballots BEFORE they leave the precinct and go into the anonymous stream. We need to retain our equal partnership of vote counting with the government in contro. (Another story, is how up in arms I am about a new bill promoting total Vote by Mail in California authored by Jared Huffman.)

Hope I answered your question.

[Wilms]

12. I meant statisticians, not advocacy groups.

I agree with a precinct count but don't know if a fixed percentage is appealing. Actually, I do know. It's not appealing.

What I do like, is Stanislevic's model that fixes a confidence level (like 99% :) ).

And while Holt is pretty-much in the dog house, he did offer a few good ideas in his OTHER bill.

http://www.democraticunderground.com/discuss/duboard.php?az=view_all&address=203x469667

[Einsteinia]

17. Yes and Yes

The security portion, which is the bulk of it, was vetted by over 200 people, including election integrity groups

and

The audit formula for the "random sample of ballots" portion was written by Jerry Adams, PhD, who has vetted it with many statisticians, (e.g. Howard, Jonathan). They both believe 10% is better than 5% that Jerry asks for. I say, if we're going to forever quibble about 5 or 10%, let's just do the 10%.

But with that said, the Titanium allows each state can choose whichever random sample of ballots formula it deems is "best."

The importance of a "random sample of ballots" style audit is that it is not based on statistical weighting or other variables that can be contested in court.

The Titanium is chock full of checks and balances, and so it doesn't stop there: it wants two or more tabulation methodos, with separate oversight, all to come to the same conclusion within a margin that would not overturn an election.

[Wilms]

19. I don't need a PhD to know that 10% is better than 5%.

But these authors have shown that even ten percent can't provide a reasonable confidence level in some races. And that it would be overkill in others. And that a higher OVERALL CONFIDENCE LEVEL is possible for the same amount of effort.

These guys aren't calibrating for number of votes audited. They're methodology calls for the establishment of a desired CONFIDENCE LEVEL and counting how many/few ballots in order to assure not the count...but the OUTCOME.

To be sure of the count, you'd need a 100% Hand Count...which their method essentially would call for in a really tight race.

So fixed percentage counting is unappealing compared with auditing to a fixed percentage CONFIDENCE LEVEL.

[Einsteinia]

20. Wilms, I think you lost me, BUT

this is the assertion and please let me know if you disagree:

The Titanium asserts that for California 5% of ballots counted at the PRECINCT would confer at least 99% statistical confidence that the ballots cast are same ballots tabulated.

This is the ultimate in L&A testing and trumps all the other phoney "oversight" protections that we Californians now have, i.e. intermittent observation of people feeding ballots into machines that shows nothing and proves even less.

Jerry Adams contends that 5% is more than sufficent for California, but getting to your point of "Confidence," then if it takes our citizens a 10% figure to feel confident then so be it. As for the cost difference, it would be the difference between hand counting, on average, 60 ballots instead of 30.

If that's the issue that keeps us from moving forward, then I say, let's follow the precautionary principle and err on the side of excess.

Finally, 100% hand count at the county is less verifiably accurate than a 10% audit at the precinct.

[OnTheOtherHand]

21. that claim needs to be clarified

No sample count can "confer at least 99% statistical confidence that the ballots cast are (the) same ballots tabulated." A sample of 10% (or whatever) in each precinct will yield an estimate of the true vote proportion, with a margin of error (or, more generally, a series of confidence levels). It can't tell you whether the original count was right -- in any precinct or in all precincts -- although it can tell you if the original count is likely to have been way off.

A 10% sample in a California statewide race is certainly large enough (if everything else is shipshape) to confirm the winners of all but the very closest races. A 10% sample in a House race often won't be large enough to confirm the winner. (I'm using "often" loosely here: it's pretty good down to perhaps a 2-point margin or less depending on the confidence level you want, and close races are not all that common -- but they are the ones people tend to worry most about verifying.) If Californians want to count 10% of ballots in statewide races even when 2% audits would be ample, I don't object, but we shouldn't be purveying the idea that a 10% figure justifies confidence when it doesn't.

I see the appeal of 'instant auditing' -- although it seems to me that you are going to have to secure the ballots anyway, so it doesn't fundamentally solve any problem. (If the audit raises doubts about the outcome, what happens next?) It may be possible to combine 'instant' and after-the-fact sampling techniques. And it has been a lo-o-o-o-ong time since I've read the Titanium Standard proposal, so I may be forgetting important provisions.

[Einsteinia]

24. Details to Clarify

OTOH: Yes, with the goal of simplicity, I did incorrectly conflate the 99% statistical accuracy of the audit formula with the Titanium demands for security, which is that the formula take place at the PRECINCT.

Because, as to the security component, it seems safe to say that:

If we have voter verified paper evidence (as we now do in California) that we can be at least 99% confident while that evidence is in our control at the precinct, that the 10% random selection of ballots we sample, are indeed the same ballots that were cast--without manipulation and in their entirety.

However, once the ballots leave our control and are anonymously sent on their journey to a centralized location for tabulation, the degree of confidence must go down by some degree merely because of the number of people involved and the fact that a portion of the journey is unobservable to our citizenry. The fact is that even if we were able to afford to transport these anonymous ballots using armored trucks, etc., we could not confer the same level of confidence that the ballots cast are indeed the ballots tabulated as we can at the PRECINCT site.

In a nutshell: The Titanium calls for a paper ballots. Then it requires a precinct audit of at least 5% (more depending on the desires of each state), and it asks separation of powers and redundancy at every possible juncture -- with information publicly available to all at the same time in virtually real time. It then goes several steps further, by requiring "consilience," whereby two or more tabulation methods concur within a margin that would not overturn an election, lest state-paid hand counts would be triggered. Finally, the hardest sell of this whole thing is that no one can declare the winner until this whole process plays out.

For example, in California, immediately upon the close of polls the digital data would be sent to the election officials. Then we would randomly hand count 10% of all ballots cast (on average 60 ballots) at every precinct, and this total would be immediately available by way of public posting and websites. Then at the end of the evening, the paper evidence would be sent NOT to the county election officials, but to county AUDITOR's office for safekeeping. This would prevent "hacking the stack," (or making totals concur by force of shenanigans).

If the various totals differ by more than a margin that would overturn the outcome of the election, then a hand count would ensue.

And if, as you say, the 10% might not be adequate in tight Congressional races, and all this redundancy cannot improve that situation (which I doubt), then in the interest of practicality we'll revert to the concept of achieving "voter confidence" as the bar rather than accommodating the extreme worst case scenario. After all, it's got to be a huge improvement over current practices.

For your records, here's the Titanium: >>>> http://www.califelectprotect.net/Titanium.pdf

[OnTheOtherHand]

27. no necessary conflict

It would always be possible to go back and audit more if necessary, yes?

[Einsteinia]

29. Oh, YES, you're right

that does in fact solve it. Good!

[Stevepol]

3. A pdf containing information about Dopp's approach, one of several, is the following:

http://electionarchive.org/ucvInfo/US/EI-FedLegProposal-v2.pdf

[Bill Bored]

7. One of several? That's part of the problem.

First she was for probabilities;

then she was for probabilities with a vote shift assumption;

then she was for probabilities with a vote shift assumption and a precinct size adjustment.

All that made sense as far as it went, but then she started with those tiered audits because she heard that Holt was considering something similar. That was a bad idea because the tiered approach itself, as currently defined, is a bad idea. There aren't enough tiers for one thing; and the other disadvantage is that percentages are NOT the right way to calculate sample sizes in the first place -- integers and probabilities are. The tiers can also be gamed unless there are a LOT of them. (Add a few votes to the margin and the size of the audit is cut in half!)

If she had stuck with what worked best (the third option above), she would have been better off. Once you have something that works, why change it to something that won't? Stick with it and improve it.

[WillYourVoteBCounted]

11. ok, I like it

how many states have audits of the paper,
and do any states have an audit like this yet??

Also, you might want to put some pretty pictures or a
colorful chart in there for the simpleton laypeople like myself.

[Wilms]

13. I think this picture is pretty. Hopefully the authors can get their graphics on...so to speak.

http://deepkeel.com/2006/01/08/214

[WillYourVoteBCounted]

14. the sticking point I believe is that

the politicians are scared not to just name a flat out percent,
rather than trust each state to arrive at a certain probability.

When our state mandated audits of a "statistically significant" amount,
everyone slammed us for having a loophole that a highway of trucks could drive
through. (We couldn't agree on a number, so had to leave it open, plus no one could
argue against doing an audit of a significant number)

However, our SBOE actually hired a respected statistician to work something out,
and it turned out to be about 7% in some cases.

[Wilms]

15. Instead of quibbling over vote count percentage...

...argue about CONFIDENCE LEVEL!

I think it's a better frame, too. Ask, "Do you want to be 99% sure the right person was elected, or 50% sure?".

[Einsteinia]

18. and on that point, we can have NO confidence

in the 1-to-1 correspondence of a ballot cast to what is ultimately tabulated once our ballots leave each PRECICNT's citizen-controlled purview at and enter the anonymous stream.

[Bill Bored]

30. Confidence

Well, how about this:

1. You audit 100% of all precinct totals on election night, so that you know they match the down-stream tabulator totals. This is in the other Holt bill.

2. Then you audit the ballots in enough whole precincts to be 99% certain that there aren't enough corrupt precincts to change the outcome. This audit could be done later as long as the time between the announcement of the randomly selected precincts and the time the ballots are counted is minimized. That's the time window when someone could alter the paper ballots so you want to keep it short -- the same day if possible.

The machine counts, on the other hand, were indelibly recorded on Election Night in step 1, so they cannot be changed.

[Einsteinia]

31. Hmmm? How do you. . .

How do you "audit 100% of precinct totals"

What are precinct totals?

I assume they are not the paper evidence, but instead this is some sort of machine Logic & Accuracy testing???

If so, we still have a "1-to-1 correspondence" problem, in that we cannot be certain that the ballots tabulated are indeed the ballots cast as was demonstrated in the Princeton hack.

So, again, after much research over the past 3 years, I see no way around the citizen-controlled precinct 10% audit.

However, the *redundancy* thereafter, to serve as a check & balance, as in what you propose in item 2 sounds good to me.

[WillYourVoteBCounted]

22. I agree, I am just saying what the mental block is for lawmakers

I am guessing that the lawmakers can't get their minds around
the probable.

Its not me you have to sell, its the lawmakers who want something
that appears concrete.

I am with you, but you guys understand this better than me,
so I need your help is making it appear doable to lawmakers.

IF they think its open to alot of interpretation, they
won't go for it.

Thats what I mean.

The first 2-3 sentences have to make the message clear on
how this works and can't be easily abused.

Again, I repeat - I am for the probability formula, but its not
me who has to be sold on it.

[OnTheOtherHand]

23. well, some of what the lawmakers are worried about isn't clear

I have no problem with what you're saying, and I assume it is true of at least some lawmakers. There are various ways to reconcile a need for 'concreteness' with a need for confidence. I'm not sure that we will find a foolproof rhetorical formula on our own, but we figure that if we explain some of the issues to enough people, folks can find something that works.

You actually did that. Purists may cringe at "the size... shall be chosen to produce a statistically significant result," but the effect was to express (very cryptically!) a legislative purpose and to draw in some relevant experts to work on implementation.

Suppose we could get someone to pass a law that the minimum audit size, in precincts or audit units, would equal 200 (or so) divided by the winning margin in percent, with a minimum maybe 15 or 20? That would be pretty good in general. Would have to deal with some extreme size differentials (especially early and absentee votes). Ideally that would be a baseline for a confidence level, it wouldn't replace a confidence level.

[Einsteinia]

25. the Titanium proposes this formula:

The Random Sample of Ballots Formula (“RSOB-Formula”)

Determining the Recount Percentage
How to Determine the Percentage to Be Recounted

Example: Figuring the number of ballots to be counted at the county level for a
state with 3,000,000 ballots cast in order to verify a statewide race.

1. Divide the sample size by the total ballots cast in the state. This provides the
percentage of ballots to be counted.

2. Multiply the total ballots cast in the county by this percentage. (This provides
the number of ballots for the sampling procedure.)

For example, if 3,000,000 ballots are cast in a state, then the percentage of
ballots to collect for each county in that state is:

45,000 divided into 3,000,000 or .015 (1.5%).

Every county in that state would randomly sample 1.5% of its ballots.

For example, a county with 5,000 ballots cast would draw 75 (1.5% of 5,000) ballots.

NOTE: This formula may at first appear counterintuitive in the sample sizes required. One way to think about it is to realize that increasing a sample size beyond 45,000 ballots cast does NOT correspond to a significant reduction in the margin of error. Also, on the other end of the spectrum, when the total number of ballots cast is less than 250,000, then the corresponding proportional sample size required will increase sharply.

[OnTheOtherHand]

26. you sort of skipped the formula part (grin)

I mean, that's a formula, of course -- but probably the most useful information for many readers is the total sample size, not how the sample is allocated among counties.

Page 10 of the Titanium Standard shows that the total audit size in ballots counted ranges from 41,000 in the smallest jurisdictions described (100,000 to 500,000 ballots counted) to 45,000 in the largest.

That's in fact much larger than a 10% sample in congressional races. (I'm just reporting.)

(EDIT to try to clarify my dry 'humor')

[Einsteinia]

28. I have to admit

I am not qualified to speak to the statistical analyst side of this stuff, which is quite obvious. Thank you for catching what the Titanium does offer and where.

I'd be curious now that you've seen what's proposed if you have any major concerns?

And I could get Jerry on here, who is qualified to explain things better as far as the audit forumulas, if desired.

[OnTheOtherHand]

32. I hope I got a lot of it

Every once in a while big gobs of California get stuck to New Mexico in my brain, that sort of thing.

Concerns -- well, personally I prefer sampling precincts and counting the sampled precincts completely, because that gives you a better estimate of the voting machine error rate (or "error" rate, as the case may be). If you don't count entire precincts, you don't really know how well the machines are working unless they utterly misfire.

Also, if I'm right that you're looking at 20% or so audits in all House races, I think that is a very tough sell -- although if you can sell it and make it work, that's fine with me.

And, again, I think you ought to be prepared to go back and audit or recount more in very close races.

[Einsteinia]

33. No, it's. . .

No, it's not entire precincts (as you like) to show machine error, but it is 10% of ALL precincts, when looked at in their totality will tell us -- with 99% certainty -- whether the election result is accurate or not.

Again, if then the outcome of the 10% audit, based on all the precincts, differs from the computerized tally by a margin that would change the result of election,

Then a hand count will ensue.

What do you think of that?

[OnTheOtherHand]

34. well, like I said...

with a tweak for really close races, it should assure the outcome, so I like that part fine.

[Bill Bored]

35. I don't think 10% is enough for all races.

All this depends on the margin of error of the sample vs. the margin of victory of each race. And it even depends on turnout!

Example: You sample 10% of a House race with a total of 200,000 votes
and you sample 10% of another House race with a total of 100,000 votes.

You hand count 20,000 ballots in the first race and only 10,000 in the second one.

The first race will have a margin of error for 99% confidence of +/- .91% but the second race will have a margin of error for 99% confidence of +/- 1.29%.

If the margin of victory in both races is 2%, your audit will be much more likely to detect outcome altering fraud in the first race than in the second race, because the margin of error in the second race is larger than the margin of victory.

Also, you'd do all that counting with no additional information as to what might be causing the errors (or fraud) because the margin of error WITHIN each precinct is much too high to tell you anything. (Like the exit polls.)

So you will have counted lots of ballots and gotten very little information about what the problem might be if the numbers don't match.

If you can identify every ballot uniquely (in both the machine and the paper), you could do ballot sampling and count far fewer ballots for a high confidence level, but this raises voter privacy issues and also the issue of auditing the aggregated counts, which is easier to do transparently with whole precincts than with individual ballots, I think. You would also need an electronic copy of every paper ballot. It's a pipe dream but it's possible if all the above problems could be solved.

We are so far from even getting folks to understand the basics though that almost no one will have confidence in the pipe dream, even if it works. And we do want there to be voter confidence right?

[Einsteinia]

36. Two Things:

1. Yes, the rate of confidence does improve with larger samples. But I believe 10% is sufficient in almost all but the most extreme instances. And in those cases, it's important to remember what I am proposing is more than one method with separate oversight. I think the redundancy. I think the more uniform and simple to achieve the highest standard, on average, is more than sufficient for our purposes. Don't you?

2. I don't understand why you think privacy of the ballot would be infringed just because it's at the precinct. The way I envision it would be: The voter reviews their, for example, Optiscan ballot then places it in a box. At the end of the evening the poll workers take x number of ballots out of the box to be included in the sample. (Of course, in reality, it's a bit more complicated because we have ballots in different forms, i.e. handed in vote by mail ballots, disabled voters ballots.)

[Febble]

37. hmmm....

I'm a bit worried about the "taking X number of ballots out of the box" part. They won't be randomly stacked. How do you randomly sample ballots from the box?

[OnTheOtherHand]

39. Appendix A discusses that -- it's a bit iffy

It seems to offer two alternatives: (1) program the scanners to stop on every "n"th ballot, or (2) rig up a device with cut-down cardboard boxes and a yardstick that you can use to identify every "n"th ballot. (Personally, I'm not that handy with an Exacto knife, but you could probably pull it off.)

I would certainly want it to be pinned down in actual legislation. I think that manually counting off "n"th ballots is probably preferable to messing with cardboard boxes -- and some error in counting "n" shouldn't matter as long as there is no room for outright discretion. I might run that by a magician (seriously).

Another problem with the document is that it blithely asserts that "independent election witnesses, such as the Carter Center," rely upon exit polls; that "properly conducted" exit polls are accurate within a 1% margin of error; and that they should cover every (non-primary) race for Congress(!!), governor, and president.

I had thought that this was a random one-off nod to someone's hobby horse obsession, but it may actually be a bit worse than that. Pages 5-6 somewhat ambiguously indicate that an election result is validated ("consilience is achieved") if there is no outcome-altering discrepancy between (a) official tally versus random sample of ballots ("RSOB-Formula") "or" (b) ballot sample (RSOB) vs. exit polls "or" (c) RSOB versus "spot-check method" (sampling precincts).

I don't understand this. The official tally is what we are trying to check. If the official tally essentially matches the RSOB, and the exit polls are way off, surely only the truest of Exit Poll True Believers would take this as reason to question both the official tally and the RSOB?!

[Febble]

40. Magician sounds good

I'd want James Randi.

Oh how I wish that Exit Poll arguments would Go Away!

They devalue everything they touch.

[Einsteinia]

42. Aha!

I think you have caught the weak point in this proposal. That whole exit poll thing was added in and I think it's needlessly redundant and in fact problematic--as you say.

The point was to not dismiss exit polls. I take it that you disagree?

[OnTheOtherHand]

43. I guess I would not use that word

If memory serves, Febble and I have actually been accused (by others!) of "dismissing" exit polls, which is ironic given the time we've put into analyzing them. But I think you put it well: needlessly redundant and in fact problematic. Any given exit poll may or may not be accurate for reasons that have nothing to do with the integrity of the count. I don't obdurately oppose experimentation with election verification exit polls, but I wouldn't explicitly fold them into an audit protocol.

[Einsteinia]

44. Superbly stated. . . .

& Yes, I'll push to remove that problematic exit polls rendundancy in the next edition

[Febble]

46. Good job!

:applause:

And thanks for the clarification re ballot sampling :)

[Einsteinia]

41. The Appendix allows. . . .

Just a quick back up. We had the Gold Standard, and then we hooked up with Oregon activists to make a audit protocol for their state because their vote by mail system had never been audited, etc.

So, we decided to take the essence of our Gold Standard and it something large states and small states could use.

Back to the point: We left the appendix to be how each state could interpret things like how to determine random. For California, I propose setting all the stacks on a table and then count up all the ballots. Then here are just a couple of proposed solutions:

1. Drawing a Genuinely Random Selection.
The state-paid verification process can be easily subverted if the selection samples are not
genuinely random. A random selection cannot be based on any information that is already
existent before the moment of selection, and it must pass the scrutiny of at least three state
certified gaming officials. In the absence of good solutions, most states already have lotteries that
can be accessed to draw up random numbers. There are a number of methods that fulfill the
physical pulling of the random sample of ballots needed. See Appendix G.
--------------
Every Nth Ballot
The fastest way to conduct a random sample when an optical scanner is used is to have the
scanner stop every nth ballot (“n” is defined by dividing the anticipated voter turnout by the
approximate sample size, e.g. 1,400,000 voters divided by 40,000 sample size results in pulling
every 35th ballot), have election workers pull and note the vote by hand, put the ballot back in the stack, and keep going. See Appendix C for an alternative method if the
scanner cannot be stopped on a predictable basis.

This method which builds verification into the process of the initial counting ballots may be the least disruptive and fastest way of doing it. It also allows the results to be verified on the day of the election.

If the counter/tabulator cannot be stopped at a prescribed place, the following method could be
used. Put n blank ballots into each of two piles. Cut two cardboard boxes, the type that paper
comes in, to exactly the height of the two piles. Put a stack of n ballots in each box. Place an
aluminum yardstick or similar straight device over the two boxes, with a 12" space between them.
Pull a random ballot out of the first n real ballots before feeding any into the scanner. Start with that ballot as the first verification ballot. Count the next n ballots after the verification ballot by placing them between the boxes to measure n ballots. (When you can't get another ballot into the stack between the two boxes of blank ballots, you have n.) Count each "verification" ballot and then restore it to its place in the stack; feed the stack into the scanner.

[OnTheOtherHand]

38. actually, their document doesn't call for a 10% audit

http://www.califelectprotect.net/Titanium.pdf

In fact, it expressly argues against a flat percentage. It is based on the sample size needed to confer a 1% margin of error (on the margin) with 99% confidence, which tends to be a bit over 40,000 votes for jurisdictions/races from 100,000 votes up.

You're certainly right that a 1% margin of error isn't good enough for all races. And 40,000+ ballots is a lot of ballots in an uncompetitive House race.

If there is a discrepancy over 1%, then I think all the ballots get recounted, so that will yield more information. But smaller errors (under 1%) could go undetected for quite a while, which I don't like from a QA perspective.

[Einsteinia]

45. attaining legislation: 10% or flat rate

Do you think there's anything wrong with just saying we want a random sample of 10% of all ballots cast at all precincts?

It's harder sell to have to go to the public and legislators and ask for what the "R-SOB Formula" determines, which would almost always be under 10% of ballots. . . .

It's a mouthful and why I've been just saying 10% for simplicity's sake in selling the idea.

Or do you have any suggestions on this?

[OnTheOtherHand]

47. oh, I had wondered about that

Just for your own sake, I'm not sure it's a great idea to say that the Titanium Standard calls for a 10% sample when it actually doesn't. (I don't know what the process was, but if I and Febble hashed out some sort of recommendation, and then she went off and said we had agreed on something else, I might hit one of my legendary sour notes.) Of course that's a separate matter from the merits of a 10% sample.

If you want to audit House races effectively (to verify the winner), a 10% sample would probably suffice down to about a 2-point margin. After that, it gets iffy. Most races don't have 2-point margins, so that isn't so bad as long as you are prepared to "go higher" in the close races. (RSOB will consistently be considerably more than 10% of ballots in House races -- certainly less in CA statewide races.)

What can I say? your own document makes the case why it isn't a great idea to rely upon a flat percentage. But if one must, a high flat percentage will usually work provided that people are prepared to accept it.

I think a lot of us are still worried that a 10% election night sample is a logistical loser, at least in some places -- but I'm not going to pound the table on account of my hunches and assumptions about such things. (Of course I think you should pay close attention to the logistical concerns, valid or not.) There is more than one way to get to high confidence, and different states may well take different approaches depending on size, institutions, culture, luck, maybe even whimsy.

[Einsteinia]

48. Great feedback & one last thing

It seems you're talking about the lack of security "srong arming" that has been known to occur in some precincts. It's true bad stuff can happen at precincts.

And it's true bad stuff can happen by election officials at the central (county) tabulation.

I think we take a sample at the precinct before the ballots enter their anonymous stream, AND we compare it to the digital data sent to the county, etc., only then can we have something airtight (Titanium) or whatever metaphor for that you want to use.

The important concept that a government of the people ought to have an EQUAL partnership with those in power--and not to have us banging on the window looking inside, begging our election officials (such as Blackwell, Harris) to do right by us.