U. S. ELECTION ASSISTANCE COMMISSION
VOTING SYSTEM TESTING AND CERTIFICATION PROGRAM
1225 New York Avenue, NW, Suite 1100
Washington, DC. 20005
May 2, 2008
To: Election Officials
From: Brian Hancock, Director
United States Election Assistance Commission
Testing and Certification Program
RE: An update on the state of the EAC’s Testing and Certification Program
All,
It has been 15 months since the EAC opened the doors to its voting system testing and certification program, and although much has been accomplished in that time, we have also heard several concerns regarding our program. I wanted to take this opportunity to address some of the concerns that have been expressed to me by state and local officials in recent months and to make you aware that an annual report outlining all our activities, in detail, will be forwarded to you within the next several weeks.
Current Program Challenges Since the initiation of our Certification Program, there has been much anticipation surrounding the EAC certifying its first system. There have been many concerns expressed as to why the EAC has not yet certified a voting system and if any systems will be certified in time for the 2008 General Election. I want to take this opportunity to address those concerns and explain why we are being deliberate in our review of applicant voting systems.
The EAC Certification Program represents the first time the Federal Government is testing and certifying voting systems. Prior to the creation of the EAC’s program, voting systems were, as many of you know, qualified by the National Association of State Election Directors (NASED). NASED operated with little to no funding or staff dedicated to this effort. When the EAC’s program was established in January of 2007 the EAC made the difficult but correct decision not to “grandfather” NASED qualified systems. Instead the EAC chose to start anew and require that all systems applying for EAC testing and certification must be fully tested as a new system under our program in
order to receive an EAC certification. This decision not to grandfather NASED qualified systems had the predictable effect of lengthening the initial testing and certification process. It is important to note that after a system receives an EAC certification any modifications to that system will be tested under a more streamlined process which will test the modification (delta testing) and those systems or subsystems altered or impacted by the modification (regression testing). The system will then be subjected to integration testing to ensure overall functionality. In addition, the EAC program has developed procedures to deal with De Minimus changes to a voting system and to deal with emergency Pre-Election modifications to EAC certified voting systems. These procedures are outlined in Sections 3.5 and 3.6 of the Certification Program Manual.
During our first year of operation, the EAC’s Certification Program has experienced reluctance from many of the participating parties to accept the need to change the very culture surrounding the testing and certification of voting systems. The EAC has intentionally moved to shift the paradigm away from the reliance on previous practices and towards a more structured and rigorous process. The EAC understands the need to have certified systems available to jurisdictions and appreciates that many states rely on EAC certification as a prerequisite for implementing a voting system in their State. That is precisely why the EAC is committed to fostering a certification environment that fully evaluates the voting systems against the applicable standards in order to ensure that they are capable of performing securely, accurately and reliably in an election.
As a result of this new certification paradigm, the EAC has seen several predictable issues arise that have had the effect of slowing the certification process. Voting system manufacturers were slow at times to adjust their business practices to the new requirements of the EAC’s program. Over the past year the EAC has issued three notices of non-compliance to manufacturers who have failed to abide by the EAC’s program requirements and requested them to come into compliance with the program or face possible suspension of their registration. In addition the EAC has issued numerous letters to manufacturers regarding the key differences between the EAC and NASED program and the need for manufacturers to implement additional measures to meet the EAC’s program standards. Also, the EAC has held several meetings with its registered manufacturers in order to create a better understanding of our expectations and to better open the lines of communication between the two groups. All official correspondence between the EAC and the manufacturers can be found at
http://www.eac.gov/voting%20systems/voting-system-certification/correspondence. The EAC has been pleased to see that in general the manufacturers have been willing to cooperate and even more willing to offer ideas on how better to streamline the EAC’s process.
The paradigm shift has also had a profound effect on the Voting System Test Laboratories (VSTL). The differences between the NASED and EAC’s program has greatly affected the way the VSTLs (particularly those who had formerly participated in the NASED program) prepared test plans, issued test reports, and evaluated the voting systems. As evidenced by the multiple drafts of test plans received for each voting system, the VSTLs had trouble adjusting to the level of detail required for an EAC
approved test plan. However, the VSTLs have shown continuing improvements to their processes and procedures and are now offering test plans that have the level of detail necessary to receive approval from the EAC.
Another challenge the VSTLs have encountered is the current lack of a standardized test methodology. The EAC and NIST are currently pursuing ways to expedite the test methodology development process in order to provide the VSTLs a set of test methods for the 2005 VVSG in the near future. This set of test methods will greatly increase the efficiency of the test plan and test report review process since the general test method will remain consistent from system to system with changes incorporated to accommodate the unique variations between different voting systems.
The VSTLs are currently undergoing the required reaccreditation review by NVLAP. The first laboratory accredited by EAC/NVLAP, SysTest Labs. Inc., underwent its review in early March and the second laboratory, iBeta Quality Assurance Inc., had its review last week. It is the EAC’s intent that the NVLAP reassessments will serve to validate the EAC’s expectations for test methodology, experience and training of test engineers, and quality monitoring for systems while in the testing processes. The EAC remains extremely confident in its VSTLs and appreciates their willingness to cooperate with the EAC and in many cases change their laboratory practices in order to meet the expanded requirements of the EAC’s program.
Conclusion The passage of HAVA and the creation of the EAC’s Certification Program has created a new environment for the testing and certification of voting systems. The basic premise and procedures of the certification program are sound and borrow important aspects from successful certification efforts in other agencies. In addition, the certification program model is based upon U.S. and international Standards (ISO/IEC 65) used in numerous other industries. The full benefits and value of the EAC program will only emerge with time and experience.
Other Federal certification efforts reflect the importance of allowing due time to develop mature programs and processes. The Federal Communications Commission (FCC) was created by the Communications Act of 1934 and has been testing and certifying communications equipment since that time. Depending on the FCC workload, certification of products averages from 60 to 100 days. To speed up certifications (and time to market for manufacturers) in 2000, the FCC allowed manufacturers of certain kinds of equipment to self-certify their products under a Declaration of Conformity program. It also instituted a program of allowing contract examiners working at organizations called Telecommunications Certification Bodies (TCBs) to perform the test report and certification function under FCC authority and supervision. There are approximately 13 TCBs and over 500 test labs doing the testing and preparing the reports that they review. The TCB examiners now average approximately 35 days to review a test report and certify a product. It is important to remember that these products need only pass 30 or fewer tests to achieve certification. Voting systems, on the other hand, must conform to approximately 1100 separate testable requirements and each test report is a very custom document, often exceeding several thousand pages.
Non-Federal certification efforts also reflect efficiencies made possible only by hard won experience and continuous work with the program participants. The Nevada Gaming Control Board certifies all gaming devices used in the State of Nevada. While gaming systems are significantly different than voting systems, they are nevertheless similar in that they must engender the utmost trust and confidence by their users and be highly secure from tampering. The Gaming Control Board started testing gaming devices with the emergence of microprocessor controlled slot machines in the late 1980’s and early 1990’s. Today, the Board runs its own testing laboratory supported by a Technology Division of over 60 employees. New gaming system approvals (certifications) take anywhere from 6- 18 months. Additionally, before the Board grants final approval to anew gaming system, that system must undergo a field trial (run in a sanctioned casino) for no less than 60 nor more than 180 days.
These examples show that conformity assessment testing is neither an easy nor a quick process, but that a commitment to continuously improve the process does ultimately lead to successful certification programs and confidence by the end user of the system.
Unfortunately for the EAC, VSTLs, manufacturers, and election officials, the 2008 General election cannot be postponed to allow the EAC’s program to grow and mature. The EAC is in the difficult position of needing to develop its program while actively testing and certifying voting systems. Because this challenge is so great the EAC is being very deliberate in creating a program that will be effective, useable and cost effective for the States in the long term. Simply rubber stamping systems through our certification program would only serve to undermine the confidence of both election officials and voters in our program. The EAC understands that election officials across the country are depending on us to get certified systems to them as soon as possible. We are committed to proceeding expeditiously but prudently to certify voting systems. Efficiency will improve as the program matures and all involved become more familiar with our systematic and rigorous approach to voting system certification. Currently, we are pursuing with States ways to facilitate cost efficiencies in the testing and certification process at both the Federal and State levels. The ultimate goal of this project is to affect cost savings for the States by moving as much repetitive (and often expensive) testing into the Federal process, hopefully saving States both time and money.
I hope this letter serves to inform you on some of the issues facd by the EAC Certification Program over the last year. Please check the EAC’s website www.eac.gov for any information you may need, we work very hard to ensure that any information we have is provided to you. If you have any questions regarding this letter or the EAC program in general please do not hesitate to contact me.
http://www.eac.gov/voting%20systems/docs/eac-cert-program-update-to-election-officials-5-2.08/attachment_download/file
Printer-friendly format
Email this thread to a friend
Bookmark this thread
(1000+ posts)
Send PM |
Profile |
Ignore
