What Don't We Know About Internet Voting and Usability?

by Whitney Quesenbery

Unfortunately, we know very little about the user experience of voting and what affects it. In the absence of substantive research to report on, this segment of the workshop will focus on what we do know about usability in official public access systems and what we have learned from other Internet applications. These issues will be discussed in an open forum, exploring how the legal, technical and security requirements for a voting system interact with usability issues.

The Usability Challenge of Voting

Voting may be one of the most difficult usability challenges because it is a task completed by virtually anyone, it is done infrequently, it is never exactly the same because the actual ballot differs for each election, and privacy requirements make it difficult for voters to seek help in using the voting system.

We have taken few opportunities to study the usability of voting systems. This is, in part, because the system was generally regarded as working. Errors rarely made a statistical difference in the election results and the task itself is not seen as difficult. In addition, the privacy issues which surround an election can make it difficult to construct meaningful situations for observation. It is not impossible, however. Susan Roth King, the author of one of the few papers on voting and usability, reports however that the people in her studies were "willing and able to provide constructive and valuable feedback" - if someone was just willing to ask.

What Is Usability?

Usability describes a quality of a product, whether that product is a web site, a software tool, a physical object or some combination. The ISO standard on usability (ISO 9241:1998) defines it as:

"The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use"

This definition reflects the origins of usability practice in the ergonomics of work. Except for the relatively weak final term, "satisfaction," it does not address the emotional aspects of the user experience, focusing instead on measurable metrics such as time on task or error rates. Colloquial definitions tended to use softer terms, such as "user friendly" or "easy to use" as a way of shifting attention to the user and their perception of the experience, especially in regard to initial barriers to effective use. Professional discussions have suggested several means of broadening the definition to take into account different types of products. I summarized these as "the five E's" or the five dimensions of usability. They are:

• Effective - the completeness and accuracy with which users achieve their goals.
• Efficient - the speed (with accuracy) with which users can complete their tasks.
• Engaging - the degree to which the tone and style of the interface makes the product pleasant or satisfying to use.
• Error Tolerant - how well the design prevents errors, or helps with recovery from those that do occur.
• Easy to Learn - how well the product supports both initial orientation and deepening understanding of its capabilities.

Each of these dimensions are important in varying degrees for all products. They are of interest as a design tool because a clear statement of how they apply to the domain acts as a guide for decision making, especially when trade-offs are needed.

For example, in a voting system, an analysis of the dimensions of usability might be:

• Effective - The correct casting of votes is the primary goal for users of the system, so the interface must provide strong cues and feedback towards that goal.
• Error Tolerant - Paired with "effectiveness," the system must be able to help prevent errors such as under- or over-voting, but must not interfere with the ability of the voter to make their own decisions or be overly intrusive.
• Easy to Learn - The voting system will be used by a broad cross-section of citizens, with different abilities and comfort levels. The system is used infrequently, and voters will typically have little or no advance training. However, it is also important that any familiarity voters do gain is not "thrown out" with new interfaces from election to election.
• Engaging - The tone of the interface should not interfere with the voting process or cause anxiety. The wording of prompts or other instruction should be carefully tested to ensure that it is unambiguous and unthreatening.
• Efficient - Within tolerable limits, efficiency is not a major consideration in a voting system, but its use should not seem tedious or include annoying repetitions.

Usability Guidelines

In 1990 (and revised in 1994), Jakob Nielsen published a list of heuristics for usability. These guidelines are important because they are not specific to any one application or context, but provide a common base for discussing the usability of a variety of products. They can be refined into more specific guidelines for voting systems, and can provide a means for comparing the usability strengths and weaknesses of different voting systems. The most important guildeins for a voting interface are:

• Visibility of system status
  The system should be transparent, leaving no doubt as to how a vote will be cast, or the current location of the voter in the system.
• Recognition rather than recall
  The voter should not be forced to remember instructions or options from screen to screen
• Error prevention and Help
  U sers recognize, diagnose and recover from errors. The system has an opportunity to prevent mis-cast votes, especially those through under- or over-voting.
• Aesthetic and minimalist design
  The visual presentation should help voters make choices using good typography and other presentation techniques, not obscure the meaning with unnecessary graphics.

Why is Usability Important

Many people I have talked to have wondered why usability is important in a task as simple as voting. One answer is that it is only deceptively simple: the CalTech/MIT Voting Project concluded that there are opportunities for error at every step of the process, including voter authentication, ballot preparation by voters, verification and deposit, tabulation.

Part of the answer lies in the question of what error rate we are willing to tolerate in each step. Perhaps the goal should be a system with no user errors, rather than an acceptance of designs that disenfranchise a percentage of the electorate.

"There should be no need for disqualified ballots. Each voter should get immediate feedback from the voting machine if they have voted improperly, and should be given a chance to correct their ballot, or fill out a new one." -- Paul Resnick, University of Michigan

A saying in the usability community is that "to the user, the interface is the product." This means that the presentation and interaction layers are just as important in voter perception (and acceptance) as the underlying technology. Usability is important because it provides guidance as to how standards should be met, and can be the link between standards and operational guidelines. Too often, systems are designed without a clear model of the user experience in mind. The result is an interface that is patched together on top of a collection of sub-systems, rather than a system in which all of the elements come together to create a seamless, usable experience.

Topics for Discussion on Internet Voting Usability

Each participant in the workshop was asked to suggest topics for discussion by the workshop.

How can we guarantee an appropriate level of usability in the initiation and authentication stage of the voting process? What kind of design process and usability testing is needed?

Even when the actual application is easy to use, the installation and individual system setup can still cause usability problems. For example, in the Voting Over the Internet Project, "Of the 157 digital certificates issued to volunteer voters, nearly half (75) had to be replaced. Approximately one-third of these digital certificates were reissued for expired documents; the other two-thirds were reissued because of forgotten passwords." With just 91 people registered to vote in the project, and 84 votes cast, the project help desk logged 71 support calls.

"As with most computer systems, increased security and higher levels of privacy can be provided by increasing the complexity and the burden on the user of the system. The success or failure of Internet voting in the near-term may well depend on the ability of computer programmers and election officials to design a system where the burden of the additional duties placed on voters does not outweigh the benefits derived from the increased flexibility provided by the Internet voting system."

The California Internet Voting Task Force
"Report on the Feasibility of Internet Voting", January 2000

What are the requirements for readability of the on-screen ballot to ensure that voters of all ages and physical abilities can read the ballot?

The current resolution of a typical computer screen is vastly inferior to paper (72 dpi vs. 2400 dpi). As part of the design research for an online testing service, readability tests were conducted at a range of screen resolutions from 800x600 to 1600x1200. The results showed dramatically fewer comprehension errors at the higher resolution. The California Task Force recommends that "No contest, either for an office or a proposition, should be split across two pages." With large numbers of candidates in some races, screens may be very crowded.

How can we determine the differences in effectiveness (a dimension of usability: how accurately the vote is cast) between full-face and paged ballots?

The use of a computer screen most likely implies a design that uses a aged ballot rather than a full-face ballot because of the relatively low screen resolution. Although several current voting technologies used paged systems, there may be added usability issues because of the addition of navigational elements to the voting interaction, as well as making it more difficult for the voter to see, at a glance, all of the votes. The California Task Force recommends "Multi-page ballots should be easily navigable by voters, with no way to get lost or leave the balloting process except deliberately."

What usability testing is needed for the general ballot format; and what for each specific ballot, and what is the right methodology for this testing.

In general, ballot designs are examined for accuracy, but not for usability - quality tested, rather than usability tested. After Florida's 2000 election, the Palm Beach County Supervisor of Elections, Teresa LaPore said, "I was trying to make the print bigger so elderly people in Palm Beach County can read it. We sent out sample ballots to all registered voters, and no one said a word."

How can we be sure that a confused voter (or one afraid that the system is failing in some way) will be willing to ask for help? What kind of training will be needed for poll workers to feel confident assisting voters?

The California reports suggests that, "It will be essentially impossible (to) offer phone support to help voters who have technical difficulties voting from institutional computers. Vendors should prepare and counties will need to distribute an extensive instruction sheet for voters." We have evidence from the events in the 2000 election in Florida that voters do not ask for assistance at the polls. Caroline Jarrett, a usability consultant for the Inland Revenue, suggests that people "don't realize that they have a right to complain if they can't understand official forms" and that they will only complain when "the effort of making a complaint has some sensible relationship to the chance of the complaint making a difference."

For more information...

• UPA Voting and Usability Project
• Other articles on voting by Whitney Quesenbery

This paper was presented at the Georgia Tech Research Institute Internet Voting Workshop, held November 13-14, 2001
The URL for this article is: http://www.wqusability.com/articles/voting-gtri.html