Research team

Introduction

Research Approaches

Timeline

Results & Publications

Advisory Board and Funding Agency

Participation

Results

SERPA: Streamlined Evaluation and Reporting Process for Accessibility

Introduction & Purpose

Since the mid 1990s, there have been hundreds of articles about the mechanics of making websites usable for people with disabilities. Recently, some researchers have begun investigating why, with so many design resources available, a majority of websites are still inaccessible.

We believe there is a need to look beyond the "how-to" of designing for accessibility and consider the constraints, context, and roles of the people involved in actually making sites accessible.

With the following new process our ultimate goal is to improve the accessibility of the site for end-users, by meeting the needs of “end-users?of accessibility recommendations embodied in reports and conversations with accessibility specialists.

A step-by-step process (SERPA) is outlined here for use by website accessibility evaluators, to facilitate implementation of accessibility fixes, focusing on the site's programmers. These steps are proposed to complement existing resources on coding, site evaluation processes, and accessibility guidelines.

It is hoped that this process will result in reports which programmers balance the volume and scope of fixes needed to improve site accessibility, with the constraints imposed by work capacity, and typical programming and design psychology.

SERPA Steps

The following is a suggested outline only. It can be modified as needed to meet your organization's needs. It can be used when retrofitting a site for accessibility, or as part of a process which includes accessibility in the building of a new site from the ground up.

Step 1: Discuss the needs and the project goals with team members

The first step is an informal discussion that enables the accessibility specialist to develop a better understanding of project goals and development team member needs.

The main thing to bear in mind is that an accessibility specialist must know the goals and objectives of the other team members so that he or she will be able to provide recommended fixes which meet their needs and goals.

If accessibility is being introduced for the first time, many development team members may see this as additional work that they do not have time for, so you may initially need to use tact and diplomacy until it can be shown that the fixes do not involve an inordinate amount of extra work for the programmers.

Step 2: Establish an agreed-upon scope and resources

The altruistic tendency may be to attempt to "champion" the cause of end-users with disabilities by recommending, for example, meeting all three W3C priority levels. However, this might requires a big budget for extensive testing. This approach can easily backfire, causing accessibility efforts to get cut drastically or completely.

Focus the accessibility evaluation and reporting to a manageable, agreed upon scope.

Step 3: Prepare a programmer-centric report template

The evaluation report you keep in your file cabinet does not have to be the same one you give to the programmers. They generally have very little time, and get testy when you dump a hundred pages of automated evaluation tool outputs on their desk. So, before you even start, work out the template for the report in consultation with the programmers. Ask what they want, and try to give them things in the format they can easily use. A few points to bear in mind:

Make a list of fixes first; fill in checklists second

If you start with accessibility guidelines/checklists and go looking for instances where the guideline is violated, you could be looking for needles which may or may not be in a big haystack.

Instead, evaluate the site first for usability by people with disabilities, and then fill in checklists of the problems you found. You should find that this speeds up your process dramatically (i.e. a number of hours evaluating the site and then 5 or 10 minutes filling in the checklist).

Separate content-related from presentation-related findings

Programmers generally program.

Content-providers (i.e. the language displayed on the site) are usually not programmers. Content providers work in a different department.

Give content fixes to content providers, and programming fixes to programmers. And make sure they talk to each other so that al the needed fixes get onto the published website.

Identify easy fixes versus more difficult fixes

Many evaluators are not programmers and so find it difficult to demonstrate solutions to complex problems with coding; but then again many programmers will not want to be told how to code by "armchair quarterbacks".

More complicated coding problems (e.g., how to create an expanding menu that can be used by both the mouse and the keyboard) may offer programming issues specific to the programming tools used. They may even offer an enjoyable challenge to the programmer. In these cases, presenting the problem in terms of what the end-user with a disability experiences, and which guideline is not being met, is enough.

Keep in mind your end-users, the programmers

The end-users of your report are not people with disabilities. The end-users of your report are programmers. Managers might like to skim it, but the people who will use the report will be programmers. Write the report with their needs in mind.

Step 4: Evaluate the website for accessibility

Use pragmatic approaches for visually capable evaluators. Employing someone with a visual disability to use the site and give feedback may be useful, but in most organizations the people evaluating the site are visually capable. It makes sense that if you are visually capable and do not have ready access to someone who regularly uses the web non-visually, you should use methods that are highly efficient for visually capable evaluators, as described below.

Use a cheap and simple screen reader / voice browser.

JAWS is the most commonly used screen reader in North America. However, JAWS is designed to be used nonvisually. It takes a long time to learn, and you may find you spend most of your time struggling to master it rather than evaluating your site. It is also quite expensive. Home Page Reader (HPR), on the other hand, is cheaper and has more visual features which will help you follow the screen reading process. HPR is not as advanced as JAWS or other similar screen readers, and it only works on web pages (not other desktop applications).

If you need to do a double-check with JAWS, seek out a JAWS user to do a double-check for you after you have done the first round of fixes to your site.

Reduce your own vision

The web is primarily a visual medium. Accessibility specialists have traditionally focused on screen reader accessibility, but reduced vision is also important to consider.

You can blur your own vision using low vision simulation glasses, available from Lighthouse International (www.lighthouse.org).

You can also attaching your PC or Laptop to a projector and blur the focus.

The goal is to reduce the visual function while using the interface, and then use technologies available to people with disabilities to compensate. While blurring the screen:

enlarged text can be used on the browser (to make sure your fonts scale up and down in size)
a screen magnifier is built into Mac and Windows operating systems

Use only one tool to go through pages for final checks

The W3C lists a whole bunch of tools to evaluate and repair accessibility. If you have time to look through them all, then by all means do. We recommend using only one tool, "WAVE", which generates output that is highly graphical and readable for visually capable evaluators. If you look at a representative sample of pages with WAVE, you should be able to find most of the accessibility problems with a site.

The rest of the evaluation

Taking into account the above considerations, use the remainder of the W3C recommended method for "Preliminary Evaluation" to evaluate the actual site.

Step 5: Report progress scores

When reporting to programmers, if there are 500 instances of one type of problem, treat it as 1 problem. Explain and describe the problem only once, tell the programmers it happens in many places in the site, and let the figure out how to fix it.

Report numerical scores of progress. We use an Accessibility Programming Progress Formula (APPF):

T = Total number of guidelines (checklist) items (this number depends on which source of guidelines you are using, e.g., Priority 1 and 2 of the W3C guidelines)

A = Guidelines already met
B = Sort of met some places
C = Not applicable
D = Broken and need fixing

To give a numerical score of progress:
APPF = (A+B) / (T-C)

Step 6: Hand over only what is necessary

It may be tempting to include everything about the evaluation in the report, but voluminous appendices as justifications for changes, and recommendations for further work beyond that in the scope of the project, are not going to win you any friends in the programming department.

We recommend a simple report structure as follows:

Very brief overview and progress score (APPF)
A list of programming fixes which are needed

Simple fixes should be described in terms of what the problem is, perhaps an example of where it occurs, and what the suggested programming solution might be, if appropriate.
For more involved fixes describe the problem and how the interface should behave for users with disabilities, then leave finding the programming solution to the programmer.

That's it!

Add as an appendix the guidelines checklist with indications of how you rated them (see APPF, above).

It is not recommended that you hand over all of your evaluation materials, printouts, data etc., but you should make this material available for inspection should the programmers need to see it to help further understand or deal with problems.

Step 7: Follow-up after programmer changes have been made

Do not leave it at that after the main fixes have been made. Regularly inspect the site(s) and meet with programmers to fix glitches and address new issues. If appropriate, keep a running chart of the APPF scores.

Acknowledgment

This work is part of the Universal Design in Practice (UDiP) project at the School of Industrial and Systems Engineering, Georgia Institute of Technology, funded by the National Institute on Disability and Rehabilitation Research (NIDRR), US Department of Education, grant number H133G040151. The opinions and content are those of the grantees. They do not necessarily represent the policy of the Department of Education.

Home | Contact