A comparative test of web accessibility evaluation methods
Author:
Open Access
- 13 October 2008
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 113-120
Abstract
Accessibility auditors have to choose a method when evaluating accessibility: expert review (a.k.a. conformance testing), user testing, subjective evaluations, barrier walkthrough are some possibilities. However, little is known to date about their relative strengths and weaknesses. Furthermore, what happened for usability evaluation methods is likely to repeat for accessibility: that there is uncertainty about not only pros and cons of methods, but also about criteria to be used to compare them and metrics to measure these criteria. After a quick review and description of methods, the paper illustrates a comparative test of two web accessibility evaluation methods: conformance testing and barrier walkthrough. The comparison aims at determining merits of barrier walkthrough, using conformance testing as a control condition. A comparison framework is outlined, followed by the description of a laboratory experiment with 12 subjects (novice accessibility evaluators), and its results. Significant differences were found in terms of correctness, one of the several metrics used to compare the methods. Reliability also appears to be differentKeywords
This publication has 10 references indexed in Scilit:
- Beyond Conformance: The Role of Accessibility Evaluation Methods2008
- The relationship between accessibility and usability of websites2007
- Accessibility 2.02007
- Web Accessibility Testing: When the Method Is the CulpritLecture Notes in Computer Science, 2006
- Is your web page accessible?2005
- Criteria For Evaluating Usability Evaluation MethodsInternational Journal of Human–Computer Interaction, 2003
- Usability inspections by groups of specialists2002
- The Evaluator Effect: A Chilling Fact About Usability Evaluation MethodsInternational Journal of Human–Computer Interaction, 2001
- Damaged Merchandise? A Review of Experiments That Compare Usability Evaluation MethodsHuman–Computer Interaction, 1998
- Heuristic Walkthroughs: Finding the Problems Without the NoiseInternational Journal of Human–Computer Interaction, 1997