Evaluating the effectiveness of automatic image captioning for web accessibility
Open Access
- 22 August 2022
- journal article
- research article
- Published by Springer Nature in Universal Access in the Information Society
- Vol. 22 (4) , 1293-1313
Abstract
The web has become a fundamental tool for carrying out many activities spanning from education to work and private life. For this reason, it must be accessible to every user regardless of any form of impairment or disability. Images on the web are a primary means for communicating information, and specific HTML elements were defined to enrich images with textual descriptions, which can be read aloud by screen readers or rendered by braille displays. A relevant problem is that adding a text describing each image published on a website is a demanding task requiring a non-negligible effort for web developers. Several tools based on machine learning have emerged, which can automatically return descriptions for the images. In this work, we evaluate the correctness of their outputs by comparing the generated descriptions with human-defined references. More specifically, we selected 60 images from Wikipedia and their corresponding descriptions as defined by Wikipedia contributors. We then generated the corresponding descriptions employing four state of the art tools (Azure Computer Vision Engine, Amazon Rekognition, Cloudsight, and Auto Alt-Text for Google Chrome) and asked 76 computer science students to blindly evaluate the perceived correctness of the descriptions without being aware of their source. The results show that the descriptions available in Wikipedia are still perceived as the best ones. However, some tools generate good results for specific categories of images, and they can represent proper candidates for the automated and massive addition of image descriptions to websites, helping to increase the accessibility level of the web drastically.Keywords
This publication has 16 references indexed in Scilit:
- Deep Learning for Image-to-Text Generation: A Technical OverviewIEEE Signal Processing Magazine, 2017
- Understanding Blind People's Experiences with Computer-Generated Captions of Social Media Images2017
- Automatic Alt-text2017
- Rich Image Captioning in the Wild2016
- Guiding Novice Web Workers in Making Image Descriptions Using TemplatesACM Transactions on Accessible Computing, 2015
- AutoCaption: Automatic caption generation for personal photos2014
- Microsoft COCO: Common Objects in Context2014
- Benchmarking web accessibility evaluation tools2013
- Automatic Checking of Alternative Texts on Web PagesLecture Notes in Computer Science, 2010
- Experimentation in Software Engineering2000