Research advancements allow computational systems to automatically caption social media images. Often, these captions are evaluated with sighted humans using the image as a reference. Here, we explore how blind and visually impaired people experience these captions in two studies about social media images. Using a contextual inquiry approach (n=6 blind/visually impaired), we found that blind people place a lot of trust in automatically generated captions, filling in details to resolve differences between an image's context and an incongruent caption. We built on this in-person study with a second, larger online experiment (n=100 blind/visually impaired) to investigate the role of phrasing in encouraging trust or skepticism in captions. We found that captions emphasizing the probability of error, rather than correctness, encouraged people to attribute incongruence to an incorrect caption, rather than missing details. Where existing research has focused on encouraging trust in intelligent systems, we conclude by challenging this assumption and consider the benefits of encouraging appropriate skepticism.

/pdf/understanding-blind-people-s-experiences-with-computer-5eg7x1sqjy.pdf

Understanding Blind People's Experiences with Computer-Generated Captions of Social Media Images

Tactile relief maps are used by visually impaired people to acquire mental representation of space, but they retain important limitations (limited amount of information, braille text, etc.). Interactive maps may overcome these limitations. However, usability of these two types of maps had never been compared. It is then unknown whether interactive maps are equivalent or even better solutions than traditional raised-line maps. This study presents a comparison of usability of a classical raised-line map vs. an interactive map composed by a multi-touch screen, a raised-line overlay and audio output. Both maps were tested by 24 blind participants. We measured usability as efficiency, effectiveness and satisfaction. Our results show that replacing braille with simple audio-tactile interaction significantly improved efficiency and user satisfaction. Effectiveness was not related to the map type but depended on users' characteristics as well as the category of assessed spatial knowledge. Long-term evaluation of acquired spatial information revealed that maps, whether interactive or not, are useful to build robust survey-type mental representations in blind users. Altogether, these results are encouraging as they show that interactive maps are a good solution for improving map exploration and cognitive mapping in visually impaired people.

/pdf/interactivity-improves-usability-of-geographic-maps-for-42lkgm3evp.pdf

Interactivity Improves Usability of Geographic Maps for Visually Impaired People

Tactile maps are commonly used to give visually impaired users access to geographical representations. Although those relief maps are efficient tools for acquisition of spatial knowledge, they present several limitations and issues such as the need to read braille. Several research projects have been led during the past three decades in order to improve access to maps using interactive technologies. In this chapter, we present an exhaustive review of interactive map prototypes. We classified existing interactive maps into two categories: Digital Interactive Maps (DIMs) that are displayed on a flat surface such as a screen; and Hybrid Interactive Maps (HIMs) that include both a digital and a physical representation. In each family, we identified several subcategories depending on the technology being used. We compared the categories and subcategories according to cost, availability, and technological limitations, but also in terms of content, comprehension, and interactivity. Then we reviewed a number of studies showing that those maps can support spatial learning for visually impaired users. Finally, we identified new technologies and methods that could improve the accessibility of graphics for visually impaired users in the future.

Accessible Interactive Maps for Visually Impaired Users

We report on the design process leading to the creation of MapSense, a multi-sensory interactive map for visually impaired children. We conducted a formative study in a specialized institute to understand children's educational needs, their context of care and their preferences regarding interactive technologies. The findings (1) outline the needs for tools and methods to help children to acquire spatial skills and (2) provide four design guidelines for educational assistive technologies. Based on these findings and an iterative process, we designed and deployed MapSense in the institute during two days. It enables collaborations between children with a broad range of impairments, proposes reflective and ludic scenarios and allows caretakers to customize it as they wish. A field experiment reveals that both children and caretakers considered the system successful and empowering.

/pdf/mapsense-multi-sensory-interactive-maps-for-children-living-12sonj81ag.pdf

MapSense: Multi-Sensory Interactive Maps for Children Living with Visual Impairments

Alt text (short for "alternative text") is descriptive text associated with an image in HTML and other document formats. Screen reader technologies speak the alt text aloud to people who are visually impaired. Introduced with HTML 2.0 in 1995, the alt attribute has not evolved despite significant changes in technology over the past two decades. In light of the expanding volume, purpose, and importance of digital imagery, we reflect on how alt text could be supplemented to offer a richer experience of visual content to screen reader users. Our contributions include articulating the design space of representations of visual content for screen reader users, prototypes illustrating several points within this design space, and evaluations of several of these new image representations with people who are blind. We close by discussing the implications of our taxonomy, prototypes, and user study findings.

/pdf/rich-representations-of-visual-content-for-screen-reader-16pq3yqcpa.pdf

Rich Representations of Visual Content for Screen Reader Users

Traditional tactile cartography is complicated by problems associated with braille labeling and feature annotation. Audio-tactile display techniques can address many of these issues by associating spoken information and sounds with specific map elements. This article introduces Talking TMAP – a collaborative effort between The Smith-Kettlewell Eye Research Institute and Touch Graphics, Inc. Talking TMAP combines existing tools such as the World Wide Web, geographic information systems, braille embossers and touch tablet technology in new ways to produce a system capable of creating detailed and accurate audio-tactile street maps of any neighborhood. The article describes software design, user interface and plans for future implementation.

http://www.touchgraphics.com/publications/BJVI-Final.pdf

Talking TMAP: Automated generation of audio-tactile maps using Smith-Kettlewell's TMAP software:

This article compares two methods of employing novice Web workers to author descriptions of science, technology, engineering, and mathematics images to make them accessible to individuals with visual and print-reading disabilities. The goal is to identify methods of creating image descriptions that are inexpensive, effective, and follow established accessibility guidelines. The first method explicitly presented the guidelines to the worker, then the worker constructed the image description in an empty text box and table. The second method queried the worker for image information and then used responses to construct a template-based description according to established guidelines. The descriptions generated through queried image description (QID) were more likely to include information on the image category, title, caption, and units. They were also more similar to one another, based on Jaccard distances of q-grams, indicating that their word usage and structure were more standardized. Last, the workers preferred describing images using QID and found the task easier. Therefore, explicit instruction on image-description guidelines is not sufficient to produce quality image descriptions when using novice Web workers. Instead, it is better to provide information about images, then generate descriptions from responses using templates.

https://dl.acm.org/doi/pdf/10.1145/2764916

Steven Landau

Papers

Talking TMAP: Automated generation of audio-tactile maps using Smith-Kettlewell's TMAP software:

Guiding Novice Web Workers in Making Image Descriptions Using Templates