Electronic Mail has become an essential tool of communication and collaboration for sighted, visually impaired, and blind people However, due to inconsistent interface design, lack of logical order of navigational items, the diverse set of screen sizes and orientations, complicated text-entry layouts, and inadequate mapping of haptic feedback, the existing email-related activities on smartphone contribute to several issues In addition, blind people also confront problems in precisely accessing the non-visual items on touchscreen interfaces to perform common email-related activities like sending, receiving, organizing, deleting, filtering, searching, and managing spam emails Due to these problems, blind people are facing difficulties not only in operating a smartphone but also in performing several email-related activities Furthermore, spam and junk emails cause frustration and contribute to cognitive overload We proposed TetraMail, a usable blind-friendly email client to overcome the challenges pertaining to the accessibility and usability of email-related activities on a smartphone The proposed email client is evaluated through an empirical study of 38 blind participants by performing 14 email activities The results of this prototype implementation show an improved user experience, accuracy in task completion, and better control over touchscreen interfaces in performing basic activities of managing emails The results demonstrate that TetraMail is an accessibility-inclusive email client enabling blind people to have a better user interaction experience and minimal cognitive overload in managing emails

TetraMail: a usable email client for blind people

Access to ubiquitous devices such as smartphones, smartwatches, and wearable assistive bands is an emerging trend. Smartphones and smartwatches have become an essential tool for visually impaired and blind people in performing daily life activities, promoting independence, productivity, and self-reliance. The accessibility services such as talkback and voice assistants facilitate blind people in using smartphones to perform several activities such as placing calls, sharing pictures, reading books, and sending messages, etc. However, existing accessibility services face several problems, including late learning, accessing, and selecting non-visual items on the screen, finding objects of interest in a complex layout, multimodal interaction, and operating multiple connected devices. One solution to these problems is to design a blind-friendly interface to simplify and distribute User Interface Artefacts (UIAs), including labels, buttons, layouts, and panels across multiple devices. The framework tailor’s UI components into simplified blind-friendly user interfaces on smartphones and smartwatches, resulting in improving the user experience. Forty-nine blind people have participated in this empirical study revealing a considerable improvement in the user experience, task completion accuracy, and productivity.

A mechanism for blind-friendly user interface adaptation of mobile apps: a case study for improving the user experience of the blind people

Re-finding electronic documents from personal computers is a frequent demand For a simple re-finding task, people can use many methods to retrieve the target document, such as navigating directly to its folder, searching with desktop search engines, or checking the Recent Files List When encountering difficult re-finding tasks, people usually cannot rely on attributes exploited by conventional re-finding methods, the re-finding would fail We propose a new method to support difficult re-finding tasks, by collecting extra new attributes of a document, such as number of pages, number of images, reading frequency, and coverage percentage If the document is quested later, the collected attributes and experiences are used to filter out it A question and answer wizard interface is utilized to alleviate cognitive burden when recollecting Finally, we develop a PDF document re-finding system to evaluate the effectiveness of the method

A PDF document re-finding system with a Q&A wizard interface

Mobility and continuous connection entail the emergence of heterogeneous devices with multiple forms of interaction. However, it is challenging for developers and corporations to keep up with the devices and provide applications adapted to them. Besides, better user experiences attuned to users’ needs and desires are increasingly in demand. User interfaces play a major role because they must be distributed through different devices and offer a customized experience for each user–device combination. We take advantage of the component-based applications easiness to build customized interfaces that can give optimal solutions to fulfill the requirements for adapting themselves to cross-device applications with multiple forms of interaction. User interaction on mashup interfaces can generate a great deal of data, which can be analyzed for improving the interaction and usefulness of the applications. In our paper, we have created a microservice-based architecture that generates datasets which contain the user behavior for further analysis. Therefore, the user experience and usability in distributed user interfaces may be improved through prediction models generated from the data. Each microservice autonomously fetches its own data and performs consistently so that it can transform datasets optimally by using feature engineering techniques. Thus, data analysis and algorithms can create accurate yet simple prediction models that provide useful knowledge to enhance the user experience. A REST API web service is added to each microservice to facilitate their communication with other microservices and/or third-party clients. The entire microservice architecture, including feature engineering and RESTful API web services for each microservice, offers an infrastructure to handle and process data interaction of cross-devices applications with multiple forms of interaction. This approach has been deployed in a real mashup application where new datasets have been created, processed and validated.

A microservice-based architecture for enhancing the user experience in cross-device distributed mashup UIs with multiple forms of interaction

Over the last decade, we have seen an increasing number of solutions for distributed user interfaces (DUIs). This paper provides a detailed overview of existing DUI approaches and classify the different solutions based on the granularity of the distributed UI components, the supported interaction space as well as their support for the distribution of state. After the analysis of existing solutions, we discuss a DUI scenario and derive a number of requirements for end-user DUI development. We propose an approach where users can author their customised user interfaces based on a hypermedia metamodel and the concept of active components. We further discuss possibilities for the configuration and sharing of customised distributed user interfaces by end users where the focus is on an authoring rather than a programming approach.

https://link.springer.com/content/pdf/10.1007%2Fs10209-017-0601-5.pdf

Towards end-user development of distributed user interfaces

Over the last decade, we have witnessed an emergence of Personal Information Management (PIM) solutions. Despite the fact that paper documents still form a significant part of our daily working activities, existing PIM systems usually support the organisation and re-finding of digital documents only. While physical document tracking solutions such as RFID- or computer vision-based systems are recently gaining some attention, they usually focus on the paper document tracking and offer limited support for re-finding activities. We present PimVis, a solution for exploring and re-finding digital and paper documents in so-called cross-media information spaces. The PimVis user interface enables a unified organisation of digital and paper documents through the creation of bidirectional links between the digital and physical information space. The presented personal cross-media information management solution further supports the extension with alternative document tracking techniques as well as augmented reality solutions. A formative PimVis evaluation revealed the high potential of fully integrated cross-media PIM solutions.

PimVis: Exploring and Re-finding Documents in Cross-Media Information Spaces

Over the last decade we have seen various research on distributed user interfaces (DUIs). We provide an overview of existing DUI approaches and classify the different solutions based on the granularity of the distributed UI components, location constraints as well as their support for the distribution of state. We propose an approach for user-defined cross-device interaction where users can author their customised user interfaces based on a hypermedia metamodel and the concept of active components. Furthermore, we discuss the configuration and sharing of customised distributed user interfaces by end users where the focus is on an authoring rather than programming approach.

Towards User-Defined Cross-Device Interaction

Over the last decade, research on adaptive and distributed user interfaces (DUIs) has increased. We also witness a growing number of Internet of Things (IoT) devices, allowing digital user interfaces (UIs) to communicate with physical objects and vice versa through so-called hybrid user interfaces. There exist various solutions to manage adaptive, distributed or hybrid UIs. However, none of them covers all three aspects and users have to deal with multiple applications and configurations when developing adaptive distributed hybrid user interfaces. We introduce the eSPACE reference framework and conceptual model unifying the domains of adaptive, distributed and hybrid interfaces. While our reference framework has been inspired by the CAMELEON reference framework, the conceptual model is based on the Resource-Selector-Link (RSL) hypermedia metamodel. We propose an approach for adaptive distributed hybrid user interfaces where users can author their user interfaces based on the different levels of abstraction introduced by our reference framework. We further present a use case illustrating the extensibility, flexibility and reusability offered by our unified approach and discuss potential future work.

A Unifying Reference Framework and Model for Adaptive Distributed Hybrid User Interfaces

Knowledge bases are used to store and centralize information on certain topics in a domain. Using a well-structured and machine-readable format is a prerequisite for any AI-based processing or reasoning. The use of semantic technologies (e.g., RDF, OWL) has the advantage that it allows to define the semantics of the information and supports advanced querying. However, using such technologies is a challenging task for subject matter experts from a domain such as life science who are, in general, not trained for this. This means that they need to rely on semantic technology experts to create their knowledge bases. However, these experts are usually IT-experts and they are, in turn, not trained in the subject matter, while knowledge of the domain is essential for the construction of a high-quality knowledge base. In this paper, we present an end-user development (EUD) tool that supports subject matter experts in the construction of ontology–based knowledge bases. The tool is using the jigsaw metaphor for hiding the technicalities of the semantic technology, as well as to guide the users in the process of creating a knowledge base. The approach and the tool is demonstrated for building a knowledge base in the toxicology domain. The tool has been evaluated by means of a preliminary user study with nine subject matter experts from this domain. All participants state that with a little practice they could become productive with our tool and actually use it to represent and manage their knowledge. The results of the evaluation resulted in valuable suggestions for improving the tool and highlighted the importance of well adapting the terminology to the target audience.

Audrey Sanctorum

Papers

PimVis: Exploring and Re-finding Documents in Cross-Media Information Spaces

Towards User-Defined Cross-Device Interaction

A Unifying Reference Framework and Model for Adaptive Distributed Hybrid User Interfaces

Towards end-user development of distributed user interfaces

A Jigsaw-Based End-User Tool for the Development of Ontology-Based Knowledge Bases