Context: In recent years, many usability evaluation methods (UEMs) have been employed to evaluate Web applications. However, many of these applications still do not meet most customers' usability expectations and many companies have folded as a result of not considering Web usability issues. No studies currently exist with regard to either the use of usability evaluation methods for the Web or the benefits they bring. Objective: The objective of this paper is to summarize the current knowledge that is available as regards the usability evaluation methods (UEMs) that have been employed to evaluate Web applications over the last 14years. Method: A systematic mapping study was performed to assess the UEMs that have been used by researchers to evaluate Web applications and their relation to the Web development process. Systematic mapping studies are useful for categorizing and summarizing the existing information concerning a research question in an unbiased manner. Results: The results show that around 39% of the papers reviewed reported the use of evaluation methods that had been specifically crafted for the Web. The results also show that the type of method most widely used was that of User Testing. The results identify several research gaps, such as the fact that around 90% of the studies applied evaluations during the implementation phase of the Web application development, which is the most costly phase in which to perform changes. A list of the UEMs that were found is also provided in order to guide novice usability practitioners. Conclusions: From an initial set of 2703 papers, a total of 206 research papers were selected for the mapping study. The results obtained allowed us to reach conclusions concerning the state-of-the-art of UEMs for evaluating Web applications. This allowed us to identify several research gaps, which subsequently provided us with a framework in which new research activities can be more appropriately positioned, and from which useful information for novice usability practitioners can be extracted.

Usability evaluation methods for the web: A systematic mapping study

A VAXcluster is a highly available and extensible configuration of VAX computers that operate as a single system. To achieve performance in a multicomputer environment, a new communications architecture, communications hardware, and distributed software were jointly designed. The software is a distributed version of the VAX/VMS operating system that uses a distributed lock manager to synchronize access to shared resources. The communications hardware includes a 70 megabit per second message-oriented interconnect and an interconnect port that performs communications tasks traditionally handled by software. Performance measurements show this structure to be highly efficient, for example, capable of sending and receiving 3000 messages per second on a VAX-11/780.

/pdf/vaxcluster-a-closely-coupled-distributed-system-4kestcl07x.pdf

VAXcluster: a closely-coupled distributed system

When implementing a system specified as a number of layers of abstraction, it is tempting to implement each layer as a process. However, this requires that communication between layers be via asynchnonous inter-process messages. Our experience, especially with implementing network protocols, suggests that asynchronous communication between layers leads to serious performance problems. In place of this structure we propose an implementation methodology which permits synchronous (procedure call) between layers, both when a higher layer invokes a lower layer and in the reverse direction, from lower layer upward. This paper discusses the motivation for this methodology, as well as the pitfalls that accompany it. 1 I n t r o d u c t i o n This paper is concerned with a methodology for program structure, a methodology suitable for operating system programs, especially programs dealing with communications and networks. This methodology arose out of our earlier research in the implementation of network protocols, in which recurring performance problems with protocol software led us to the conclusion that many operating systems failed to provide the correct runtime support for highly interactive parallel software packages such as protocols. This paper describes and motivates this methodology, and discusses the operating system, Swift, which we built to explore it. The methodology described in this paper is relevant to programs which have been modularized according to the This research was supported by the Advanced Research Projects Agency of the Department of Defense and was monitored by the Office of Naval Research under contrac~ N00014-75-C-0681 and N00014-83-K-0125. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission. © 1985 A C M 0 8 9 7 9 1 1 7 4 1 12/85-0171 $ 0 0 . 7 5 principle of layering. Traditionally, a layer is thought of as providing services to the layer above, or the client layer. The client uses some mechanism for invoking the layer, perhaps a subroutine call. The layer performs the service for the client and then returns. In other words, service invocation occurs from the top down. As we will discuss below, there are organizational and modularity reasons why this downward flow of control is appealing in a layered system. However, the natural flow of control is not always downward. In a network driven environment, for example, most of the actions are initiated, not by the client from above, but by the network from below. The natural flow of control is thus upward, not downward. Especially where such an upward flow of control crosses a protection boundary, most systems do not permit this flow to be implemented as a procedure call. Instead, some more cumbersome and asynchronous mechanism must be used, such as an interprocess communications signal. Substantial inefficiencies and complexities can result from asynchronous upward flows. In our methodology, the system is organized so that the programmer has the choice as to whether an upward flow is implemented by procedure calls or asynchronous signals. We call this feature upcalls. We chose the word upcall to distinguish from the structured view of service invocation, organized around downward flow. An upcall need not go precisely upward. Our goal is that procedure flow should map on to the natural flow of control in the program, whether that is up, down, or sideways. With procedural invocation available in this way, there is thus no reason to use processes and interprocees communication unless there is an intrinsic source of asynchrony. The other half of this programming methodology is an approach towards structuring a layer. In many systems, a layer is implemented as a task or process. In our methodology a layer is organized as a collection of subroutines which live in a number of tasks, each subroutine callable as appropriate from above or below. Subroutines in different task that make up a layer constitute, in our terminology, a multi-task module. A multitask module also contains a collection of state variables, which are accessible, using shared memory, from the different tasks which execute the subroutines of the layer. The purpose of this paper, therefore, is to describe the programming methodology which we call upcalls and multi-task"

/pdf/the-structuring-of-systems-using-upcalls-m0el4bdsmc.pdf

The structuring of systems using upcalls

This paper describes an empirical study of the problems encountered by 32 blind users on the Web. Task-based user evaluations were undertaken on 16 websites, yielding 1383 instances of user problems. The results showed that only 50.4% of the problems encountered by users were covered by Success Criteria in the Web Content Accessibility Guidelines 2.0 (WCAG 2.0). For user problems that were covered by WCAG 2.0, 16.7% of websites implemented techniques recommended in WCAG 2.0 but the techniques did not solve the problems. These results show that few developers are implementing the current version of WCAG, and even when the guidelines are implemented on websites there is little indication that people with disabilities will encounter fewer problems. The paper closes by discussing the implications of this study for future research and practice. In particular, it discusses the need to move away from a problem-based approach towards a design principle approach for web accessibility.

http://www-course.cs.york.ac.uk/advt/classes/class03/materials/Power,%20Freire,%20Petrie,%20Swallow%202012.pdf

Guidelines are only half of the story: accessibility problems encountered by blind users on the web

Data sharing is fundamental to computer-supported cooperative work: People share information through explicit communication channels and through their coordinated use of shared databases. This paper examines the data management requirements of group work applications on the basis of experience with three prototype systems and on observations from the literature. Database and object management technologies that support these requirements are briefly surveyed, and unresolved issues in the particular areas of access control and concurrency control are identified for future research.

Data sharing in group work

Web Content Accessibility Guidelines (WCAG) 2.0 covers a wide range of recommendations for making Web content more accessible. Following these guidelines will make content accessible to a wider range of people with disabili ties, including blindness and low vision, deafness and hearing loss, learning disabilities, c ognitive limitations, limited movement,

Web Content Accessibility Guidelines (WCAG) 2.0

Since the Web Content Accessibility Guidelines 1.0 (WCAG) became a W3C recommendation in May 1999, the Web has changed dramatically. This paper describes some of the major issues encountered because of these changes and the approaches developed to address them in WCAG 2.0.

/pdf/wcag-2-0-a-web-accessibility-standard-for-the-evolving-web-3hek26j6p4.pdf

WCAG 2.0: a web accessibility standard for the evolving web

Speech activity detection (or endpointing) is an important processing step for applications such as speech recognition, language identification and speaker diarization. Both audio- and vision-based approaches have been used for this task in various settings, often tailored toward end applications. However, much of the prior work reports results in synthetic settings, on task-specific datasets, or on datasets that are not openly available. This makes it difficult to compare approaches and understand their strengths and weaknesses. In this paper, we describe a new dataset which we will release publicly containing densely labeled speech activity in YouTube videos, with the goal of creating a shared, available dataset for this task. The labels in the dataset annotate three different speech activity conditions: clean speech, speech co-occurring with music, and speech co-occurring with noise, which enable analysis of model performance in more challenging conditions based on the presence of overlapping noise. We report benchmark performance numbers on AVA-Speech using off-the-shelf, state-of-the-art audio and vision models that serve as a baseline to facilitate future research.

AVA-Speech: A Densely Labeled Dataset of Speech Activity in Movies

This paper addresses the problem of supporting accessibility in applications that run in multiple operating environments. It analyzes the commonalities of existing platform-specific Accessibility APIs, and defines a platform-independent accessibility API, the Accessible DOM.The Accessible DOM encompasses the features of existing APIs and overcomes the limitations of existing APIs to express dynamic, complex document contents.The Accessible DOM can be used to support existing and future platform-specific accessibility APIs. It will also allow the development of platform-independent accessibility clients.

/pdf/platform-independent-accessibility-api-accessible-document-5b8yxmq50k.pdf

Loretta Guarino Reid

Papers

Web Content Accessibility Guidelines (WCAG) 2.0

WCAG 2.0: a web accessibility standard for the evolving web

AVA-Speech: A Densely Labeled Dataset of Speech Activity in Movies

Platform-independent accessibility API: accessible document object model

AVA-Speech: A Densely Labeled Dataset of Speech Activity in Movies