The recent shift of emphasis to user experience (UX) has rendered it a central focus of product design and evaluation. A multitude of methods for UX design and evaluation exist, but a clear overview of the current state of the available UX evaluation methods is missing. This is partly due to a lack of agreement on the essential characteristics of UX. In this paper, we present the results of our multi-year effort of collecting UX evaluation methods from academia and industry with different approaches such as literature review, workshops, Special Interest Groups sessions and an online survey. We have collected 96 methods and analyzed them, among other criteria, based on the product development phase and the studied period of experience. Our analysis reveals development needs for UX evaluation methods, such as early-stage methods, methods for social and collaborative UX evaluation, establishing practicability and scientific quality, and a deeper understanding of UX.

User experience evaluation methods: current state and development needs

World Wide Web authors must cope in a hypermedia environment analogous to second-generation computing languages, building and managing most hypermedia links using simple anchors and single-step navigation. Following this analogy, sophisticated application environments on the World Wide Web will require third- and fourth-generation hypermedia features. Implementing third- and fourth-generation hypermedia involves designing both high-level hypermedia features and the high-level authoring environments system developers build for authors to specify them. We present a set of high-level hypermedia features including typed nodes and links, link attributes, structure-based query, transclusions, warm and hot links, private and public links, hypermedia access permissions, computed personalized links, external link databases, link update mechanisms, overviews, trails, guided tours, backtracking and history-based navigation. We ground our discussion in the hypermedia research literature, and illustrate each feature both from existing implementations and a running scenario. We also give some direction for implementing these on the World Wide Web and in other information systems.

/pdf/fourth-generation-hypermedia-1zhksyw55s.pdf

Fourth generation hypermedia

In recent years we have seen information visualization technology move from an advanced research topic to mainstream adoption in both commercial and personal use. This move is in part due to many businesses recognizing the need for more effective tools for extracting knowledge from the data warehouses they are gathering. Increased mainstream interest is also a result of more exposure to advanced interfaces in contemporary online media. The adoption of information visualization technologies by lay users --- as opposed to the traditional information visualization audience of scientists and analysts --- has important implications for visualization research, design and development. Since we cannot expect each of these lay users to design their own visualizations, we have to provide them tools that make it easy to create and deploy visualizations of their datasets.

Creation and Collaboration: Engaging New Audiences for Information Visualization

https://www.cs.umd.edu/~ben/papers/Goldberg2011Usability.pdf

Usability and accessibility in consumer health informatics current trends and future challenges.

The paper considers the usability of the World Wide Web in the light of a decade of research into the usability of hypertext and hypermedia systems. The concepts of virtual hierarchies and virtual networks are introduced as a mechanism for alleviating some of the shortcomings inherent in the current implementations of the web, without violating its basic philosophy. It is suggested that virtual hierarchies and virtual networks will assist users to find task-relevant information more easily and quickly and also help web authors to ensure that their pages are targeted at the users who wish to see them.The paper first analyses the published work on hypermedia usability, identifying the assumptions that underlie this research and relating them to the assumptions underlying the web. Some general conclusions are presented about both hypermedia usability principles and their applicability to the web. These results are coupled with problems identified from other sources to produce a requirements list for improving web usability. A possible solution is then presented which utilizes the capabilities of existing distributed information management software to permit web users to create virtual hierarchies and virtual networks. Some ways in which these virtual structures assist searchers to find useful information, and thus help authors to publicize their information more effectively, are described. The explanation is illustrated by examples taken from the GENIE Service, an implementation of some of the ideas. This uses the World Wide Web as a means of allowing global environmental change researchers throughout the world to find data that may be relevant to their research.

Virtual hierarchies and virtual networks

The phenomenal growth of the Internet over the last few years, coupled with the development of various multimedia applications which exploit the Internet presents exciting opportunities for educators. In the context of distance education, the World Wide Web provides a unique challenge as a new delivery mechanism for course material allowing students to take a course (potentially) from anywhere in the world. In this paper, we describe our approach to the development of an Internet-based course designed for distance education. Using this experience, we provide general observations on the opportunities and constraints which the web provides and on the pedagogic issues which arise when using this delivery mechanism.We have found that the process of developing web-based courses is one area which requires careful consideration as technologies and tools for both the authoring and the delivery of courses are evolving so rapidly. We have also found that current tools are severely lacking in a number of important respects?particularly with respect to the design of pedagogically sound courseware.

Experience with developing multimedia courseware for the World Wide Web

User Interface Design and Evaluation (The Morgan Kaufmann Series in Interactive Technologies) (The Morgan Kaufmann Series in Interactive Technologies)

Publisher Summary This chapter discusses the procedure for heuristic evaluation and also provides several other inspection methods that practitioners can use, either individually or with groups, to eliminate usability defects from their products. The primary goal of a heuristic evaluation is to reveal as many usability or design problems as possible at relatively low cost. The evaluation also aims to train members of the product team to recognize potential usability problems so they can be eliminated earlier in the design process. Theoretically, collating and summarizing data from a heuristic inspection is a relatively simple matter of gathering together the forms that the inspectors have used. There are several benefits of conducting heuristic evaluations and inspections: inspections can sometimes be less expensive than the user observation, during an inspection, inspectors more often than not suggest solutions to the usability defects that they identify, and inspecting the user interface (UI) first can help in revealing a large number of defects. Besides heuristic, other techniques such as guideline reviews, cognitive walkthrough (CW), and peer review can also be used to reveal usability defects.

Debbie Stone

Papers

Experience with developing multimedia courseware for the World Wide Web

User Interface Design and Evaluation (The Morgan Kaufmann Series in Interactive Technologies) (The Morgan Kaufmann Series in Interactive Technologies)

Inspections of the User Interface