Remote Usability Evaluation Methods and Tools: A Survey
01 Jan 2007-
TL;DR: The purpose of this paper is to categorize and evaluate numerous approaches and techniques proposed to allow remote usability evaluation of software applications, and to discuss relevant issues such as user data collection and analysis.
Abstract: Effective and efficient usability evaluation methodologies are required in order to develop and maintain software applications that meet user requirements and expectations. This is particularly relevant to applications that change frequently during their lifecycle . Remote usability evaluation provides such a solution. Numerous approaches and techniques have been proposed to allow remote usability evaluation of software applications, and the purpose of this paper is to categorize and evaluate these approaches. We also discuss relevant issues such as user data collection and analysis. After surveying the proposed approaches and identifying pros and cons, we conclude with promising directions for future research in this area. Keywords: Human Computer Interaction, Usability Evaluation, Remote Usability Evaluation Methods, Remote Usability Evaluation Tools 1. Introduction The rapidly expanding research in software usability evaluation is based on the premise that usable software products increase user satisfaction, effectiveness and efficiency, improve product quality while simultaneously reduce maintenance and support costs [Nielsen (1993), Dix et Al. (1998)]. Usability refers to whether a system can be used with effectiveness, efficiency, and satisfaction with which specified users achieve specified goals in a particular context of use [ISO 9241-11]. Methodologies for software development need to take advantage of user centred design approaches which give extensive attention to the needs and limitations of the users at each stage of the design process [Vredenburg et. Al.(2002)]. Furthermore, a relatively recent trend in web applications which is related to the
Citations
More filters
26 Sep 2008
TL;DR: The development of e-RUE is a cheap possible solution for usability evaluation and can be run from different geographical locations between the respondents, experts and researchers.
Abstract: Interface is one of the most important aspects as it is the medium that users used to communicate and interact with a system. A system has more chances to be successful if its design makes emphasis on usability. Usability and interface design are fields that are becoming more and more important in a system design. Both fields consider the interactions between the user and a system. Usability evaluation is an important phase in a system development cycle. There are various methods that can be used to run a usability test. Usability lab has been identified as the best solutions for usability evaluation. However, usability labs are costly to set up plus to the travel and lodging expenses. As Internet technologies evolve, there are possibilities to run usability evaluation remotely. Usability test can be run from different geographical locations between the respondents, experts and researchers. This paper discusses remote usability evaluation via e-RUE. The development of e-RUE is a cheap possible solution for usability evaluation.
12 citations
TL;DR: An approach to avoid any further download or installation is suggested to offer a collaborative and extensible web environment which will cover a series of domains highly demanded by different kinds of working groups, in which it is crucial to have tools which facilitate the exchange of information and the collaboration among their members.
11 citations
25 Oct 2011
TL;DR: The proposed model allows the definition of tasks using a simple and intuitive approach, which can be applied to large and dynamic Web applications, and indicates that the approach is effective towards identifying usability problems in Web applications.
Abstract: The availability of tools and facilities for the creation and publishing of information on the Web has allowed people without a Web development background to build and deploy applications on the Web. In this scenario, usability evaluation tasks are rarely adopted in the application's development process. Therefore, there is an increasing interest on studies related to the automatic or semi-automatic remote evaluation of usability. This paper presents USABILICS, a system targeted for the semi-automatic remote evaluation of usability based on an interface model. The proposed model allows the definition of tasks using a simple and intuitive approach, which can be applied to large and dynamic Web applications. USABILICS analyses the execution of tasks by calculating the similarity among sequence of events produced by users and those previously captured by the developer. The results produced by USABILICS, when compared to laboratory-based tests, indicate that our approach is effective towards identifying usability problems in Web applications.
6 citations
Dissertation•
01 Jan 2013
TL;DR: In this article, the authors used the dimension Uncertainty Avoidance to define culture, which refers to the extent to which people feel threatened by uncertainty and ambiguity and try to avoid these situations.
Abstract: The main focus of this study is to look into the relationship between entrepreneurial
processes and culture. The entrepreneurial process is a part of entrepreneurship and
entrepreneurship plays a very core and fundamental role in any economy. Different facets of
entrepreneurship have been proven to be influenced by national culture, but research on
the influence of culture on entrepreneurial processes are lacking.
A relatively new process theory by Sarasvathy (2001) caused quite a stir. She discovered that
there are different types of entrepreneurial processes used and developed a model where
she put her findings against each other at the opposite level. She defined these two
opposites as causation and effectuation. This model is well suited for both theoretical
examination and for practical application which makes it a relevant model for this study.
Hofstede (2001) is one of the pioneers in developing a theory about culture at the country
level. Hofstede proposed that cultures can be measured on five dimensions. This study uses
the dimension Uncertainty Avoidance to define culture. Uncertainty Avoidance refers to the
extent to which people feel threatened by uncertainty and ambiguity and try to avoid these
situations. Uncertainty Avoidance on country level is to what extent countries try to cope
with uncertainty through the domains of technology, law and religion.
This study is part of the EPICC project where multiple students have tried to find a relation
between culture and entrepreneurial processes. The dimension Uncertainty Avoidance has
been used before, with mixed conclusions. This motivated to make use of the theory of
Tightness – Looseness, which has similarities with the concept of Uncertainty Avoidance, to
see which concept is a better predictor for entrepreneurial processes. The differences
between Tight and Loose cultures are that in tight cultures there are many rules, norms, and
standards for correct behavior and in loose cultures it is the opposite: there are few rules,
norms, or standards.
Data has been gathered under 20 Belgium student entrepreneurs. The 20 student
entrepreneurs have worked on a case in which they had to put themselves in the role of an
entrepreneur who starts up a fictional venture. Therefore the think aloud method was used.
This method allows the subjects to think aloud during the process of solving the case. The
interviews have been recorded, transcribed and coded. To find out if there is a relation
between culture and entrepreneurial processes, data of a different country is used to
compare the data obtained from Belgium. Malaysia is used because it is very different to
Belgium at the dimension Uncertainty Avoidance and Tightness – Looseness.
Analysis of the results shows little support that culture influences entrepreneurial processes.
Uncertainty Avoidance predicted only one out of 3 hypotheses, which confirms earlier
conclusions that it might not be a good predictor for causation and effectuation. However,
also Tightness turned out not to be a good predictor for entrepreneurial processes. A
tentative conclusion can be given that culture influences the entrepreneurial process used
by entrepreneurs on one element: Entrepreneurs from a high uncertainty avoiding country
rely more on expected returns than entrepreneurs from a low uncertainty avoiding country.
4 citations
Cites methods from "Remote Usability Evaluation Methods..."
...Skype is a classified and widely used remote usability tool (Christos et al., 2007)....
[...]
References
More filters
Book•
01 Jan 1993
TL;DR: This guide to the methods of usability engineering provides cost-effective methods that will help developers improve their user interfaces immediately and shows you how to avoid the four most frequently listed reasons for delay in software projects.
Abstract: From the Publisher:
Written by the author of the best-selling HyperText & HyperMedia, this book provides an excellent guide to the methods of usability engineering. Special features: emphasizes cost-effective methods that will help developers improve their user interfaces immediately, shows you how to avoid the four most frequently listed reasons for delay in software projects, provides step-by-step information about which methods to use at various stages during the development life cycle, and offers information on the unique issues relating to informational usability. You do not need to have previous knowledge of usability to implement the methods provided, yet all of the latest research is covered.
11,929 citations
"Remote Usability Evaluation Methods..." refers background in this paper
...The rapidly expanding research in software usability evaluation is based on the premise that usable software products increase user satisfaction, effectiveness and efficiency, improve product quality while simultaneously reduce maintenance and support costs [Nielsen (1993), Dix et Al....
[...]
Book•
01 Jan 1983
TL;DR: The GOMS Model of Manuscript Editing as mentioned in this paper has been used in many applications, e.g., for text selection and text editing in computer science, and for circuit design.
Abstract: Contents: Preface. An Applied Information-Processing Psychology. Part I: Science Base. The Human Information-Processor. Part II: Text-Editing. System and User Variability. An Exercise in Task Analysis. The GOMS Model of Manuscript Editing. Extensions of the GOMS Analysis. Models of Devices for Text Selection. Part III: Engineering Models. The Keystroke-Level Model. The Unit-Task Level of Analysis. Part IV: Extensions and Generalizations. An Exploration into Circuit Design. Cognitive Skill. Applying Psychology to Design Reprise.
4,999 citations
TL;DR: The survey analyzes existing techniques, identifies which aspects of usability evaluation automation are likely to be of use in future research, and suggests new ways to expand existing approaches to better support usability evaluation.
Abstract: Usability evaluation is an increasingly important part of the user interface design process. However, usability evaluation can be expensive in terms of time and human resources, and automation is therefore a promising way to augment existing approaches. This article presents an extensive survey of usability evaluation methods, organized according to a new taxonomy that emphasizes the role of automation. The survey analyzes existing techniques, identifies which aspects of usability evaluation automation are likely to be of use in future research, and suggests new ways to expand existing approaches to better support usability evaluation.
906 citations
20 Apr 2002
TL;DR: Results show that cost-benefit tradeoffs are a key consideration in the adoption of UCD methods and that there is a major discrepancy between the commonly cited measures and the actually applied ones.
Abstract: This paper reports the results of a recent survey of user-centered design (UCD) practitioners. The survey involved over a hundred respondents who were CHI'2000 attendees or current UPA members. The paper identifies the most widely used methods and processes, the key factors that predict success, and the critical tradeoffs practitioners must make in applying UCD methods and processes. Results show that cost-benefit tradeoffs are a key consideration in the adoption of UCD methods. Measures of UCD effectiveness are lacking and rarely applied. There is also a major discrepancy between the commonly cited measures and the actually applied ones. These results have implications for the introduction, deployment, and execution of UCD projects
672 citations
13 Apr 1996
TL;DR: Traditional user interface evaluation usually is conducted in a laboratory where users are observed directly by evaluators, but methods for remote usability evaluation wherein the evaluator is separated in space and/or time from the user are considered.
Abstract: Traditional user interface evaluation usually is conducted in a laboratory where users are observed directly by evaluators. However, the remote and distributed location of users on the network precludes the opportunity for direct observation in usability testing. Further, the network itself and the remote work setting have become intrinsic parts of usage patterns, difficult to reproduce in a laboratory setting, and developers often have limited access to representative users for usability testing in the laboratory. In all of these cases, the cost of transporting users or developers to remote locations can be prohibitive. These barriers have led us to consider methods for remote usability evaluation wherein the evaluator, performing observation and analysis, is separated in space and/or time from the user. The network itself serves as a bridge to take interface evaluation to a broad range of networked users, in their natural work settings. Several types of remote evaluation are defined and described in terms of their advantages and disadvantages to usability testing. The initial results of two case studies show potential for remote evaluation. Remote evaluation using video teleconferencing uses the network as a mechanism to transport video data in real time, so that the observer can evaluate user interfaces in remote locations as they are being used. Semi-instrumented remote evaluation is based on critical incident gathering by the user within the normal work context. Additionally, both methods can take advantage of automat ing data collection through questionnaires and instrumented applications.
217 citations