Enhancing Learning Management Systems Utility for Blind Students: A Task-Oriented, User-Centered, Multi-Method Evaluation Technique.
01 Jan 2013-Vol. 12, Iss: 1, pp 1-32
TL;DR: It is demonstrated how TUME can be used to identify the unique problems and challenges of specific user types in using Web-based applications and suggests po-tential solutions.
Abstract: This paper presents a novel task-oriented, user-centered, multi-method evaluation (TUME) tech-nique and shows how it is useful in providing a more complete, practical and solution-oriented assessment of the accessibility and usability of Learning Management Systems (LMS) for blind and visually impaired (BVI) students. Novel components of TUME include a purposeful integra-tion of a multi-theoretic foundation and multiple methods to accurately identify users’ accessibil-ity and usability problems in Web interaction and identify design problems and solutions to en-sure technical feasibility of recommendations. The problems identified by TUME remain hidden from extant evaluation methods - therefore, these problems remain in Web-based applications. As a result, evaluation of Web-based applications remains confounded by users’ Web interaction challenges; their utility for specific user types remains unclear. Without appropriate evaluation of users’ problems and challenges in using Web-based applications, we cannot begin to solve these problems and challenges. This paper demonstrates how TUME can be used to identify the unique problems and challenges of specific user types in using Web-based applications and suggests po-tential solutions. The outcome is an accurate understanding of specific design elements that pre-sent roadblocks and challenges for the user in interacting with the Web-based application and feasible design modifications to potentially improve the utility of these applications for specific user types.
Citations
More filters
TL;DR: The findings of this study show that the experimental group encountered fewer number of help-seeking situations than the control group when interacting with the experimental and baseline versions of a DL.
Abstract: Blind and visually impaired (BVI) users experience vulnerabilities in digital library (DL) environments largely due to limitations in DL design that prevent them from effectively interacting with DL content and features. Existing research has not adequately examined how BVI users interact with DLs, nor the typical problems encountered during interactions. This is the first study conducted to test whether implementing help features corresponding to BVI users’ needs can reduce five critical help-seeking situations they typically encounter, with the goal to further enhance usability of DLs. Multiple data collection methods including pre-questionnaires, think-aloud protocols, transaction logs, and pre and post search interviews, were employed in an experimental design. Forty subjects were divided into two groups with similar demographic data based on data generated from pre-questionnaires. The findings of this study show that the experimental group encountered fewer number of help-seeking situations than the control group when interacting with the experimental and baseline versions of a DL. Moreover, the experimental group outperformed the control group on perceived usefulness of the DL features, ease of use of the DL, and DL satisfaction. This study provides theoretical and practical contributions to the field of library and information science. Theoretically, this study frames vulnerabilities of BVI users within the social model of disability in which improper DL design impairs their ability to effectively access and use DLs. Practically, this study takes into account BVI users’ critical help-seeking situations and further translates these into the design of help features to improve the usability of DLs.
25 citations
Cites background from "Enhancing Learning Management Syste..."
...Prior research (Babu & Singh, 2013; Clark, 2006; Di Blas et al., 2004; Leuthold, Bargas-Avila & Opwis, 2008) has consistently shown that while these supports are necessary for technical accessibility, they do not fully account for the differing abilities, needs, and challenges of BVI users in…...
[...]
TL;DR: This paper raises awareness of design choices that can unintentionally bar blind information seekers from DL access, and further suggests solutions to reduce these design problems for blind users.
Abstract: Purpose
The purpose of this study is to explore design issues hampering the accessibility of digital libraries (DLs) for first-time blind users.
Design/methodology/approach
A combination of questionnaire, pre-interview, think-aloud and post-interview methods was used to collect data on non-visual interaction experiences with American Memory Digital Collection (AMDC) from 15 blind participants. Qualitative analysis via open coding revealed recurring themes on design problems and consequent difficulties for blind users in accessing DLs.
Findings
It was found that AMDC is not blind-friendly. Five categories of design problems were identified. Participants faced difficulty perceiving, operating and understanding content and controls needed for information retrieval.
Research limitations/implications
This paper does not offer a comprehensive set of design issues prevalent across DL design models, instead it focuses on design problems observed in a publicly available DL.
Practical implications
This paper raises awareness of design choices that can unintentionally bar blind information seekers from DL access, and further suggests solutions to reduce these design problems for blind users.
Originality/value
The paper’s originality is its identification of unique design problems that prevent blind users from effectively interacting with DLs.
13 citations
Cites background from "Enhancing Learning Management Syste..."
...The Web Content Accessibility Guidelines (WCAG) (W3C, 2005) addressed Web page interoperability with the SR. WCAG compliance is necessary but not sufficient to create a blind-friendly digital environment (Babu and Singh, 2013)....
[...]
...The sight-centred design of digital resources presents challenges in SR-mediated interaction (Babu and Singh, 2013)....
[...]
...WCAG compliance is necessary but not sufficient to create a blind-friendly digital environment (Babu and Singh, 2013)....
[...]
...However, many users prefer listening to the page in chunks, rather than this continuous audio stream (Babu and Singh, 2013)....
[...]
Dissertation•
01 Sep 2017
TL;DR: This research aims to provide a further understanding of the problems blind users have on the web by comparing and contrasting problems between blind and sighted users and testing how design solutions to prevalent problems benefit blind users’ experience.
Abstract: The web is an eminently visual medium. However, not everyone accesses web content visually. Research shows that using the web is challenging for blind users. To create a good user experience for blind users on the web, we need a comprehensive understanding of the users’ problems. Currently, there is little knowledge about the problem differences between blind and sighted users, which makes it difficult to suggest and test design solutions that address these problems.
This research aims to provide a further understanding of the problems blind users have on the web by comparing and contrasting problems between blind and sighted users and testing how design solutions to prevalent problems benefit blind users’ experience.
The first study draws together the research literature into a common unified definition of web accessibility that was used to operationalise studies. The second study compared which verbal protocol (concurrent or retrospective) is better in user-based studies. The results showed that retrospective verbal protocol is a better option for eliciting problems on the web for blind and sighted users.
Then, an empirical study compared the problems between blind and sighted users on the web. The results showed that the problems the two user groups encounter largely differ. There are specific problem types distinct to blind users, but also the characteristics of the problem types that had instances by both user groups were very different. Moreover, many problems blind users encounter were in relation to the search and browse features of the websites.
A further investigation by two studies with blind users of how specific design solutions to prevalent problems users had (poor page structure, lack of feedback and excessive effort) in this specific design aspect showed that simple design solutions improve specific aspects of users’ experience. Although, for major improvements in the overall user experience a combination of design solutions is needed.
11 citations
Additional excerpts
...…2004; Lazar et al., 2012; Ramayah, Jaafar, & Yatim, 2013; Yoon, Newberry, Hulscher, & Dols, 2013), non-descriptive links (AbuDoush et al., 2013; R. Babu & Singh, 2013a; Brebner & Parkinson, 2006; Byerley & Beth Chambers, 2002; Disability Rights Commission, 2004; Federici et al., 2005; André…...
[...]
DOI•
23 Jan 2014TL;DR: It shows how blind users think, act and perceive in performing common social media functions non-visually, and has implications for the design of non-visual user interfaces to access social media through ‘Internet of Things’ and in multi-tasking situations.
Abstract: Social media allow people to communicate, collaborate and socialize for personal and professional matters. However, their sight-centered design can present access and usability problems for the blind. Existing quantitative approaches to usability testing do not provide in-depth assessment of the problem. This paper presents a qualitative approach to test social media usability, and illustrates its application to evaluate Facebook for the blind. Think-aloud observation of six blind participants generated verbal evidence of their Facebook interaction experiences. Verbal protocol analysis explained the nature of interaction challenges in performing common Facebook functions. Design standards analysis explained design errors in Facebook interface. It helped identify remedial measures to potentially improve Facebook usability. Findings demonstrate the utility of the qualitative approach to feasibly evaluate social media usability for blind users. It shows how blind users think, act and perceive in performing common social media functions non-visually. This has implications for the design of non-visual user interfaces to access social media through ‘Internet of Things’ and in multi-tasking situations.
9 citations
Cites background or methods from "Enhancing Learning Management Syste..."
...Its hallmark is a combination user-centered and task-based approach, [15] with verbal protocol analysis, [5] an integrated problemsolving framework [11] and design standards analysis [12] for an in-depth, contextually-situated and experiential understanding of the problem....
[...]
...Design standard analysis [12] identified design errors responsible and potential remedial measures....
[...]
...Babu and Singh [12] employed design standards analysis to determine the accessibility and usability character of an interaction challenge....
[...]
...[5] Babu, Singh and Ganesh [15] adapted VPA for a closer examination of blind users’ actions in completing online tasks....
[...]
01 Jan 2015
TL;DR: Results show that locating Friend's profile and Timeline, reading, writing, and posting messages were significantly challenging, and participants needed additional time and effort to perform these basic SNS functions that are integral parts of informal learning activities.
Abstract: Social Networking Sites (SNS) are increasingly used in academia to facilitate informal learning, knowledge co-creation, and collaborative problem-solving. Deriving these pedagogic benefits requires SNS usability for all. This paper reports results of an early investigation into SNS usability for blind students. Think-aloud observations generated verbal evidence of six participants' Facebook interaction experiences. Verbal protocol analysis revealed where and how challenges arose in online social interactions. Design standards analysis identified responsible interface elements and potential remedial measures. Results show that locating Friend's profile and Timeline, reading, writing, and posting messages were significantly challenging. Participants needed additional time and effort, and occasionally sighted help, to perform these basic SNS functions that are integral parts of informal learning activities. Feasible design improvements are proposed that merit further investigation. Implications for educators, learning technologists, cognitive scientists, and usability experts are discussed.
8 citations
References
More filters
Journal Article•
TL;DR: To improve students' retention rates in general chemistry, online homework was introduced into our curriculum Replacing quizzes directly by online homework significantly improved (p < 0005) success rates in second-term general chemistry as discussed by the authors.
Abstract: To improve students’ retention rates in general chemistry, online homework was introduced into our curriculum Replacing quizzes directly by online homework significantly improved (p < 0005) success rates in second-term general chemistry Attitudinal Likert survey results indicate that the majority of students completed the online homework assignments (90%) and viewed the assignments as worth the effort (835%) Students were overwhelming (857%) in their recommendation that online homework use should continue More consistent study habits were reported by 756% of students, and students reported using a suite of effective problem-solving approaches for questions marked as incorrect Our instructors have willingly embraced the use of online homework and point to the incredible amount of time savings for the instructor as reason enough to use online homework
81 citations
TL;DR: The paper explores the extent to which four frameworks that have been used in researching networked learning contexts can provide insights into the patterns of user behaviour that the authors see in Cloudworks.
Abstract: This paper describes a new social networking site, Cloudworks, which has been developed to enable discussion and sharing of learning and teaching ideas/designs and to promote reflective academic practice. The site aims to foster new forms of social and participatory practices (peer critiquing, sharing, user-generated content, aggregation, and personalisation) within an educational context. One of the key challenges in the development of the site has been to understand the user interactions and the changing patterns of user behaviour as it evolves. The paper explores the extent to which four frameworks that have been used in researching networked learning contexts can provide insights into the patterns of user behaviour that we see in Cloudworks. The paper considers this within the current debate about the new types of interactions, networking, and community being observed as users adapt to and appropriate new technologies.
81 citations
TL;DR: In this paper, the authors explored the correlation between teachers' attitudes towards change, contextual pedagogic and technological knowledge, and teachers' perception of school as a learning organization in the context of smart classes.
Abstract: Introduction Recent studies indicate that the implementation of innovative technologies in schools demand a systemic change in the school-culture (H. Becker, 2001; Eshet, 2007; Coffman, 2009; Cuban, 1988; Kent & MacNergney, 1999; Wallace, 2004) and involves a wide range of pedagogical, technological, and managerial factors, whose inter-relationships are not fully understood (Cunningham, 2009; Darling-Hammond, 2000; Day & Smethem, 2009; Fullan & Smith, 1999; Halverson & Smith, 2009). Most studies of technology-implementation in school systems report on disappointment from the projects' outcomes (Avidov-Ungar, 2010; Day & Smethem, 2009; De Val & Fuentes, 2003; Halverson & Smith, 2009; Mioduser, Nachmias, Forkosh, & Tubin, 2004) and emphasize the pivotal role of the teachers' pedagogical perspectives and beliefs in its success (De Freitas & Oliver, 2005; Selwyn, 2010). This claim was illustrated in the meta-analysis of thousands of technology-integration projects (Hattie, 2009), which found that the best predictors of students' achievements are related to teachers' activities and not to the technology itself. This emphasizes the importance of studying the teachers' perspectives and beliefs in the context of a technology-implementation project and the interrelationships between teacher-related variables that affect the project's success (Levin & Fullan, 2008). Studies of the factors that affect the success of implementing change in school in general and innovative technologies in particular (Fullan, 2000; Kontoghiorghes, Awbre, & Feurig, 2005; Sandy, 2010) indicate that the three major success factors are: (1) the teachers' attitudes towards change, (2) the teachers' contextual pedagogic and technological knowledge, and (3) the teachers' perception of school as a learning organization. This study explores the correlation between these three factors, in the context of a systemic change, related to the implementation of innovative "smart classes" in school. Below is a brief description of each of the three factors. 1. Teachers' Attitudes towards Change The teachers' attitudes towards change and their readiness to become active partners is considered a critical success factor (Avidov-Ungar, 2010; Coffman, 2009; Day & Gu, 2007; Fullan & Smith, 1999). Similarly, resistance to change is considered one of the main reasons for failure of processes that involve change in organizations in general and in the educational systems in particular (Fullan & Hargreaves, 1996; Zimmerman, 2006). In the case of innovative technology implementation in schools, teachers' resistance is reported by some studies to be the most important factor in the project's success (Del Val & Fuentes, 2003), mainly because the technology doesn't fit to their pedagogical practices and beliefs (Halverson & Smith, 2009; Harris & Hofer 2009). According to Del Val and Fuentes (2003), resistance to change is divided into cognitive resistance (focused on identifying and presenting weaknesses of the change and enlisting claims and reasons for maintaining the existing situation) and emotional resistance (focused on expressing negative feelings towards the change, such as anger, disdain, hostility or sadness). Emotional resistance is also accompanied by psychological symptoms such as tension, impatience, pessimism, apathy and disinterest. In many cases, resistance to change becomes active resistance, in which participants actively sabotage the process of change (Zimmerman, 2006). 2. Teachers' Technological Pedagogical Content Knowledge (TPACK) Teaching in a technological environment faces teachers with a wide range of pedagogical, cognitive and ergonomic challenges (Eshet, 2004; 2007; Koehler & Mishra, 2009, 2008; Mishra & Koehler, 2006). Unfortunately, most teachers are not trained to teach in technological environments (Day & Smethem, 2009), and many of them report on difficulties in effective integration of technologies in their teaching (De Freitas & Oliver, 2005). …
80 citations
TL;DR: The history of curriculum design and the implications of chaos and complexity for the development of new forms of curriculum and teaching implementation are discussed in this paper. But the focus of this paper is not on curriculum design, but on curriculum implementation.
Abstract: This paper has two main foci: (1) the history of curriculum design, and (2) implications from the new sciences of chaos and complexity for the development of new forms of curriculum design and teaching implementation. Regarding the first focus, the paper posits that there exist—to use Wittgenstein's phrase—‘family resemblances’ between Peter Ramus’ 16th century curriculum design and that of Ralph Tyler in the 20th century. While this 400‐year linkage is by no means linear, there are overlapping strands from Ramus to Comenius to the Puritans to colonial New England to Horace Mann to Ralph Tyler. What unites these strands, all belonging to the Protestant Methodization movement that swept across northern Europe into colonial America and the USA, is the concept of Method. Taylor's ‘time and motion’ studies set the stage for Tyler's Basic Principles of curriculum design—those starting with set goals and concluding with measured assessment. The second focus draws on the new sciences of chaos and complexity to d...
78 citations