In 2009, we published a paper in which we showed how three independent sources of data indicated that, rather than being a unidimensional measure of perceived usability, the System Usability Scale apparently had two factors: Usability (all items except 4 and 10) and Learnability (Items 4 and 10). In that paper, we called for other researchers to report attempts to replicate that finding. The published research since 2009 has consistently failed to replicate that factor structure. In this paper, we report an analysis of over 9,000 completed SUS questionnaires that shows that the SUS is indeed bidimensional, but not in any interesting or useful way. A comparison of the fit of three confirmatory factor analyses showed that a model in which the SUS's positive-tone (odd-numbered) and negative-tone (even-numbered) were aligned with two factors had a better fit than a unidimensional model (all items on one factor) or the Usability/Learnability model we published in 2009. Because a distinction based on item tone is of little practical or theoretical interest, we recommend that user experience practitioners and researchers treat the SUS as a unidimensional measure of perceived usability, and no longer routinely compute Usability and Learnability subscales.

Revisiting the factor structure of the system usability scale

The System Usability Scale (SUS) is the most widely used standardized questionnaire for the assessment of perceived usability. This review of the SUS covers its early history from inception in the ...

The System Usability Scale: Past, Present, and Future

We present a theoretical and empirical explication of the intervention of Concept-Oriented Reading Instruction (CORI) that is designed to increase students' reading comprehension and motivation for reading. The framework specifies a set of five motivational constructs that represent goals for the instructional intervention. Necessary cognitive goals in reading are also presented. For this intervention, the five instructional practices of relevance, choice, success, collaboration, and thematic unit that are prominent in CORI are portrayed as components that are aligned with motivational constructs. The impact of CORI on some of the motivational processes, cognitive competencies, and reading comprehension are presented in the form of a meta-analysis of 11 CORI studies with 75 effect sizes on 20 outcome variables. The CORI motivational intervention is compared to laboratory treatments and other field studies.

Contributions of concept-oriented reading instruction to knowledge about interventions for motivations in reading.

The aim of this study was to investigate gender differences in the relationship between reading ability, frequency of reading and attitudes and beliefs relating to reading and school. Two hundred and thirty-two 10-year-old children (117 male) completed a reading comprehension test and a questionnaire exploring the following areas: frequency of reading, attitude to reading, attitude to school, competency beliefs and perceived academic support (from peers and teacher). Overall, girls had better reading comprehension, read more frequently and had a more positive attitude to reading and school. However, smaller gender differences were found in reading ability than in attitudes and frequency of reading. Indeed, effect sizes for gender differences in reading were found to be small in this and other studies. Reading ability correlated with both boys' and girls' reading frequency and competency beliefs; however, only boys' reading ability was associated with their attitude to reading and school. Notably, gender differences were found predominantly in the relationship between factors, rather than solely in the factors themselves. Previous research has neglected to study these relationships, and has focused instead on the gender differences found in individual factors. Conclusions are made regarding the applicability of these findings to the school situation.

/pdf/gender-differences-in-reading-ability-and-attitudes-4jtwrd792k.pdf

Gender differences in reading ability and attitudes: examining where these differences lie

A meta-analysis of the relationship between attitudes in reading and achievement in reading was conducted to provide a statistical summary to the observed variability in the magnitude of previously reported effect sizes. A total of 32 studies, with a total sample size of 224,615 were used, and included a total of 118 effect sizes. A multi-level approach was used in meta-analysis to determine if variance in the magnitude of effect sizes could be partitioned to study (level 1) and moderator (level 2) levels by using a mixed model approach. Results from the meta-analysis indicated that the mean strength of the relationship between reading attitudes and achievement is moderate (Zr=.32), while stronger for students in elementary school (Zr=.44) when compared with middle school students (Zr=.24). Findings related to selected moderator variables are discussed, with suggestions for future research.

A meta‐analysis of the relationship between student attitudes towards reading and achievement in reading

Nowadays, practitioners extensively apply quick and reliable scales of user satisfaction as part of their user experience analyses to obtain well-founded measures of user satisfaction within time and budget constraints. However, in the human–computer interaction literature the relationship between the outcomes of standardized satisfaction scales and the amount of product usage has been only marginally explored. The few studies that have investigated this relationship have typically shown that users who have interacted more with a product have higher satisfaction. The purpose of this article was to systematically analyze the variation in outcomes of three standardized user satisfaction scales (SUS, UMUX, UMUX-LITE) when completed by users who had spent different amounts of time with a website. In two studies, the amount of interaction was manipulated to assess its effect on user satisfaction. Measurements of the three scales were strongly correlated and their outcomes were significantly affected by the amo...

Assessing User Satisfaction in the Era of User Experience: Comparison of the SUS, UMUX, and UMUX-LITE as a Function of Product Experience

We propose a class of multidimensional Item Response Theory models for polytomously-scored items with ordinal response categories. This class extends an existing class of multidimensional models for dichotomously-scored items in which the latent abilities are represented by a random vector assumed to have a discrete distribution, with support points corresponding to different latent classes in the population. In the proposed approach, we allow for different parameterizations for the conditional distribution of the response variables given the latent traits, which depend on the type of link function and the constraints imposed on the item parameters. Moreover, we suggest a strategy for model selection that is based on a series of steps consisting of selecting specific features, such as the dimension of the model (number of latent traits), the number of latent classes, and the specific parameterization. In order to illustrate the proposed approach, we analyze a dataset from a study on anxiety and depression...

/pdf/a-class-of-multidimensional-latent-class-irt-models-for-56qnnq05iy.pdf

A class of Multidimensional Latent Class IRT models for ordinal polytomous item responses

The results of the Progress in International Reading Literacy Study 2001 (PIRLS) were reported in 2003. In addition to data about the reading achievements of 10-year-olds in 35 countries, the study also collected questionnaire information from children, their teachers, headteachers and parents. One aspect of the data that attracted comment in the media was the fact that, despite high achievement on the reading tests, children in England were reported as having relatively poor attitudes to reading, compared to children in many other countries. A review of the results of selected surveys over the past 30 years suggests that there may be some evidence of attitudes to reading in primary schools becoming less positive. Preliminary analysis of the attitude and achievement data from PIRLS suggests a more complex picture than that presented in the summary index published in 2003.

Michela Gnaldi

Papers

Assessing User Satisfaction in the Era of User Experience: Comparison of the SUS, UMUX, and UMUX-LITE as a Function of Product Experience

A class of Multidimensional Latent Class IRT models for ordinal polytomous item responses

Good readers but at a cost? Attitudes to reading in England

MultiLCIRT: An R package for multidimensional latent class item response models

Statistical Analysis of Questionnaires: A Unified Approach Based on R and Stata