Ten years ago, we presented the DeLone and McLean Information Systems (IS) Success Model as a framework and model for measuring the complex-dependent variable in IS research. In this paper, we discuss many of the important IS success research contributions of the last decade, focusing especially on research efforts that apply, validate, challenge, and propose enhancements to our original model. Based on our evaluation of those contributions, we propose minor refinements to the model and propose an updated DeLone and McLean IS Success Model. We discuss the utility of the updated model for measuring e-commerce system success. Finally, we make a series of recommendations regarding current and future measurement of IS success.

/pdf/the-delone-and-mclean-model-of-information-systems-success-a-4qyq8geftq.pdf

The DeLone and McLean Model of Information Systems Success: A Ten-Year Update

Through qualitative and empirical research, the authors find that the service quality construct conforms to the structure of a third-order factor model that ties service quality perceptions to distinct and actionable dimensions: outcome, interaction, and environmental quality. In turn, each has three subdimensions that define the basis of service quality perceptions. The authors further suggest that for each of these subdimensions to contribute to improved service quality perceptions, the quality received by consumers must be perceived to be reliable, responsive, and empathetic. The authors test and support this conceptualization across four service industries. They consider the research and managerial implications of the study and its limitations.

https://www.jstor.org/stable/pdf/3203465.pdf

Some New Thoughts on Conceptualizing Perceived Service Quality: A Hierarchical Approach:

The issue of whether IS positivist researchers were validating their instruments sufficiently was initially raised fifteen years ago. Rigor in IS research is still one of the critical scientific issues facing the field. Without solid validation of the instruments that are used to gather data on which findings and interpretations are based, the very scientific basis of the profession is threatened. This study builds on four prior retrospectives of IS research that conclude that IS positivist researchers continue to face major barriers in instrument, statistical, and other forms of validation. It goes beyond these studies by offering analyses of the state-of-the-art of research validities and deriving specific heuristics for research practice in the validities. Some of these heuristics will, no doubt, be controversial. But we believe that it is time for the IS academic profession to bring such issues into the open for community debate. This article is a first step in that direction. Based on our interpretation of the importance of a long list of validities, this paper suggests heuristics for reinvigorating the quest for validation in IS research via content/construct validity, reliability, manipulation validity, and statistical conclusion validity. New guidelines for validation and new research directions are offered.

/pdf/validation-guidelines-for-is-positivist-research-5fusz79fbw.pdf

Validation guidelines for is positivist research

Online shopping provides convenience to Web shoppers, yet its electronic format changes information-gathering methods traditionally used by customers. This change raises questions concerning customer satisfaction with the online purchasing process. Web shopping involves a number of phases, including the information phase, in which customers search for information regarding their intended purchases. The purpose of this paper is to develop theoretically justifiable constructs for measuring Web-customer satisfaction during the information phase.By synthesizing the expectation-disconfirmation paradigm with empirical theories in user satisfaction, we separate Web site quality into information quality (IQ) and system quality (SQ), and propose nine key constructs for Web-customer satisfaction. The measurements for these constructs are developed and tested in a two-phase study. In the first phase, the IQ and SQ dimensions are identified, and instruments for measuring them are developed and tested. In the second phase, using the salient dimensions of Web-IQ and Web-SQ as the basis for formulating first-order factors, we develop and empirically test instruments for measuring IQ and SQ-satisfaction. Moreover, this phase involves the design and test of second-order factors for measuring Web-customer expectations, disconfirmation, and perceived performance regarding IQ and SQ. The analysis of the measurement model indicates that the proposed metrics have a relatively high degree of validity and reliability. The results of the study provide reliable instruments for operationalizing the key constructs in the analysis of Web-customer satisfaction within the expectation-disconfirmation paradigm.

The Measurement of Web-Customer Satisfaction: An Expectation and Disconfirmation Approach

Since DeLone and McLean (D&M) developed their model of IS success, there has been much research on the topic of success as well as extensions and tests of their model. Using the technique of a qualitative literature review, this research reviews 180 papers found in the academic literature for the period 1992–2007 dealing with some aspect of IS success. Using the six dimensions of the D&M model – system quality, information quality, service quality, use, user satisfaction, and net benefits – 90 empirical studies were examined and the results summarized. Measures for the six success constructs are described and 15 pairwise associations between the success constructs are analyzed. This work builds on the prior research related to IS success by summarizing the measures applied to the evaluation of IS success and by examining the relationships that comprise the D&M IS success model in both individual and organizational contexts.

https://link.springer.com/content/pdf/10.1057%2Fejis.2008.15.pdf

Measuring information systems success: models, dimensions, measures, and interrelationships

A recent MIS Quarterly article rightfully points out that service is an important part of the role of the information systems (IS) department and that most IS assessment measures have a product orientation (Pitt, et al. 1995). The article went on to suggest the use of an IS-context-modified version of the SERVQUAL instrument to assess the quality of the services supplied by an information services provider (Parasuraman, et al. 1985, 1988, 1991). However, a number of problems with the SERVQUAL instrument have been discussed in the literature (e.g., Babakus and Boller 1992; Carman 1990; Cronin and Taylor 1992, 1994; Teas 1993). This article reviews that literature and discusses some of the implications for measuring service quality in the information systems context. Findings indicate that SERVQUAL suffers from a number of conceptual and empirical difficulties. Conceptual difficulties include the operationalization of perceived service quality as a difference or gap score, the ambiguity of the expectations construct, and the unsuitability of using a single measure of service quality across different industries. Empirical problems, which may be linked to the use of difference scores, include reduced reliability, poor convergent validity, and poor predictive validity. This suggests that (1) some alternative to difference scores is preferable and should be utilized; (2) if used, caution should be exercised in the interpretation of IS-SERVQUAL difference scores; and (3) further work is needed in the development of measures for assessing the quality of IS services.

Measuring information systems service quality: concerns on the use of the SERVQUAL questionnaire

Information Systems IS managers are under increasing pressure to justify the value and contribution of IS expenditures to the productivity, quality, and competitiveness of the organization. This paper examines the need for IS assessment and suggests a comprehensive IS assessment framework linked to organizational performance using existing IS assessment theory as a base and incorporating measurement concepts from other disciplines. The existing models of IS success are updated to include the emerging IS success dimensions of service quality and work group impact and provide a comprehensive method for organizing the various measures of IS success. In addition, many new measures from recent research are presented to supplement the lists supplied by previous research. Additional research is suggested to advance the IS assessment contingency theory. Such a theory has the potential to contribute to the quality and productivity of the IS function and the larger organization by providing feedback to manage and improve the IS function to better meet the needs of the organization.

A Comprehensive Model for Assessing the Quality and Productivity of the Information Systems Function: Toward a Theory for Information Systems Assessment

The postmortem examination of failed IT projects reveals that long before the failure there were significant symptoms or “early warning signs.” This article describes the top 12 people-related and project-related IT project risks, based on “early warning sign” data collected from a panel of 19 experts and a survey of 55 IT project managers.

Early Warning Signs of it Project Failure: The Dominant Dozen

The SERVQUAL questionnaire (Parasuraman, Zeithaml, & Berry, 1988) is one of the preeminent instruments for measuring the quality of services as perceived by the customer. In a recent Decision Sciences article, Kettinger and Lee (1995) suggested the use of a modified SERVQUAL instrument to assess the quality of the services supplied by an information services provider. However, a number of problems with the SERVQUAL instrument are discussed in the literature. This article provides an illustrative example utilizing data collected from 138 executive and information systems professional customers of a multibillion dollar information services provider in order to examine the validity and reliability of Kettinger and Lee's (1995) modified SERVQUAL instrument. Results of analyses do not confirm the findings of Kettinger and Lee. Moreover, it appears that the use of difference scores in calculating SERVQUAL contributes to problems with the reliability, discriminant validity, convergent validity, and predictive validity of the measure. These findings suggest that caution should be exercised in the use of SERVQUAL scores and that further work is needed in the development of measures for assessing the quality of information services.

Cautions on the Use of the SERVQUAL Measure to Assess the Quality of Information Systems Services

Since 1980, the Society for Information Management (SIM), in collaboration with a team of MIS academics, has surveyed its members, asking questions about themselves and their IT management concerns. These studies have expanded over the years to become one of the most comprehensive investigations of IT and IT leadership in organizations. Continuously improving the relevance and accuracy of the studies, this year we have: (a) added questions about cybersecurity, software development and cloud computing; (b) expanded the questions on IT organization structure and governance, IT performance measures, management practices, work activities and spending; and (c) included 2014’s questions about service catalogs and chargebacks (which were omitted intentionally in 2015). We emailed a personal link to the online questionnaire to each of the 5,332 SIM members and received 1,213 responses, including 490 CIOs, representing 801 unique organizations with average annual revenues of $4.75 billion. Total revenue of these organizations was nearly

https://aisel.aisnet.org/cgi/viewcontent.cgi?article=1377&context=misqe

Leon A. Kappelman

Papers

Measuring information systems service quality: concerns on the use of the SERVQUAL questionnaire

A Comprehensive Model for Assessing the Quality and Productivity of the Information Systems Function: Toward a Theory for Information Systems Assessment

Early Warning Signs of it Project Failure: The Dominant Dozen

Cautions on the Use of the SERVQUAL Measure to Assess the Quality of Information Systems Services

The 2015 SIM IT Issues and Trends Study