The statistical tests used in the analysis of structural equation models with unobservable variables and measurement error are examined. A drawback of the commonly applied chi square test, in addit...

Evaluating Structural Equation Models with Unobservable Variables and Measurement Error

Despite the concern that has been expressed about potential method biases, and the pervasiveness of research settings with the potential to produce them, there is disagreement about whether they really are a problem for researchers in the behavioral sciences. Therefore, the purpose of this review is to explore the current state of knowledge about method biases. First, we explore the meaning of the terms “method” and “method bias” and then we examine whether method biases influence all measures equally. Next, we review the evidence of the effects that method biases have on individual measures and on the covariation between different constructs. Following this, we evaluate the procedural and statistical remedies that have been used to control method biases and provide recommendations for minimizing method bias.

Sources of Method Bias in Social Science Research and Recommendations on How to Control It

Several issues relating to goodness of fit in structural equations are examined. The convergence and differentiation criteria, as applied by Bagozzi, are shown not to stand up under mathematical or...

/pdf/structural-equation-models-with-unobservable-variables-and-1uwvf3jx9t.pdf

Structural Equation Models with Unobservable Variables and Measurement Error: Algebra and Statistics:

Despite the widespread use of exploratory factor analysis in psychological research, researchers often make questionable decisions when conducting these analyses. This article reviews the major design and analytical decisions that must be made when conducting a factor analysis and notes that each of these decisions has important consequences for the obtained results. Recommendations that have been made in the methodological literature are discussed. Analyses of 3 existing empirical data sets are used to illustrate how questionable decisions in conducting factor analyses can yield problematic results. The article presents a survey of 2 prominent journals that suggests that researchers routinely conduct analyses using such questionable methods. The implications of these practices for psychological research are discussed, and the reasons for current practices are reviewed.

/pdf/evaluating-the-use-of-exploratory-factor-analysis-in-4fpz0boat8.pdf

Evaluating the use of exploratory factor analysis in psychological research.

Purpose: This paper discusses partial least squares path modeling (PLS), a powerful structural equation modeling technique for research on international marketing. While a significant body of research provides guidance for the use of covariance-based structural equation modeling (CBSEM) in international marketing, there are no subject-specific guidelines for the use of PLS so far.Methodology/approach: A literature review of the use of PLS in international marketing reveals the increasing application of this methodology.Findings: This paper reveals the strengths and weaknesses of PLS in the context of research on international marketing, and provides guidance for multi-group analysis.Originality/value of paper: The paper assists researchers in making well-grounded decisions regarding the application of PLS in certain research situations and provides specific implications for an appropriate application of the methodology.

The Use of Partial Least Squares Path Modeling in International Marketing

Intraclass correlation reliablity estimates are based on the assumption that the various measures are equivalent. Joreskog's (1970) general model for the analysis of covariance structures can be used to test the validity of this assumption.

Intraclass Reliability Estimates: Testing Structural Assumptions

In recent years there has been an increasing emphasis on assessment results, as well as increasing concern about the nature of the most widely used forms of student assessment and uses that are made of the results. These conflicting forces have helped create a burgeoning interest in alternative forms of assessments, particularly complex, performance-based assessments. It is argued that there is a need to rethink the criteria by which the quality of educational assessments are judged, and a set of criteria that are sensitive to some of the expectations for performance-based assessments is proposed.

Complex, Performance-Based Assessment: Expectations and Validation Criteria

Use of tests and assessments as key elements in five waves of educational reform during the past 50 years are reviewed. These waves include the role of tests in tracking and selection emphasized in the 1950s, the use of tests for program accountability in the 1960s, minimum competency testing programs of the 1970s, school and district accountability of the 1980s, and the standards-based accountability systems of the 1990s. Questions regarding the impact, validity, and generalizability of reported gains, and the credibility of results in high-stakes accountability uses are discussed. Emphasis is given to three issues regarding currently popular accountability systems. These are (a) the role of content standards, (b) the dual goals of high performance standards and common standards for all students, and (c) the validity of accountability models. Some suggestions for dealing with the most severe limitations of accountability are provided.

Assessments and Accountability

TOC PART I THE MEASUREMENT AND ASSESSMENT PROCESS Chapter 1 Educational Testing and Assessment: Context, Issues, and Trends Educational Assessment: Barometer and Lever of Reform Five Decades of Test-Based Educational Reform Technological Advances in Testing and Assessment Public Concern About Testing and Assessment Chapter 2 The Role of Measurement and Assessment in Teaching Instructional Decisions Requiring Assessment Data Assessment, Test, and Measurement General Principles of Assessment Assessment and the Instructional Process Types of Assessment Procedures Summary of Assessment Categories Chapter 3 Instructional Goals and Objectives: Foundation for Assessment Instructional Objectives as Learning Outcomes From Overly Specific Objectives to the Thinking Curriculum Method of Stating Instructional Objectives Chapter 4 Validity Nature of Validity Major Considerations in Assessment Validation Content Considerations Construct Considerations Assessment-Criterion Relationships Consideration of Consequences Factors Influencing Validity Chapter 5 Reliability and Other Desired Characteristics Nature of Reliability Determining Reliability by Correlation Methods Standard Error of Measurement Factors Influencing Reliability Measures Reliability of Assessments Evaluated in Terms of a Fixed Performance Standard Usability PART II CLASSROOM TESTS AND ASSESSMENTS Chapter 6 Planning Classroom Tests and Assessments The Purpose of Classroom Testing and Assessment Developing Specifications for Tests and Assessments Selecting Appropriate Types of Items and Assessment Tasks Considerations in Preparing Relevant Test Items and Assessment Tasks Chapter 7 Constructing Objective Test Items: Simple Forms Short-Answer Items True-False or Alternative-Response Items Matching Exercises Chapter 8 Constructing Objective Test Items: Multiple-Choice Forms Characteristics of Multiple-Choice Items Uses of Multiple-Choice Items Advantages and Limitations of Multiple-Choice Items Suggestions for Constructing Multiple-Choice Items Chapter 9 Measuring Complex Achievement: The Interpretive Exercise Nature of the Interpretive Exercise Forms and Uses of the Interpretive Exercise Advantages and Limitations of Interpretive Exercises Suggestions for Constructing Interpretive Exercises Chapter 10 Measuring Complex Achievement: Essay Questions Forms and Uses of Essay Questions Summary Comparison of Learning Outcomes Measured Advantages and Limitations of Essay Questions Suggestions for Constructing Essay Questions Scoring Criteria Suggestions for Scoring Essay Questions Chapter 11 Measuring Complex Achievement: Performance-Based Assessments Types of Performance-Based Assessment Advantages and Limitations of Performance Assessments Suggestions for Constructing Performance Tasks Performance Criteria Scoring Rubrics and Rating Scales Checklists Student Participation in Rating Chapter 12 Portfolios What Qualifies as a Portfolio of Student Work? Potential Strengths and Weaknesses of Portfolios Purpose of Portfolios Guidelines for Portfolio Entries Guidelines and Students' Role in Selection of Portfolio Entries and Self-Evaluation Evaluation Criteria Using Portfolios in Instruction and Communication Chapter 13 Assessment Procedures: Observational Techniques, Peer Appraisal, and Self-Report Anecdotal Records Student Judgments and Reports Peer Appraisal Self-Report Techniques Attitude Measurement Interest Inventories Personality Measures Chapter 14 Assembling, Administering, and Appraising Classroom Tests and Assessments Assembling the Classroom Test Administering and Scoring Classroom Tests and Assessments Appraising Classroom Tests and Assessments Building a File of Effective Items and Tasks Chapter 15 Grading and Reporting Functions of Grading and Reporting Systems Types of Grading and Reporting Systems Multiple Grading and Reporting Systems Assigning Letter Grades Record-Keeping and Grading Software Conducting Parent-Teacher Conferences Reporting Standardized Test Results to Parents PART III SELECTING AND USING PUBLISHED TESTS Chapter 16 Achievement Tests Characteristics of Standardized Achievement Tests Standardized Tests versus Informal Classroom Tests Standardized Achievement Test Batteries Achievement Tests in Specific Areas Customized Achievement Tests Individual Achievement Tests Chapter 17 Aptitude Tests Achievement and Aptitude Tests Scholastic Aptitude and Learning Ability Group Tests of Learning Ability Individual Tests Cautions in Interpreting and Using Learning Ability Scores Culture-Fair Testing Differential Aptitude Testing Chapter 18 Test Selection, Administration, and Use Obtaining Information About Published Tests Selecting Appropriate Tests Administering Published Tests Using Results of Published Tests Chapter 19 Interpreting Test Scores and Norms Methods of Interpreting Test Scores Grade Norms Percentile Rank Standard Scores Profiles Skill Analysis Judging the Adequacy of Norms Using Local Norms Cautions in Interpreting Test Scores APPENDICES Appendix A Elementary Statistics Appendix B Professional Journals for Locating Measurement and Assessment Articles Appendix C Content Standards Appendix D Standards for Teacher Competence in Educational Assessment of Students Appendix E Test Publishers Appendix F Selected Published Tests Appendix G Taxonomy of Educational Objectives Appendix H Relating Assessment Procedures to Instructional Objectives

Measurement and Assessment in Teaching

The No Child Left Behind Act of 2001 substantially increases the testing requirements for states and sets demanding accountability standards for schools, districts, and states with measurable adequate yearly progress (AYP) objectives for all students and subgroups of students defined by socioeconomic background, race–ethnicity, English language proficiency, and disability. However, states’ content standards, the rigor of their tests, and the stringency of their performance standards vary greatly. Consequently, the percentage of students who score at the proficient level or higher on the state assessments varies radically from state to state. Some states have farther to go than others to meet the mandated target of 100% proficient within 12 years. These differences are illustrated and the implications for achieving AYP targets are discussed. Also addressed are possible uses of results from the biennial state-level administrations of the National Assessment of Educational Progress as a means of leveling the...

Robert L. Linn

Papers

Intraclass Reliability Estimates: Testing Structural Assumptions

Complex, Performance-Based Assessment: Expectations and Validation Criteria

Assessments and Accountability

Measurement and Assessment in Teaching

Accountability Systems: Implications of Requirements of the No Child Left Behind Act of 2001: