Showing papers in "Educational and Psychological Measurement in 1995"
TL;DR: In this paper, a multistage, four sample study was conducted to develop a conceptually consistent and psychometrically sound measure of decision-making style, and items were written to assess rational, avoidant, intuitive, and dependent decision making styles.
Abstract: A multistage, four sample study was conducted to develop a conceptually consistent and psychometrically sound measure of decision-making style. Construct definitions were developed from prior theory, and items were written to assess rational, avoidant, intuitive, and dependent decision-making styles. A series of principal-axis factor analyses with varimax rotation and subsequent item analyses were conducted to develop four conceptually distinct scales with acceptable internal consistency (alpha ranging from .68 to .94) and a stable factor structure. In the process of scale development, a fifth style (spontaneous) was identified. Tests for independence among the five decision-making style scales and concurrent validity analyses were conducted. Finally, discussion of the new instrument with reference to the extant literature is provided.
829 citations
TL;DR: The most important decision that can be made in the use of factor analysis is the number of factors to retain this article, and many studies have consistently shown that Horn's Parallel Analysis is the best parallel analysis.
Abstract: One of the most important decisions that can be made in the use of factor analysis is the number of factors to retain. Numerous studies have consistently shown that Horn's parallel analysis is the ...
645 citations
TL;DR: This paper used forward multiple regression analyses to identify the best combination for predicting GPA in a sample of 201 psychology students, using self-control, SAT score, average grade earned in high school, and 32 personality variables.
Abstract: Total SAT score, average grade earned in high school, and 32 personality variables are examined via forward multiple regression analyses to identify the best combination for predicting GPA in a sample of 201 psychology students. Average grade earned in high school enters first, accounting for 19% of the variance in GPA. Self-control enters second, and SAT third; these account for 9% and 5% of the variance, respectively. No other predictors accounted for substantial portions of variance. This pattern of results converges with findings reported by other investigators using other measures of personality. It was recommended that the global trait of self-control or conscientiousness be systematically assessed and used in college admissions decisions.
525 citations
TL;DR: Three problems with stepwise applications are explored in some detail, including the fact that computer packages use incorrect degrees of freedom in their stepwise computations, resulting in artifactually greater likelihood of obtaining spurious statistical significance.
Abstract: Stepwise methods are frequently employed in educational and psychological research, both to select useful subsets of variables and to evaluate the order of importance of variables. Three problems with stepwise applications are explored in some detail. First, computer packages use incorrect degrees of freedom in their stepwise computations, resulting in artifactually greater likelihood of obtaining spurious statistical significance. Second, stepwise methods do not correctly identify the best variable set of a given size, as illustrated by a concrete heuristic example. Third, stepwise methods tend to capitalize on sampling error and thus tend to yield results that are not replicable.
425 citations
TL;DR: The Survey of Attitudes Toward Statistics (SATS) as mentioned in this paper was designed for use in both research and instruction, and a panel of instructors and introductory statistics students identified by consensus four facets of attitudes toward statistics: affect-positive and negative feelings concerning statistics; cognitive competence-attitudes about intellectual knowledge and skills when applied to statistics; value-attitude about the usefulness, relevance, and worth of statistics.
Abstract: The Survey of Attitudes Toward Statistics (SATS) was designed for use in both research and instruction. A panel of instructors and introductory statistics students identified by consensus four facets of attitudes toward statistics: (a) Affect-positive and negative feelings concerning statistics; (b) Cognitive Competence-attitudes about intellectual knowledge and skills when applied to statistics; (c) Value-attitudes about the usefulness, relevance, and worth of statistics; and (d) Difficulty-attitudes about the difficulty of statistics as a subject. This structure was validated for a sample of undergraduate students using confirmatory factor analysis. Additional validity evidence was obtained through the correlation of the SATS with Wise's Attitudes Toward Statistics scale, which showed significant, positive relationships between the two instruments.
408 citations
TL;DR: In this article, the potential for bias in reliability estimation and for errors in item selection when alpha or unit-weighted omega are used are explored under a variety of simulated conditions, and the results suggest that composite reliability may be used as an assessment tool, but should not be used for item selection tool in structural equations modeling.
Abstract: Two composite reliability measures, coefficient alpha and coefficient omega with unit weights (otherwise known as construct reliability), are commonly used in structural equations modeling. However, a third measure, omega with unequal weights, is more theoretically appropriate. The potential for bias in reliability estimation and for errors in item selection when alpha or unit-weighted omega are used are explored under a variety of simulated conditions. The results suggest that composite reliability may be used as an assessment tool, but should not be used as an item selection tool in structural equations modeling.
353 citations
TL;DR: The Quick Discrimination Index (QDI) as mentioned in this paper is a 30-item, Likert-type self-report inventory that measures attitudes toward racial discrimination in the United States.
Abstract: A series of empirical studies was conducted to develop and validate the Quick Discrimination Index (QDI), a 30-item, Likert-type self-report inventory. The QDI measures attitudes toward racial dive...
185 citations
TL;DR: In this paper, the authors describe the development of the new fifth edition of the Sixteen Personality Factor Questionnaire (16PF) and present a check of the factor structure of this new edition.
Abstract: The purposes of the present article were to describe briefly the development of the new fifth edition of the Sixteen Personality Factor Questionnaire (16PF) and to present a check of the factor structure of this new edition. Since its initial publication in 1949, the 16PF has undergone four revisions (1956, 1962, 1968, and 1993). The latest revision (resulting in the 16PF fifth edition) was begun in 1988 with the goals of updating and improving item content, standardizing on a current population sample, and generally refining the test. Items were selected in a four-stage iterative process, beginning with an item pool containing the best items from all five current forms of the 16PF (Forms A, B, C, D, and CAQ Part 1). Specific item criteria are described. Factor analyses of the new items and scales are described and show strong support for the validity of the factor structure of the fifth edition and for its continuity with earlier versions.
133 citations
TL;DR: The authors clarified conceptual and empirical ambiguities regarding self-efficacy and self-esteem in the educational, psychological, and organizational literatures, and pointed out the validity of selfefficacy in these literatures.
Abstract: This article seeks to clarify some conceptual and empirical ambiguities regarding self-efficacy and self-esteem in the educational, psychological, and organizational literatures. The validity of se...
127 citations
TL;DR: A meta-analytic review was conducted on published studies examining the relationship between performance on the quantitative and verbal components of the Graduate Record Examination (GRE-Q and GRE-V, respectively) and graduate grade point average (GGPA) as mentioned in this paper.
Abstract: A meta-analytic review was conducted on published studies examining the relationship between performance on the quantitative and verbal components of the Graduate Record Examination (GRE-Q and GRE-V, respectively) and graduate grade point average (GGPA) The weighted average effect sizes (d) for GRE-Q and GRE-V were 445 and 591, respectively, and their resultant correlation coefficients were 22 (GRE-Q and GGPA) and 28 (GRE-V and GGPA), p = ns Thus an average of 63% of the variance in graduate level academic achievement, as represented by GGPA, was accounted for by performance on the GRE-Q and GRE-V The utility of this examination in the graduate admissions process is discussed
109 citations
TL;DR: In this paper, a study was conducted to scale the Attitudes Toward Inclusive Education Scale (ATIES) with Rasch analysis, which measured positive and negative attitudes toward integrating children with various disabilities into regular classes.
Abstract: This study was conducted to scale the Attitudes Toward Inclusive Education Scale (ATIES) with Rasch analysis. The ATIES records positive and negative attitudes toward integrating children with various disabilities into regular classes. Rasch analysis resolved the nonlinear relationship between the finite range of recorded ATIES scores and the conceptually infinite range of attitudes. Results showed that the 16-item scale defined a unidimensional attitudinal variable and yielded interval measures of attitudes toward inclusive education.
TL;DR: Kim et al. as mentioned in this paper studied the extent to which performance in selected measures of creativity involving both verbal and visual tasks was related to school achievement as revealed by a quantitative indicator (percentage score) and expressed preference for a given learning and thinking style as revealed in a standardized measure (Style of Learning and Thinking by Torrance).
Abstract: Two major purposes of this study were to ascertain for a sample of 193 11th-grade Korean high school students (92 males and 101 females) the extent to which performance in selected measures of creativity involving both verbal and visual tasks was related to (a) school achievement as revealed by a quantitative indicator (percentage score) and (b) the expressed preference for a given learning and thinking style as revealed in a standardized measure (Style of Learning and Thinking by Torrance) intended to portray left-brain, right-brain, or an integrated left-brain right-brain function. A secondary purpose was to identify possible gender differences of Korean students in the variables under study. The following conclusions became evident. First, measures of creativity translated from the Torrance Tests of Creative Thinking show little, if any, relationship to school performance. Second, Korean high school females may be expected to exhibit higher average levels of performance on creativity tests than will th...
TL;DR: In this article, an iterative procedure for improving confirmatory model fit is demonstrated within the context of Bentler's EQS and maximum likelihood factor analysis using a 21-statement attitude-type instrument.
Abstract: Using a 21-statement attitude-type instrument, an iterative procedure for improving confirmatory model fit is demonstrated within the context of Bentler's EQS and maximum likelihood factor analysis. Each iteration systematically eliminates the poorest fitting statement as identified by a variable fit index. The statements were hypothesized to define a three-factor independent cluster solution, one factor being associated with each of three a priori scales. Initially, the statements did not define the subscales as hypothesized. Through a series of 11 iterations, 11 variables were eliminated. The resulting three-factor independent cluster model fit the data extremely well.
TL;DR: In this article, a non-technical explanation of logit estimates of pediatric functional ability is presented, along with an illustration of two alternative transformations of the logit estimate of functional ability, and a pragmatic solution for obtaining a functional ability estimate for a zero or perfect performance score.
Abstract: This article presents (a) a nontechnical explanation of Rasch model "logit" estimates of pediatric functional ability, (b) an illustration of two alternative transformations of logit estimates of functional ability, and (c) a pragmatic solution to the problem of obtaining a functional ability estimate for a zero or perfect performance score. An explanation of logits is offered because their presentation to practitioners in the clinic is often complicated and confusing. The transformations are performed because they remove negative and decimal valued estimates of functioning. The issue of zero and perfect scores is addressed because these are often the scores obtained by the youngest and oldest children in a clinical setting.
TL;DR: In this paper, univariate and multivariate generalizability theory is employed to examine psychometric characteristics of listening and writing tests developed by American College Testing (ACT) for its Work Keys program.
Abstract: In this article, univariate and multivariate generalizability theory is employed to examine psychometric characteristics of Listening and Writing tests developed by American College Testing (ACT) for its Work Keys program. For both of these tests, the same tape-recorded messages provide the stimuli for written responses that are evaluated by raters. Of particular concern are the numbers of messages and raters needed for adequate measurement precision. The occurrence of relatively large person-message interactions suggested that several (at least six) messages are probably required. Also, the analyses suggest that it is highly desirable to employ at least two raters. For these sample sizes, however, the multivariate results are less encouraging in part because of correlated errors.
TL;DR: In this article, the authors evaluate the ability of different models, including the classification techniques of discriminant analysis, logistic regression, and neural networks, to predict the academic success of MBA students.
Abstract: The decision to accept a student into a graduate program is a difficult one, based upon many factors that are used to predict the success of the applicant. Typically, regression analysis has been used to develop a prediction mechanism. Unfortunately, as is shown in this article, these regression models can be ineffective in predicting success or failure. This article evaluates the ability of different models, including the classification techniques of discriminant analysis, logistic regression, and neural networks, to predict the academic success of MBA students. The conclusions of this study are that (a) classification techniques may be an appropriate approach to the problem, (b) predicting success and failure of graduate students is difficult using only the typical data describing the subjects, and (c) nonparametric procedures, such as neural networks, perform at least as well as traditional methods and are worthy of further investigation.
TL;DR: In this paper, the authors developed and validated a measure for school principal-perceived burnout, based on a questionnaire containing items relating to their feelings about themselves, their environment, and work.
Abstract: The purpose of this study was to develop and validate a measure for school principal-perceived burnout. A total of 821 elementary and secondary school principals in Israel completed a questionnaire containing items relating to their feelings about themselves, their environment, and work. Data were factor analyzed, and three factors, covering 22 items, emerged: (a) exhaustion-feelings of emotional, physical, and cognitive fatigue, direct acknowledgment of being burned out by running the school; (b) aloofness-the principal's expressed desire to disengage from teachers, students, and parents; and (c) deprecation-having negative attitudes toward teachers, students, and parents, belittling their efforts and derogating their abilities.
TL;DR: The authors examined the question of the optimal number of choices on a multiple-choice test from an information theory perspective and found that, in general, three choices to a multiplechoice test item seem optimal.
Abstract: This study examines the question of the optimal number of choices on a multiple-choice test from an information theory perspective. Results are compared to addressing this question using more traditional statistical approaches. Based on information theory, the study reveals that, in general, three choices to a multiple-choice test item seem optimal. This finding verifies what other researchers have found from statistical and observational (item analysis) approaches.
TL;DR: In this article, a translation of the Mathematics Anxiety Rating Scale for Elementary School Students (MARS-E) was completed by 154 fifth and sixth-grade students, and factor scores were computed.
Abstract: This study focused on the incidence, correlates, and effects of mathematics anxiety among Japanese children. A translation of the Mathematics Anxiety Rating Scale for Elementary School Students (MARS-E) was completed by 154 fifth- and sixth-grade students. Factor analysis identified four dimensions of math anxiety, and factor scores were computed. Measures of correlation revealed a number of significant relationships between MARS-E scores and classroom achievement grades, gender, and class. Comparison of mean factor scores for each dimension and the three independent variables revealed significant differences among low, middle, and high achievers. Findings are compared with those of a study of American children, and methodological issues are discussed.
TL;DR: One strategy for evaluating result replicability is to employ a "boots" test as discussed by the authors, which does not inform the researcher regarding the likelihood that the results will replicate, but does not require the researcher to be aware of the likelihood of the results being replicated.
Abstract: Conventional statistical significance tests do not inform the researcher regarding the likelihood that results will replicate. One strategy for evaluating result replicability is to employ a "boots...
TL;DR: The Myers-Briggs Type Indicator has achieved widespread use in organizational settings, despite the fact that research on its dimensionality has been scarce and that some studies have questioned the validity of its four-factor scoring system as mentioned in this paper.
Abstract: The Myers-Briggs Type Indicator (MBTI) has achieved widespread use in organizational settings, despite the fact that research on its dimensionality has been scarce and that some studies have questioned the validity of its four-factor scoring system. Confirmatory factor analyses of 94 MBTI items (N = 1,091) were performed, the results of which provided qualified support for the four-factor model; the qualifications arose from the fact that model-fit indexes for even the best-fitting models were considerably below their maximum desirable values. In an attempt to identify areas in which model fit could be improved, exploratory factor analyses were also conducted; these analyses strongly supported a four-factor view of the MBTI and indicated several additional factor loadings that could be freed to improve model fit.
TL;DR: In this paper, the authors describe the development of three domain-specific measures of individual differences in feedback propensities, and identify the primary dimensions, psychespecificity, and psychophysics.
Abstract: This article describes the development of three domain-specific measures of individual differences in feedback propensities. In a series of studies, the authors identify the primary dimensions, psy...
TL;DR: The authors used generalizability theory with a sample of 102 graduate students and showed that connotatively consistent and reversed connotative inconsistent items were not fully equivalent, and proposed the new term "connotatively inconsistent items" which has a broader base for generalization.
Abstract: This article redefines what has been referred to as "negatively worded items" in the literature. The new term—"connotatively inconsistent items"—is more nearly accurate because it has a broader base for generalization. Using generalizability theory with a sample of 102 graduate students, the study showed that connotatively consistent and reversed connotatively inconsistent items were not fully equivalent.
Abstract: Covariance modeling techniques were used to test the assumption of group invariance of a computer anxiety scale by comparing results from two potentially different populations: college students and members of a law enforcement agency. Results provide strong support for the hypothesis that the validity of the computer anxiety construct could be generalized to different groups.
TL;DR: The authors applied covariance structure modeling to the study of psychometric g in relation to collegiate academic performance and socioeconomic status and found that psychometric G accounted for a substantial proportion of the variance in academic performance.
Abstract: The study of psychometric g, a focus of research in both educational and occupational settings, recently has been raised to new levels by advances in covariance structure modeling, which allows relations among observed and latent variables to be investigated. This type of modeling was applied to the study of psychometric g in relation to collegiate academic performance and socioeconomic status. The results showed that psychometric g accounted for a substantial proportion of the variance in academic performance. The addition of the socioeconomic latent variable into any of the covariance structure models did not improve fit.
TL;DR: In this paper, the authors made comparisons of aptitude factor structures in large, nationally representative samples of young Black, White, and Hispanic men and women who took the Armed Services Vocational Aptitude Battery (ASVAB).
Abstract: Comparisons of aptitude factor structures were made in large, nationally representative samples of young Black, White, and Hispanic men and women who took the Armed Services Vocational Aptitude Battery (ASVAB). Confirmatory factor techniques imposing statistical constraints tested if the loadings of the tests were the same for sex and ethnic groups on a hierarchical general cognitive ability (g) factor and three residualized lower order factors, which included Verbal/Math, Technical Knowledge, and Speed. Tests that reflected content common in high school curricula, such as verbal and math, showed little sex or ethnic differences. The most outstanding difference was found between men and women on the loading of the Electronics Information test on g and on the lower order Technical Knowledge factor. In both cases, the loading was almost zero for females and moderate for males. Patterns in high school course enrollment by sex suggested the reason for the differences. Women are much less likely to enroll in c...
TL;DR: POWPAL as mentioned in this paper is a simple, interactive computer program that computes effect sizes and power estimates from summary statistics, and provides estimates of future sample sizes needed for various combinations of power, significance levels, and effect sizes.
Abstract: POWPAL is a simple, interactive computer program that computes effect sizes and power estimates from summary statistics. Additionally, the program provides estimates of future sample sizes needed for various combinations of power, significance levels, and effect sizes.
TL;DR: In this article, a 20-item Organizational Diversity Inventory (ODI) is described, which is based on a battery of 200 statements dealing with various aspects of diversity was evaluated by a diverse group of 40 middle managers.
Abstract: A newly developed 20-item Organizational Diversity Inventory (ODI) is described. A battery of 200 statements dealing with various aspects of diversity was evaluated as to the statements' relevancy by a diverse group of 40 middle managers. Thirty-five items were selected for further exploration. Four hundred and fifty managerial employees from 27 organizations completed this 35-item inventory. Relying on half of the sample, a factor analytic solution was generated that comprised 20 items and five factors. The second half of the sample was used to cross-validate these results. A structural equations confirmatory factor analysis (LISREL 8.03) also supported these five factors. The resulting factors were labeled (a) Existence of Discrimination, (b) Discrimination Against Specific Groups, (c) Managing Diversity, (d) Actions Regarding Minorities, and (e) Attitudes Toward Religion.
TL;DR: In this article, the authors analyzed various characteristics of report cards to determine their influence on the validity of the grading information and the meaningfulness of the information typically found on report cards.
Abstract: Schools rely heavily on report cards when communicating with parents about students' performance in school But how do the various characteristics of report cards contribute to or detract from the validity of the grading information? Samples of reporting forms gathered from elementary, middle, and high schools throughout Wisconsin were content analyzed to determine their influence on validity The results are presented for the kindergarten, elementary, middle school, and high school samples, and trends across grade levels are also examined The relationship of the findings to the meaningfulness of the information typically found on report cards is discussed, and suggestions are offered for improving the validity of the inferences parents make from report cards
TL;DR: The dimensionality of a new multidimensional benefits satisfaction questionnaire called the Attitudes Toward Benefits Scale was examined using a principal components analysis with an orthogonal rotation, and the parallel analysis criterion was used to determine the number of components to retain this paper.
Abstract: The dimensionality of a new multidimensional benefits satisfaction questionnaire called the Attitudes Toward Benefits Scale was examined using a principal components analysis with an orthogonal rotation, and the parallel analysis criterion was used to determine the number of components to retain. It was found that the items loaded on three separate components, as had been hypothesized, and that only one of the components, satisfaction with benefits, was statistically significantly correlated with the benefit subscale of the Pay Satisfaction Questionnaire.