Showing papers in "Educational and Psychological Measurement in 1996"
TL;DR: In this paper, the authors define statistical significance as whether a research result is due to chance or sampling variability; practical significance is concerned with whether the result is useful in the real world.
Abstract: Statistical significance is concerned with whether a research result is due to chance or sampling variability; practical significance is concerned with whether the result is useful in the real worl...
1,584 citations
TL;DR: In this article, the authors explored expectations for reporting factor analysis results as part of construct validation in the context of emerging views of measurement validity, and proposed a short computer program for conducting parallel analysis.
Abstract: Expectations for reporting factor analysis results as part of construct validation are explored in the context of emerging views of measurement validity. Desired practices are discussed regarding both exploratory factor analysis (e.g., principal components analysis) and confirmatory factor analysis (e.g., LISREL and EQS factor analyses). A short computer program for conducting parallel analysis is appended.
944 citations
TL;DR: This paper used data from 3,085 respondents from metropolitan Sydney, Australia were used to refine the 14-item Hong Psychological Reactance Scale, which consistently showed three problematic items, which led to a refined 11-item scale.
Abstract: Data from 3,085 respondents from metropolitan Sydney, Australia were used to refine the 14-item Hong Psychological Reactance Scale. Results consistently showed three problematic items, which led to a refined 11-item scale. A series of factor analyses yielded a distinctive four-factor structure that was perfectly congruent among the total sample and the four subsamples: males, females, university students, and nonuniversity students. Cronbach alpha, split-halfs, and theta coefficients were found to be very satisfactory for the refined scale. The convergent and discriminant validities for the 1 1-item scale were tested with various personality constructs.
413 citations
TL;DR: This paper developed a student rating scale that assesses the degree to which students identify with or disidentify from school and whether there are gender or racial group differences in identification. But the identification with school questionnaire was tested on a sample of 3,539 eighth-grade pupils.
Abstract: This article describes the development of a student rating scale that assesses the degree to which students identify with or disidentify from school and whether there are gender or racial group differences in identification. The Identification with School Questionnaire was tested on a sample of 3,539 eighth-grade pupils. Confirmatory factor analyses of the data provided evidence that a unidimensional scale may better reflect the degree to which students identify with school than do separate measures of belonging and valuing. Analyses of variance of the scale indicated that female students exhibited higher degrees of identification with school than did their male peers, African American students were more identified with school than were White students, and White male youngsters had the lowest levels of identification with school.
225 citations
TL;DR: The authors examines the statistical correction for attenuation and the controversies surrounding the procedure and concludes with arguments pertaining to the use of the correction formula, and it attempts to sharpen the focus of issues that have led to differences of opinion about its meaning and purpose.
Abstract: This article examines the statistical correction for attenuation and the controversies surrounding the procedure. Although originally developed for test construction purposes, the correction for attenuation is also used in meta-analysis and assessments of validity generalization. Since Spearman's classic article in 1904, correct use and interpretation of the correction for attenuation has been debated. The logic of the double and single correction formulae is discussed as well as the correction producing validity coefficients greater than 1.00. Three types of misapplications and misinterpretations of the correction in published literature are presented. The article concludes with arguments pertaining to the use of the correction formula, and it attempts to sharpen the focus of issues that have led to differences of opinion about its meaning and purpose.
221 citations
TL;DR: In this paper, the discriminant validity of the Meyer and Allen scales of organizational commitment and their relationships to commitment of different foci such as work involvement, job involvement, career commitment, and the Protestant work ethic was examined.
Abstract: The purpose of this article is to examine the discriminant validity of the Meyer and Allen scales of organizational commitment and their relationships to commitment of different foci such as work involvement, job involvement, career commitment, and the Protestant work ethic. A total of 238 usable questionnaires were returned from nurses from two hospitals in western Canada. Results of confirmatory factor analysis (LISREL) and correlation analysis showed a good discriminant validity among the seven commitment foci; the three organizational commitment foci, and the four work commitment foci. The data also showed that the fit measures of the Meyer and Allen scales were better than those of the widespread Organizational Commitment Questionnaire (OCQ).
157 citations
TL;DR: In this article, the authors developed a conceptually and methodologically sound measure of work to family conflict (W -* FC), which supported a theoretically grounded measure of W -e FC.
Abstract: Although research on work-family conflict has grown, empirical investigations have frequently been a theoretical and lacking in measurement rigor. The objective of this study was to develop a conceptually and methodologically sound measure of work to family conflict (W -* FC). Data were collected from three independent sources, and both exploratory and confirmatory factor analyses were conducted, which supported a theoretically grounded measure of W -e FC. Confirmatory factor analysis indicated that a three-factor model, including time-, strain-, and behavior-based dimensions of W -> FC, was superior to both a two-factor and a single-factor (null) model.
147 citations
TL;DR: The authors used Monte Carlo methods to investigate the accuracy of formulas presented by Kendall for estimating Pearson's r from X and by Pearson for estimating R from R. The largest asymptotic difference between a converted and actual r was -.005.
Abstract: This article used Monte Carlo methods to investigate the accuracy of formulas presented by Kendall for estimating Pearson's r from X and by Pearson for estimating Pearson's r from rs. Results indicated that the formula for approximating r from r is somewhat more accurate than the formula for approximating r from rs. However, both formulas were found to be quite accurate. The largest asymptotic difference between a converted and actual r was -.005. In addition, the standard errors of the transformed X and rs coefficients were compared to the standard error of r, and the empirically derived percentage increase in standard error was compared to theoretically derived estimates. Implications of these findings for meta-analyses of correlations are discussed.
119 citations
TL;DR: In this article, the Meyer and Allen's Affective and Continuance Commitment Scales using confirmatory factor analysis with LISREL 7 provided strong support across multiple diagnostics for the existence of a reverse coding method factor defined by the six negatively worded items in the scales.
Abstract: Examination of Meyer and Allen's Affective and Continuance Commitment Scales using confirmatory factor analysis with LISREL 7 provided strong support across multiple diagnostics for the existence of a reverse coding method factor defined by the six negatively worded items in the scales.
114 citations
TL;DR: A linear regression approach providing objectivity while producing the same results as the visual scree test is presented, based on the standard error scree for a set of points in the plot of eigenvalues.
Abstract: Many methods have been suggested for determining the number of factors in factor analysis solutions, most of them trying to match visual solutions. Presented here is a linear regression approach providing objectivity while producing the same results as the visual scree test. The standard error scree is based on the standard error of estimate for a set of points in the plot of eigenvalues. The number of factors to retain corresponds to the point where the standard error exceeds 1/m, where m is the numbers of variables in the analysis. Experience with the method to date indicates that the results are consistent with visual solutions.
110 citations
TL;DR: This article examined Kolb's Experiential Learning Theory through a psychometric analysis and revision of Honey and Mumford's Learning Style Questionnaire (LSQ) using a sample of 185 middle/senior managers in the UK and Ireland.
Abstract: The study examined Kolb's Experiential Learning Theory through a psychometric analysis and revision of Honey and Mumford's Learning Style Questionnaire (LSQ) using a sample of 185 middle/senior managers in the UK and Eire. Kolb's theoretical two bipolar-dimension orthogonal structure emerges, although it is made up of pairs that are not Kolb's configurative opposites but ones suggested by other scholars. The authors' principal finding is that no single measure can bear the weight of all of Kolb's constructs. The revision of the LSQ supports two psychometrically sound, orthogonally related, bipolar-style dimensions. Indirectly, this construct can give rise to four descriptions, based on their quadrants, that Kolb usefully treats as types. Unrelated to any measure is a four-stage process that the authors define as an ideal dynamic schema.
TL;DR: In this paper, the consequences of underextracting factors and components within and between the methods of maximum likelihood factor analysis (MLFA) and principal components analysis (PCA) were examined.
Abstract: The consequences of underextracting factors and components within and between the methods of maximum likelihood factor analysis (MLFA) and principal components analysis (PCA) were examined. Computer-simulated data sets represented a range of pattern structures. Manipulated conditions included component (factor) structure coefficients (aij = .8, .6, and .4), sample size (N = 75, 150, 225, and 450), and variable-to-component (factor) ratio (p:m = 6:1 and 4:1). The principal components score and the Anderson and Rubin factor score estimate were calculated for both the correct patterns and the incorrect (underextracted) patterns. In Study 1, underextraction led to substantial degradation of scores within both methods, but the component score degraded less rapidly. Score degradation was related to the number of original components (factors). In Study 2, between-method comparisons indicated very high similarity for baseline score patterns, but dissimilarity occurred with underextraction.
TL;DR: In this article, the latent structure of a 16-item mentoring function instrument was analyzed with the responses of 568 full-time employees and the invariance tests suggested that the structure was invariant across sex groups.
Abstract: The latent structure of a 16-item mentoring function instrument was analyzed with the responses of 568 full-time employees. Features of the analyses included (a) assessment of the items' distributional properties (i.e., skewness and kurtosis); (b) assessment of the factor structure using the Satorra-Bentler scaled test statistic; and (c) evaluation of the instrument's invariance across sex. Confirmatory factor analyses using the scaled chi-square supported a two-factor oblique model that consisted of psychosocial and career-related mentoring functions. The invariance tests suggested that the structure was invariant across sex groups.
TL;DR: In this article, an effort to develop an instrument to assess teacher efficacy for enhancing student social relations (TES) was described, and the psychometric properties of the teacher efficacy scale produced by Gibson and Dembo were examined after translation to Hebrew and administration to 218 Israeli teachers.
Abstract: This report describes an effort to develop an instrument to assess teacher efficacy for enhancing student social relations (TES). In addition, the psychometric properties of the teacher efficacy scale produced by Gibson and Dembo were examined after translation to Hebrew and administration to 218 Israeli teachers. Results indicated that the TES subscale is independent of the two original subscales and demonstrates good internal and test-retest reliability. Also, the factorial structure of the original teacher efficacy scale was replicated with the Israeli sample, and reliability levels were generally adequate. Some problems with one of the original subscales are noted.
TL;DR: A short measure of trait and state negative and positive affect (Chinese Affect Scale) was developed for Chinese-speaking people as discussed by the authors, which had reasonable internal and retest reliabilities and high convergent and discriminant validity, with NA correlating with neuroticism, negative self-appraisal, stress, and pessimism.
Abstract: A short measure of trait and state negative and positive affect (Chinese Affect Scale) was developed for Chinese-speaking people. From indigenously derived affect markers the characteristic two-factor structure of affect was derived in both a student and an adult sample in Hong Kong. Scores from the resulting negative (NA) and positive (PA) affect scales had reasonable internal and retest reliabilities and high convergent and discriminant validity, with NA correlating with neuroticism, negative self-appraisal, stress, and pessimism; and PA correlating with extraversion, positive self-appraisal, optimism, and self-esteem. Both state NA and PA versions of the scales were shown to be sensitive to episodal mood changes over life events (daily log), to laboratory mood induction, and to movie themes for moviegoers in a field setting. The psychometric properties of the CAS were replicable in the student and adult samples.
TL;DR: In this article, the unidimensionality of the Life Orientation Test (LOT) has been challenged, and the authors argue that the uni-dimensional interpretation of the test should be reconsidered.
Abstract: This study disputed the unidimensionality of the Life Orientation Test (LOT). The eight-item LOT was administered to undergraduates in its original mixed form and in consistent optimism and consistent pessimism forms. Within the original form, two factors of optimism and pessimism were identified. However, when items had the same connotations, only one factor emerged. Researchers should acknowledge the two-factor structure of the LOT or reconstruct the items to maintain the unidimensional interpretation of the test.
TL;DR: In this paper, confirmatory factor analyses revealed that a two-factor oblique model provided the best fit to the data and was also found to be invariant across sex, but given that the means differed between males and females, the use of separate norms should be continued.
Abstract: The Test Anxiety Inventory (TAI) is one of the most widely used surveys to assess overall test anxiety. In addition, scores are available for two components of anxiety: emotionality and worry. However, there is controversy about the nature of these two components. The authors of the TAI suggest that these two factors are orthogonal, whereas others suggest that they are oblique. Working with a sample of 335 male and 389 female school-leaving students from Canada, confirmatory factor analyses revealed that a two-factor oblique model provided the best fit to the data. The two-factor oblique model was also found to be invariant across sex. However, given that the means differed between males and females, the use of separate norms should be continued.
TL;DR: In this paper, the development and validation of an instrument to assess attitudes toward cultural diversity and cultural pluralism is described, which consists of 60 questions and was complete and complete by the authors.
Abstract: The development and validation of an instrument to assess attitudes toward cultural diversity and cultural pluralism are described. The initial instrument consisted of 60 questions and was complete...
TL;DR: In this paper, a procedure that involves iterative use of univariable optimal discriminant analysis (UniODA) in order to construct a classification tree model for discriminating observations from different classes is described.
Abstract: A procedure is described that involves iterative use of univariable optimal discriminant analysis (UniODA) in order to construct a classification tree model for discriminating observations from dif...
TL;DR: This paper investigated the structural equivalence of a measure of cross-cultural adjustment in culturally dissimilar samples using confirmatory factor analysis and found that the hypothesized factor structure was invariant and had a relatively good fit in both samples.
Abstract: This study investigated the structural equivalence of a measure of cross-cultural adjustment in culturally dissimilar samples using confirmatory factor analysis. The two samples consisted of 155 international students studying at a midwestern American university and 153 American expatriate managers on assignment in either Taiwan or Belgium. Results indicated that the hypothesized factor structure was invariant and had a relatively good fit in both samples. The implication of these results is that adjustment may have similar dimensions across cultures. If substantiated through future studies, these results would enable researchers to have more confidence in cross-cultural comparisons of mean levels of adjustment or correlations of adjustment with external variables.
TL;DR: In this article, a set of algebraically equivalent rater agreement indexes are presented, which are intended to provide the educational and psychological researcher and practitioner with a practical means of establishing a measure of Rater agreement.
Abstract: Under the heading of reliability, most textbooks refer to classical reliability indexes as appropriate measures for determining interrater agreement. It is argued that interrater agreement is a psychometric property that is theoretically different from classical reliability. Interrater agreement indexes measure the degree to which two or more raters agree on the observation of one or more behaviors on one or more subjects and are not theoretically related to classical test theory. A detailed set of formulas is presented to illustrate a set of algebraically equivalent rater agreement indexes that are intended to provide the educational and psychological researcher and practitioner with a practical means of establishing a measure of rater agreement. The formulas are illustrated with a data set. The formulas can be used for dichotomous and continuous data for two or more raters, on one or more subjects, on one or more behaviors. These rater agreement indexes are useful with performance assessments such as observations, porfolios, performance evaluations, essay writing evaluations, authentic assessments, and so on, where multiple facets impact rater agreement.
TL;DR: The authors examined three aspects of the WLCS in three separate samples: (a) factor structure, (b) criterion-related validity, and (c) convergent and discriminant validity.
Abstract: P. E. Spector developed the Work Locus of Control Scale (WLCS) to be used in work settings. The present study examines three aspects of the WLCS in three separate samples: (a) factor structure, (b) criterion-related validity, and (c) convergent and discriminant validity. The preponderance of evidence suggests that the WLCS consists of two factors: internal and external. In contrast to using an overall WLCS score as described in Spector's development of the measure, users of the scale are advised to compute separate subscales.
TL;DR: This article examined the extent to which the resolution of the Eriksonian final stage-related crisis of ego integrity versus despair is predicted by resolution of earlier stagerelated crises and by non-Eriksonian personality constructs.
Abstract: This study examined the extent to which the resolution of the Eriksonian final stage-related crisis of ego integrity versus despair is predicted by the resolution of earlier stage-related crises and by non-Eriksonian personality constructs. Subjects were administered an Eriksonian life stage measure called the Inventory of Psychosocial Balance, the Rosenberg Self-Esteem Scale, the Purpose-in-Life Scale, and the Self-Realization Scale. A series of alternative and hierarchically nested regression models was run to assess the direct effects of all preceding Eriksonian life stages and the non-Eriksonian personality measures on the final stage of ego integrity. The results were consistent with Eriksonian theory, which states that personality development is a continuous process in which psychosocial growth during earlier phases of life is a prerequisite for the resolution of later developmental conflicts.
TL;DR: In this article, a study of about 200 undergraduates examined the construct validity and classification stability over an academic semester of the revised Learning Style Inventory (LSI-1985) over a 10-week period.
Abstract: This study of about 200 undergraduates examined the construct validity and classification stability over an academic semester of the revised Learning Style Inventory (LSI-1985).The psychometric and factor analyses at two points in time revealed limited support for the construct validity of the LSI-1985. One major problem was the cross-loading of Concrete Experience (CE) items on the two- and four-factor solutions. This problem also carries over to the psychometric (e.g., item-total correlations) analyses. When stability of the four learning styles was examined over a 10-week period, between 42.8% and 50% of subjects remained in the same category. Overall, the revised LSI has not resolved the psychometric problems raised against the original version.
TL;DR: In this paper, the authors set out to test a series of hypotheses on the relationship between class size and the evaluation of university teaching quality through student opinions using information obtained from a sample of 1,157 classes, using a shortened version of the Complutense University Teachers Evaluation Questionnaire.
Abstract: The authors set out to test a series of hypotheses on the relationship between class size and the evaluation of university teaching quality through student opinions The information obtained from a sample of 1,157 classes, using a shortened version of the Complutense University Teachers Evaluation Questionnaire, offers empirical support for the following affirmations: (a) class size does have some impact on teaching ratings; (b) this relationship differs somewhat as a function of the two dimensions operationalized by the questionnaire; and (c) the effect sizes are quite small With regard to the controversy over the shape of the relationship, this seems to depend to a large extent on the range of class sizes, and thus some of the hypotheses proposed so far may, in fact, be less incompatible than has been supposed previously
TL;DR: The Attitudes Toward Diversity Scale (ATDS) as discussed by the authors was developed to measure the construct of attitudes toward diversity, which consists of 10 items representing each of three domains: attitudes toward divers...
Abstract: The Attitudes Toward Diversity scale (ATDS) was developed to measure the construct of attitudes toward diversity. It consists of 10 items representing each of three domains: attitudes toward divers...
TL;DR: In this paper, a multidimensional measure of pay satisfaction (the Pay Satisfaction Questionnaire; PSQ) was used to assess the discriminant validity between scores on a measure of distributive justice and the PSQ using confirmatory factor analysis.
Abstract: Using a multidimensional measure of pay satisfaction (the Pay Satisfaction Questionnaire; PSQ), this study assessed the discriminant validity between scores on a measure of distributive justice and the PSQ using confirmatory factor analysis. Data collected from a diverse sample of employees indicated that the items from both the pay satisfaction scale and the distributive justice scale loaded on the hypothesized dimensions. Various goodness-of-fit measures confirmed the discriminant validity of the two constructs.
TL;DR: The Chinese version of the depression self-rating scale (DSRS) developed by Birleson was administered to 3,786 Chinese early adolescents studying at Grades 5 to 8 in Hong Kong.
Abstract: The Chinese version of the Depression Self-Rating Scale (DSRS) developed by Birleson was administered to 3,786 Chinese early adolescents studying at Grades 5 to 8 in Hong Kong. The DSRS data were found to have adequate internal consistency and item-total correlations. Factor analysis of the scale revealed two components. The findings suggest that the Chinese version of the DSRS possesses acceptable psychometric properties and appears to be a brief and economical instrument to be used for the purposes of research and documenting change. However, the mean for the sample was found to be higher than those obtained in previous studies. Implications for future research are discussed.
TL;DR: An abbreviated, 13-item version of the BAPC was derived in this paper, and two studies were designed to examine internal consistency, test-retest reliability, and test reliability.
Abstract: An abbreviated, 13-item version of the Behavioral Attributes of Psychosocial Competence Scale (BAPC) was derived. Two studies were designed to examine internal consistency, test-retest reliability,...
TL;DR: In this article, the characterization and circumvention of Simpson's paradox in designs involving two ordered variables is discussed and the recommended procedure using a two-sample applicatio-tion is described.
Abstract: This article addresses the characterization and circumvention of Simpson's paradox in designs involving two ordered variables and illustrates the recommended procedure using a two-sample applicatio...