scispace - formally typeset
Search or ask a question

Showing papers in "Educational and Psychological Measurement in 2003"


Journal ArticleDOI
TL;DR: In this paper, the authors identify three components of organizational attraction: attractiveness, prestige, and behavioral intentions, and model their effects on organizational pursuit using Fishbein and Ajzen's theory of reasoned action.
Abstract: Organizational attractionmeasures are commonly used as surrogate assessments of organizational pursuit. Despite the range in content often encompassed by such instruments, no research has systematically examined the assumptions underlying their use. The authors address this issue by empirically distinguishing items assessing attractiveness, prestige, and behavioral intentions and by modeling their effects on organization pursuit. Undergraduates (N= 305) were randomly assigned to recruitment literature from one of five wellknown companies and were asked to respond to a series of items commonly used in past research. Analyses of the itemresponses suggested that three components of organizational attraction can be reliably distinguished and that their relation to organization-pursuit behavior corresponds to Fishbein and Ajzen’s theory of reasoned action.

603 citations


Journal ArticleDOI
TL;DR: The Personal Style Inventory (PSI) as discussed by the authors was developed to assess individuals' levels of sociotropy and autonomy, two personality characteristics considered to be associated with increased vulnerability to cyber-attacks.
Abstract: The Personal Style Inventory (PSI) was developed to assess individuals’ levels of sociotropy and autonomy, two personality characteristics considered to be associated with increased vulnerability t...

375 citations


Journal ArticleDOI
TL;DR: In this paper, the ability of emotional intelligence to predict academic achievement was examined in a sample of undergraduate psychology students, using year-end grades as the criterion, and the incremental predictive validity of each of these three domains was assessed.
Abstract: Some authors have claimed that emotional intelligence predicts success at work, at school, and in relationships, as well as or better than IQ. Little research exists to support or refute this claim at present. In this study, the ability of emotional intelligence to predict academic achievement was examined in a sample of undergraduate psychology students, using year-end grades as the criterion. The predictive validity of emotional intelligence was compared with the predictive validity of traditional cognitive abilities and the Big Five dimensions of personality. In addition, the incremental predictive validity of each of these three domains was assessed. In this setting, only some measures of Emotional Intelligence predicted academic success, and none of these measures showed incremental predictive validity for academic success over and above cognitive and personality variables. It may be that the overlap between many emotional intelligence measures and traditional measures of intelligence and personality...

371 citations


Journal ArticleDOI
TL;DR: In this article, a simulation study was conducted to examine the effect of item parceling on goodness-of-fit indices at different levels of sample size, number of indicators per factor, factor structure/pattern coefficients, interfactor correlations, and item-level data distribution.
Abstract: A simulation study was conducted to examine the effect of item parceling on goodness-of-fit indices at different levels of sample size, number of indicators per factor, factor structure/pattern coefficients, interfactor correlations, and item-level data distribution. Results revealed that the use of item parcels yielded more nonconverged solutions and Heywood cases than individual items. The likelihood of nonconverged solutions and Heywood cases increased as the number of indicators per factor (more items per parcel) decreased. Meanwhile, parcel solutions as compared with item solutions resulted in better fit as measured by the chi-square to degrees-of-freedom ratio, Goodness-of-Fit Index (GFI), Expected Cross-Validation Index (ECVI), and root mean square error of approximation (RMSEA), as well as two incremental fit indices, the Non-Normed Fit Index (NNFI) and Comparative Fit Index (CFI). The same pattern of results was found with data that varied in terms of skewness and kurtosis at the item level. Howe...

263 citations


Journal ArticleDOI
TL;DR: This paper examined the psychometric strengths and limitations of a widely used measure of ethnic identity development, the Multigroup Ethnic Identity Measure (MEIM), and provided a review of 12 studies.
Abstract: This article examines the psychometric strengths and limitations of a widely used measure of ethnic identity development: the Multigroup Ethnic Identity Measure (MEIM). A review of 12 studies incor...

233 citations


Journal ArticleDOI
TL;DR: In this paper, a comparative study of two post hoc cellwise tests in 3 X 4 contingency tables under the independence and homogeneity models is presented. But the results of the study are limited.
Abstract: MacDonald and Gardner reported the results of a comparative study of two post hoc cellwise tests in 3 X 4 contingency tables under the independence and homogeneity models. Based on their results, t...

220 citations


Journal ArticleDOI
TL;DR: A psychometrically sound instrument was designed to measure adolescents' attitudes toward school, attitudes toward teachers, goal-valuation, motivation, and general academic self-perceptions that could be used to explore the underachievement of academically able secondary school students as discussed by the authors.
Abstract: The purpose of this study was to design a psychometrically sound instrument to measure adolescents' attitudes toward school, attitudes toward teachers, goal-valuation, motivation, and general academic self-perceptions that could be used to explore the underachievement of academically able secondary school students. The final School Attitude Assessment Survey-Revised (SAAS-R) instrument consisted of 35 questions, each of which being an indicator of one of the five factors. The final model exhibited reasonable fit, X2(550) = 1,581.7, Comparative Fit Index = .91, Tucker Lewis Index = .92, root mean square error of approximation = .059, standardized root mean squared residual = .057. The scores in this study showed internal consistency reliability coefficient of at least .85 on each of the five factors. In addition, four of the five factors of the SAAS-R appear to differentiate gifted achievers from gifted underachievers. It is the authors' hope that the SAAS-R will allow researchers to more fully understand ...

190 citations


Journal ArticleDOI
TL;DR: The results of the simulation study suggest that the maximum likelihood multilevel approach is in general superior to the fixed-effects approaches, unless only a small number of studies is available.
Abstract: In this article, the authors compare the multilevel meta-analysis approach with the more traditional meta-analytical approaches. After a description and comparison of the under-lying models and some of the major techniques, the results of the multilevel approach are compared with those of the traditional approaches, using a simulation study. The results of the simulation study suggest that the maximum likelihood multilevel approach is in general superior to the fixed-effects approaches, unless only a small number of studies is available. For models without moderators, the results of the multilevel approach, however, are not substantially different from the results of the traditional random-effects approaches.

183 citations


Journal ArticleDOI
TL;DR: The authors compared a newly developed measure of self-control as outlined in the General Theory of Crime, the Retrospective Behavioral Self-Control Scale (RBS), with the most widespread measure of Self-control.
Abstract: The present study contrasts a newly developed measure of self-control as outlined in the General Theory of Crime, the Retrospective Behavioral Self-Control scale (RBS), with the most widespread mea...

149 citations


Journal ArticleDOI
TL;DR: This article developed a model for predicting statistics achievement in a quantitative-based educational research course at a southeastern university and found that statistics anxiety and achievement expectation played a central role in the model, mediating the relationship between statistics achievement and the following variables.
Abstract: The purpose of the present investigation was to develop a model for predicting statistics achievement. Participants were 130 graduate students enrolled in a quantitative-based educational research course at a southeastern university. A path analysis revealed that statistics anxiety and achievement expectation played a central role in the model, mediating the relationship between statistics achievement and the following variables: research anxiety, study habits, course load, and the number of statistics courses taken. Findings are interpreted with respect to Onwuegbuzie, Bailey, and Daley’s Anxiety-Expectation Mediation model of foreign language learning.

137 citations


Journal ArticleDOI
TL;DR: A summary of 23 separate exploratory factor analytic validation studies of the Bem Sex Role Inventory (BSRI) conducted during the 25 years since its publication is reported in this paper.
Abstract: This study reports on a summary of 23 separate exploratory factor analytic validation studies of the Bem Sex Role Inventory (BSRI) conducted during the 25 years since its publication. The ERIC and PsycLit databases were used to locate those validation studies. Number of items factor analyzed, sample characteristics, extraction and rotation methods used, names of factors, and factor structures reported in each of 23 studies were tabulated and analyzed. Two to four factors were retained in most of the studies, with masculine factors showing more factorially complex tendencies. Findings from this study suggest that masculinity and femininity have not been adequately operationalized in the BSRI and that the true structure of masculinity/femininity may be more complex than it appears when measured by the BSRI, which suggests that the BSRI may not capture the complex nature of masculinity/femininity. Implications of the results and the potential usefulness of the BSRI are discussed.

Journal ArticleDOI
TL;DR: In this paper, the validity of the revised Math Anxiety Rating Scale (MARS-R) was assessed using a large sample (N = 815), and the construct validity of MARS-R was assessed.
Abstract: Psychometric properties of recently developed measures of mathematics anxiety have not adequately been studied. Using a large sample (N = 815), the construct validity of the revised Math Anxiety Rating Scale (MARS-R) was assessed. Participants were randomly assigned to a validation or a replication sample. In the validation sample, confirmatory analyses revealed that the original model fits data poorly. The instrument was modified, resulting in the elimination of 12 items, and the fit of the two-factor model was improved considerably. The revised measure was applied to the replication sample, with comparable fit indices obtained and excellent generalization across male and female participants. Internal consistency reliability coefficients of the revised measure were strong, and good convergent and divergent validity was shown with the original MARS-R and other anxiety-related measures. The revised measure may represent a more parsimonious and valid measurement approach for assessing mathematics anxiety.

Journal ArticleDOI
TL;DR: In this article, the quality of multilevel model parameter estimates and standard errors as a function of varying magnitudes of correlation among level 1 predictors and model characteristics was investigated and the results showed that with multicollinearity presented at Level 1 of a two-level mixed-effects linear model, the fixed-effect parameter estimates produce relatively unbiased values; however, the variance and covariance component estimates produce downwardly biased values except for Level 1 variance (< 5%).
Abstract: This study investigates the quality of multilevel model parameter estimates and standard errors as a function of varying magnitudes of correlation among Level 1 predictors and model characteristics. The results of the study showthat with multicollinearity presented at Level 1 of a two-level mixed-effects linear model, the fixed-effect parameter estimates produce relatively unbiased values; however, the variance and covariance component estimates produce downwardly biased values except for Level 1 variance (< 5%). The standard errors associated with the parameter estimates are also biased under varied magnitudes of Level 1 predictor correlation.

Journal ArticleDOI
TL;DR: In this article, the authors discussed the assumptions underlying the use of rating scales, and described the use information available within the context of Rasch measurement that may be useful for optimizing rating scales.
Abstract: This article (a) discusses the assumptions underlying the use of rating scales, (b) describes the use of information available within the context of Rasch measurement that may be useful for optimizing rating scales, and (c) demonstrates the process in two studies. Participants in the first study were 330 fourth- and fifth-grade students. Participants provided responses to the Index of Self-Efficacy for Writing. Based on category counts, average measures, thresholds and category fit statistics, the responses on the original 10-point scale were better represented by a 4-point scale. The modified 4-point scale was given to a replication sample of 668 fourth- and fifth-grade students. The rating scale structure was found to be congruent with the results from the first study. In addition, the item fit statistics and item hierarchy indicated the writing self-efficacy construct to be stable across the two samples. Combined, these results provide evidence for the generalizability of the findings and hence utility...

Journal ArticleDOI
TL;DR: This article examined the cross-age comparability of the widely used Rosenberg Self-Esteem Scale (RSES) in a sample of adolescents and adults in families receiving Aid to Families With Dependent Children (AFDC).
Abstract: Evidence of equivalence of measures is necessary before comparisons can be made across groups. Comparability of a measure is a matter of degrees from weak to strong. Many examinations of factorial invariance examine the factor pattern coefficients but not the stronger requirement of equivalence of intercept terms. This study examined the cross-age comparability of the widely used Rosenberg Self-Esteem Scale (RSES) in a sample of adolescents and adults in families receiving Aid to Families With Dependent Children (AFDC). Multiple-group mean and covariance structures (MACS) analysis was used to demonstrate strong invariance of the measures and make comparisons across age groups. Similarities of means in the RSES were found across groups.

Journal ArticleDOI
TL;DR: In this article, the authors investigated potentially biased scale items on the Center for Epidemiologic Studies Depression (CES-D) using two binary methods (presence and persistence) and one ordinal method.
Abstract: The present study investigated potentially biased scale items on the Center for Epidemiologic Studies Depression (CES-D). The 20-item CES-D was scored using two binary methods (presence and persistence) and one ordinal method. Gender differential item functioning (DIF) was explored using Zumbo’s OLR method with corresponding logistic regression effect size estimator with all three scoring methods. Gender DIF was found with the CES-D item “crying” for the ordinal and presence methods of scoring. The persistence scoring method identified two DIF items (effort and hopeful); however, this scoring method appears to be of limited use due to low variability on some items. Overall, the results indicate that the scoring method has an effect on DIF; thus, DIF is a property of the item, scoring method, and purpose of the instrument.

Journal ArticleDOI
TL;DR: In this article, the authors describe the development and initial validation of scores from a survey designed to measure teachers' reported use of technology in their classrooms, based on data obtained from a sample of approximately 2,000 practicing teachers, factor analytic and correlational methods were used to obtain evidence of the validity of scores derived from responses to the instrument.
Abstract: This article describes the development and initial validation of scores from a survey designed to measure teachers’ reported use of technology in their classrooms. Based on data obtained from a sample of approximately 2,000 practicing teachers, factor analytic and correlational methods were used to obtain evidence of the validity of scores derived from responses to the instrument. In addition, analyses of Web and paper versions of the survey suggest relatively minor differences in responses, although the response rates for the paper version were substantially higher. The results were interpreted in terms of the utility of the instrument for measuring the confluence of factors that are critical for inquiry related to technology use in classrooms.

Journal ArticleDOI
TL;DR: The structure of the Purpose in Life test (PIL) was investigated using confirmatory factor analytic techniques and 17 items of the PIL demonstrated good measurement reliability for both groups of drinkers and good criterion-related validity.
Abstract: The aim of the present research was to further investigate (a) the structure of the Purpose in Life test (PIL) using confirmatory factor analytic techniques,(b) the reliability of PIL scores, and (...

Journal ArticleDOI
Xitao Fan1
TL;DR: In this article, the authors focused on the issue of measurement reliability and its attenuation on correlation between two composites and two seemingly different approaches for correcting the attenuation, and found that correlation coefficients uncorrected for measurement error are systematically biased downward.
Abstract: This study focused on the issue of measurement reliability and its attenuation on correlation between two composites and two seemingly different approaches for correcting the attenuation. As expected, correlation coefficients uncorrected for measurement error are systematically biased downward. For the data conditions examined, the two correction approaches provided not only near identical and unbiased estimates but also near identical confidence intervals for the sampling distribution of the corrected correlation coefficients. The highly comparable results from the two approaches suggest that these two approacheswork equally well for these data. It is pointed out that the confirmatory factor analysis (CFA) modeling approach may be less applicable because of more difficult data conditions at the item level in research practice. The findings point to the importance of reporting measurement reliability information in substantive research. The findings further suggest that, in research practice, correction f...

Journal ArticleDOI
TL;DR: In this paper, the authors meta-analytically synthesized this research and evaluated the systematic effects of the most common modifications on the level of the resulting cutoff scores and the degree of consensus among judges.
Abstract: The Angoff method for setting cutoff scores on examinations has been researched extensively over the past 30 years, including investigations of several procedural modifications to the method In the current study, the authors meta-analytically synthesized this research and evaluated the systematic effects of the most common modifications on the level of the resulting cutoff scores and the degree of consensus among judges The authors found the following: (a) when judges use a common definition of minimally competent test-takers, this tends to increase judgment consensus; (b) when judges discuss their estimates, this tends to result in higher cutoff scores with increased consensus; and (c) when judges view normative data, this tends to systematically lower cutoff scores In addition, the authors found an interaction effect which revealed that when judges use a common definition and later discuss their estimates, this tends to produce the highest standards on average, with the highest degree of consensus am

Journal ArticleDOI
TL;DR: This article examined the appropriate unit of theory of school climate while simultaneously conducting a psychometric analysis of a modification of the School Climate Survey for an elementary school population, and found that the average school climate within each school within each schools is found to predict a statistically significant amount of the between-school variation in children's academic achievement and cognitive functioning.
Abstract: This study examines the appropriate unit of theory of school climate while simultaneously conducting a psychometric analysis of a modification of the School Climate Survey for an elementary school population. A conceptualization of school climate as an individual-level property unique to each school participant is compared to the idea that climate is a characteristic of the school with school participants having the role of informants. These two theories are compared by testing their differing implications. The School Climate Survey is found to have a stable and identical factor structure both within and between schools, ratings by different respondent groups are found to be moderately correlated, and average school climate within each school is found to predict a statistically significant amount of the between-school variation in children’s academic achievement and cognitive functioning. Differences between individual raters within each school are not found to have a meaningful relationship to child outc...

Journal ArticleDOI
TL;DR: In this paper, the authors examined the relationship between the constructs of depression as measured by the Center for Epidemiological Studies Depression Scale (CES-D) and the revised Beck Depression Inventory (BDI) in a sample of 261 college students and found that the two scales have a moderately high correlation and a similar pattern of relationships to measures of anxiety, hostility, self-esteem, hypochondriasis, and locus of control.
Abstract: This study examines the relationships between the constructs of depression as measured by the Center for Epidemiological Studies Depression Scale (CES-D) and the revised Beck Depression Inventory (BDI) in a sample of 261 college students. The two scales were found to have a moderately high correlation and a similar pattern of relationships to measures of anxiety, hostility, self-esteem, hypochondriasis, and locus of control. However, the relations between the subscales and the factor structure of the subscales derived from both instruments suggest that the BDI and CES-D measure different aspects of depression despite sharing some symptoms. Therefore, the authors argue that the BDI and CES-D should not be used interchangeably. The results are discussed in terms of the applicability of the two measures of depression for use with nonclinical populations.

Journal ArticleDOI
TL;DR: The authors evaluated the reliability of scores from the Alcohol Use Disorders Identification Test (AUDIT) in a reliability generalization study and found that the median internal consistency reliability across 24 samples was.81, with a range of.59 to.91.
Abstract: The present study evaluated the reliability of scores from the Alcohol Use Disorders Identification Test (AUDIT) in a reliability generalization study. This reliability generalization had two primary goals: (a) to characterize the typical reliability of scores for the AUDIT, and (b) to examine factors that may be related to the reliability of AUDIT scores. The median internal consistency reliability across 24 samples was .81, with a range of .59 to .91. Results suggest that the AUDIT is capable of generating generally reliable scores across some varied sample conditions. After controlling for score variability, no sample characteristic was a statistically significantly predictor of score reliability and effects were small. Only 17 of 104 empirical journal articles contained adequate psychometric information to be included in the present study.

Journal ArticleDOI
TL;DR: In this article, the authors define an approximate confidence interval for effect size in correlated (repeated measures) groups designs and show that their method is much more accurate than the interva...
Abstract: This article defines an approximate confidence interval for effect size in correlated (repeated measures) groups designs. The authors found that their method was much more accurate than the interva...

Journal ArticleDOI
TL;DR: Untilbootstrap analysis becomes an automated program option in standard statistical software programs (e.g., SPSS, SAS), quantitative researchers may have to make do with these or other creative approaches to accomplish bootstrap analysis in their research.
Abstract: Bootstrap analysis, both for nonparametric statistical inference and for describing sample results stability and replicability, has been gaining prominence among quantitative researchers in educational and psychological research Procedurally, however, it is often quite a challenge to implement bootstrap analysis because it is typically not an automated program option in statistical software programs This article uses a few heuristic analytical examples to show how bootstrap analysis can be accomplished through the use of some commonly available statistical software programs (AMOS, EQS, SAS) Until bootstrap analysis becomes an automated program option in standard statistical software programs (eg, SPSS, SAS), quantitative researchers may have to make do with these or other creative approaches to accomplish bootstrap analysis in their research

Journal ArticleDOI
TL;DR: This paper examined the validity and reliability of scores on the Academic Self-Concept Scale (ASCS) in a group of 291 European American and 396 African American college students and found that there are important ethnic differences in the structure of academic self-concept regarding beliefs about ability, and the relationship between effort and grades.
Abstract: This study examined the validity and reliability of scores on the Academic Self-Concept Scale (ASCS) in a group of 291 European American and 396 African American college students. The study compared the factor structure and scale and subscale internal consistencies for these samples to previously reported findings by Reynolds. The seven-factor solution reported by Reynolds was essentially replicated with the European American sample, whereas an eight-factor solution was found with the African American sample. Results suggest that there are important ethnic differences in the structure of academic self-concept regarding beliefs about ability, and the relationship between effort and grades.

Journal ArticleDOI
TL;DR: For instance, this paper found that higher socioeconomic status is associated with increased internal consistency in emotion scales, while age and majority ethnic status also positively associated with internal consistency of emotion scales.
Abstract: Because self-report is a valuable and often irreplaceable source of information about emotions, it is important to learn how measurement properties change across different demographic samples. Previous findings of mean differences in the level or variability of trait emotions for men versus women or across ethnic groups suggest that there may be differences in the internal consistency of the same measure when applied to different groups. Research groups using the Differential Emotions Scale (DES) have also begun to use the DES or derivative measures in children as young as 5 years old, raising questions about the psychometric performance of the measure across the life span. The present study uses reliability generalization to synthesize findings from 30 samples of raw data involving a total of 2,407 participants. Results indicate that higher socioeconomic status is associated with increased internal consistency in emotion scales. Age and majority ethnic status also were positively associated with internal...

Journal ArticleDOI
TL;DR: In this paper, a non-technical introduction of mixed-effects models in the context of reliability generalization (RG) studies is provided along with an example using internal consistency reliability coefficients from the Beck Depression Inventory that compares results under the mixed-versus.
Abstract: Traditionally, reliability generalization (RG) studies have used some form of the regression model to summarize score reliability of a measure across samples and examine conditions under which the reliability varies. Oftentimes, the assumptions underlying the use of multiple regression are not satisfied in RG studies. This article describes how the assumptions have been violated and introduces a more sophisticated technique, mixed-effects modeling, that can overcome many of the shortcomings of traditional approaches. A nontechnical introduction of mixed-effects models in the context of RG studies is provided along with an example using internal consistency reliability coefficients from the Beck Depression Inventory that compares results under the mixed-versus the fixed-effects models.

Journal ArticleDOI
TL;DR: The problem of missing data at the second level of a two-level hierarchical data structure was investigated and listwise deletion and the EM algorithm performed equally well for the variable having missing values and for the variables having no missing data.
Abstract: The problem of missing data at the second level of a two-level hierarchical data structure was investigated. Using data generated to simulate the 1982 High School and Beyond data set, five missing ...

Journal ArticleDOI
TL;DR: In this article, the degree of factor structure invariance of self-concept measurement across ethnic and gender groups was investigated for the National Educational Longitudinal Survey of 1988 (NELS: 88) data.
Abstract: This study investigated the degree of factor structure invariance of self-concept measurement across ethnic and gender groups. The National Educational Longitudinal Survey of 1988 (NELS: 88) data w...