Showing papers in "Journal of Psychoeducational Assessment in 2011"
TL;DR: The present article provides a current overview of these areas in an effort to provide researchers with up-to-date methods and considerations in both exploratory and confirmatory factor analysis.
Abstract: Researchers must make numerous choices when conducting factor analyses, each of which can have significant ramifications on the model results. They must decide on an appropriate sample size to achieve accurate parameter estimates and adequate power, a factor model and estimation method, a method for determining the number of factors and evaluating model fit, and a rotation criterion. Unfortunately, researchers continue to use outdated methods in each of these areas. The present article provides a current overview of these areas in an effort to provide researchers with up-to-date methods and considerations in both exploratory and confirmatory factor analysis. A demonstration was provided to illustrate current approaches. Choosing between confirmatory and exploratory methods is also discussed, as researchers often make incorrect assumptions about the application of each.
774 citations
TL;DR: In this article, the authors emphasize the importance of testing for measurement invariance (MI) and provide guidance when conducting these tests and discuss potential causes of non-invariant items, the difference between measurement bias and invariance, remedies for non-informal measures, and considerations associated with model estimation.
Abstract: Researchers commonly compare means and other statistics across groups with little concern for whether the measure possesses strong factorial invariance (ie, equal factor loadings and intercepts/thresholds) When this assumption is violated, inaccurate inferences associated with statistical and practical significance can occur This manuscript emphasizes the importance of testing for measurement invariance (MI) and provides guidance when conducting these tests Topics discussed are potential causes of noninvariant items, the difference between measurement bias and invariance, remedies for noninvariant measures, and considerations associated with model estimation Using a sample of 491 teachers, a demonstration is also provided that evaluates whether a newly constructed behavior and instructional management scale is invariant across elementary and middle school teachers Analyses revealed that the results differ slightly based on the estimation method utilized although these differences did not greatly in
440 citations
TL;DR: In this article, the authors argue that researchers should consider alternatives to coefficient alpha and use structural equation modeling (SEM) estimates of reliability because they are informative and allow for an empirical evaluation of the assumptions underlying them.
Abstract: Coefficient alpha is almost universally applied to assess reliability of scales in psychology. We argue that researchers should consider alternatives to coefficient alpha. Our preference is for structural equation modeling (SEM) estimates of reliability because they are informative and allow for an empirical evaluation of the assumptions underlying them. An example is presented to illustrate the advantages of SEM estimates of reliability.
349 citations
TL;DR: This article proposed exploratory structural equation modeling (ESEM), an integration of the best aspects of CFA and traditional exploratory factor analyses (EFA) to fit the data much better and results in substantially more differentiated (less correlated) factors than corresponding CFA models.
Abstract: The most popular measures of multidimensional constructs typically fail to meet standards of good measurement: goodness of fit, measurement invariance, lack of differential item functioning, and well-differentiated factors that are not so highly correlated as to detract from their discriminant validity. Part of the problem, the authors argue, is undue reliance on overly restrictive independent cluster models of confirmatory factor analysis (ICM-CFA) in which each item loads on one, and only one, factor. Here the authors demonstrate exploratory structural equation modeling (ESEM), an integration of the best aspects of CFA and traditional exploratory factor analyses (EFA). On the basis of responses to the 11-factor Motivation and Engagement Scale (n = 7,420, Mage = 14.22), we demonstrate that ESEM fits the data much better and results in substantially more differentiated (less correlated) factors than corresponding CFA models. Guided by a 13-model taxonomy of ESEM full-measurement (mean structure) invarianc...
169 citations
TL;DR: The authors analyzed the relationship between trait emotional intelligence and academic performance, controlling for the effects of IQ, personality, and self-concept dimensions, using a sample of 290 students from MIT.
Abstract: This article analyses the relationship between trait emotional intelligence and academic performance, controlling for the effects of IQ, personality, and self-concept dimensions. A sample of 290 pr...
119 citations
TL;DR: Wang et al. as mentioned in this paper described the development and initial validation of a new instrument to measure academic stress, the Educational Stress Scale for Adolescents (ESSA), which contains five latent variables: pressure from study, workload, worry about grades, self-expectation, and depression.
Abstract: This article describes the development and initial validation of a new instrument to measure academic stress—the Educational Stress Scale for Adolescents (ESSA). A series of cross-sectional questionnaire surveys were conducted with more than 2,000 Chinese adolescents to examine the psychometric properties. The final 16-item ESSA contains five latent variables: Pressure from study, Workload, Worry about grades, Self-expectation, and Despondency, which together explain 64% of the total item variance. Scale scores showed adequate internal consistency, 2-week test–retest reliability, and satisfactory concurrent validity. A confirmatory factor analysis suggested the proposed factor model fits well in a different sample. For researchers who have a particular interest in academic stress among adolescents, the ESSA promises to be a useful tool.
87 citations
TL;DR: In this article, the authors discuss the importance of conducting differential item functioning (DIF) analyses using a priori hypotheses whenever possible, and demonstrate how to test for DIF using logistic regression and DIFPACK.
Abstract: The purpose of this manuscript was to help researchers better understand the causes and implications of differential item functioning (DIF), as well as the importance of testing for DIF in the process of test development and validation. The underlying theoretical reason for the presence of DIF is explicated, followed by a discussion of how to test for the presence of DIF using logistic regression and DIFPACK, which includes SIBTEST, PSIBTEST and Crossing SIBTEST. This manuscript stresses the importance of conducting DIF analyses using a priori hypotheses whenever possible. However, the example that is provided, to show researchers and practitioners how to conduct a DIF analysis, utilizes an exploratory DIF analyses paradigm which may often be needed in practical DIF applications. This example uses PSIBTEST to test for DIF, using data from an international assessment that includes a mixture of polytomous and dichotomous items. In addition to demonstrating how to test for DIF, this manuscript demonstrates h...
86 citations
TL;DR: This article examined the psychometric properties of the PSSM to refine its utility for researchers and practitioners using a sample of 504 Australian high school students using exploratory and confirmatory factor analyses.
Abstract: Despite its widespread use, there has been limited examination of the underlying factor structure of the Psychological Sense of School Membership (PSSM) scale. The current study examined the psychometric properties of the PSSM to refine its utility for researchers and practitioners using a sample of 504 Australian high school students. Results from exploratory and confirmatory factor analyses indicated that the PSSM is a multidimensional instrument. Factor analysis procedures identified three factors representing related aspects of students’ perceptions of their school membership: caring relationships, acceptance, and rejection
82 citations
TL;DR: In this article, the authors describe the development of initial validity data for an Infrequency Index for the Conner's Adult Attention Deficit/Hyperactivity Rating Scale (CII), which was obtained from a large sample of nontreatment seeking university students, including individuals with a selfreported history of ADHD diagnosis.
Abstract: There is a clinical need for measurement of noncredible self-reporting of symptoms of attention deficit hyperactivity disorder (ADHD) in adults presenting for ADHD evaluation. The present study describes the development of initial validity data for an Infrequency Index for the Conner’s Adult Attention Deficit/Hyperactivity Rating Scale (CII). Items for the CII were obtained from a large sample of nontreatment seeking university students, including individuals with a self-reported history of ADHD diagnosis. Items endorsed infrequently in the sample, including those with ADHD diagnoses, were identified and summed to create the CII. Initial validation data were gathered from a sample of individuals seeking clinical evaluation for ADHD. The CII was strongly related to noncredibly high symptom report and was also related to noncredible performance on cognitive measures. Results provide initial support for the CII’s use in assessing noncredible overreporting on the Conner’s Adult Attention-Deficit/Hyperactivity...
62 citations
TL;DR: There is a paucity of information surrounding maths anxiety levels in the British undergraduate student population, and, due to terminological issues, existing measures of maths anxiety may not be....
Abstract: There is a paucity of information surrounding maths anxiety levels in the British undergraduate student population, and, due to terminological issues, existing measures of maths anxiety may not be ...
62 citations
TL;DR: The Audiovisual Test of Emotional Intelligence (AVEI) as mentioned in this paper is a test of emotional intelligence aimed at candidate selection in educational settings, which has been developed since the 1990s.
Abstract: Two studies describe the development of the Audiovisual Test of Emotional Intelligence (AVEI), aimed at candidate selection in educational settings. Study I depicts the construction of the test and...
TL;DR: The psychometric properties of the Test Anxiety Scale for Elementary Students (TAS-E) scores were examined in this article, which produced a four-factor solution: Physiological hyperarousal, social concern, task irrelevant behavior, and worry.
Abstract: The psychometric properties of the Test Anxiety Scale for Elementary Students (TAS-E) scores were examined. In Study 1, an exploratory factor analysis (EFA) was performed on the responses of 997 students in Grades 2 to 6 on the TAS-E. The results of the EFA produced a four-factor solution: Physiological Hyperarousal, Social Concerns, Task Irrelevant Behavior, and Worry. Gender differences were also found on each factor, except the Social Concerns factor. In Study 2, a confirmatory factor analysis (CFA) was performed on the TAS-E scores with a sample of 421 elementary students. The CFA results provided support for the four-factor solution in Study 1. The results also indicated that the TAS-E scores had adequate internal consistency reliability and test score stability. Evidence for convergent and discriminant validity of the TAS-E scores was found.
TL;DR: In this article, the authors test the psychometric properties of the Italian version of the Vigil-Colet et al. Scale (SAS), taking into account evidences based on (a) internal structure (factorial structure and cross-country invariance) and (b) relationships to other variables (the statistics anxiety's nomological network).
Abstract: The aim of the research was to test the psychometric properties of the Italian version of the Vigil-Colet et al.’s Statistical Anxiety Scale (SAS), taking into account evidences based on (a) internal structure (factorial structure and cross-country invariance) and (b) relationships to other variables (the statistics anxiety’s nomological network). Consistent with the original Spanish version, the Italian version showed a three-factor structure, and results indicated a substantial equivalence of factor model parameters across countries. SAS results appeared to be inversely related to mathematical ability, self-efficacy, and attitudes toward statistics. The overall findings of the present study provide evidence for the validity and reliability of the SAS as a measure of statistics anxiety across the Italian and Spanish educational frameworks.
TL;DR: The Widener Emotional Learning Scale (WELS) as discussed by the authors is a self-report measure for assessing students' social and emotional competence in higher education, which consists of 33 items with five subscales: Awareness of Emotions in Self and others, Tolerance of Difference or Conflict, Interpersonal Relationship Skills, Flexibility in Perspective-Taking and Behavior, and Self-Management Skills.
Abstract: This article reports the development and validation studies of the Widener Emotional Learning Scale (WELS), a self-report measure, for assessing students’ social and emotional competence in higher education. Conceptual specifications, item development, psychometric properties, and factor structure of the instrument are reported in the article. The resulting instrument consists of 33 items with 5 subscales: Awareness of Emotions in Self and Others, Tolerance of Difference or Conflict, Interpersonal Relationship Skills, Flexibility in Perspective-Taking and Behavior, and Self-Management Skills. A cross-validation study of the WELS with the Bar-On Emotional Quotient Inventory (Bar-On EQ-i) provides evidence that, although the WELS assesses some aspects of social and emotional competence also assessed by the Bar-On EQ-i, it makes a unique contribution in assessing flexibility in perspective taking and behavior. A cross-validation study of the WELS with the Mayer—Salovey—Caruso Emotional Intelligence Test (MSC...
TL;DR: The Test of Nonverbal Intelligence (TONI-4) as discussed by the authors is a nonverbal, motor-reduced format that assesses common elements of intelligence without the confounding effects of motor or linguistic skills.
Abstract: The Test of Nonverbal Intelligence–Fourth Edition (TONI-4) is an individually administered instrument created to assess intelligence. The distinguishing characteristic of the TONI-4 is the nonverbal, motor-reduced format that assesses common elements of intelligence without the confounding effects of motor or linguistic skills. The TONI-4 purports to assess the theoretical construct of fluid intelligence (Horn, 1965; Horn & Cattell, 1966) and general intelligence (Spearman, 1904); more specifically, the test aims to measure two components of intelligence: abstract reasoning and problem solving. The TONI-4 evolved from three predecessors to improve the manual, theoretical basis, extension of the ceiling and floor, normative sample size, reliability, and validity. TONI-4 administrators need training in assessment measures and an understanding of mental ability testing and measurement. Brown, Sherbenou, and Johnsen (2010) stated that the TONI-4 can be used to estimate aptitude and general intellectual functioning, identify intellectual impairments, verify the validity of referrals for treatment, formulate hypotheses for evaluation, and conduct research. The TONI-4 is appropriate for populations with language, hearing, or motor impairment or that are unfamiliar with mainstream American culture. However, the TONI-4 is not intended to test individuals with visual impairments.
TL;DR: This article examined the utility of inferential norming using small samples drawn from the larger Wechsler Intelligence Scales for Children-Fourth Edition (WISC-IV) standardization data set.
Abstract: We examined the utility of inferential norming using small samples drawn from the larger Wechsler Intelligence Scales for Children–Fourth Edition (WISC-IV) standardization data set. The quality of ...
TL;DR: In this paper, the reliability and validity of the scores on a brief strengths-based assessment, the 22-item Personal Strengths Inventory (PSI), was established. But the results are discussed in comparison with other research on social-emotional competence, and further research is suggested.
Abstract: The purpose of this study was to establish the reliability and validity of the scores on a brief strengths-based assessment, the 22-item Personal Strengths Inventory (PSI). In Study 1, findings from exploratory factor analysis of 410 adolescents provided evidence for a five-factor solution—social competence (four items), emotional awareness (five items), goal setting (six items), emotional regulation (four items), and empathy (three items). Preliminary evidence of convergent validity was also reported. In Study 2, results from confirmatory factor analysis of 598 adolescents provided additional support for the results obtained from Study 1. Moderate to high reliability coefficients were obtained for the PSI scores. The results are discussed in comparison with other research on social—emotional competence, and further research is suggested.
TL;DR: This article examined the reliability and validity of a Chinese-translated version of the Substance Use Risk Profile Scale (SURPS) and explored the effect of gender and age on each of the subscales.
Abstract: This study examines the reliability and validity of a Chinese-translated version of the Substance Use Risk Profile Scale (SURPS) and explores the effect of gender and age on each of the subscales. The sensitivity of SURPS in differentiating substance users and nonusers using the SURPS is also investigated. Data were collected from a convenience sample that consisted of 250 students from secondary schools in Hong Kong. Results indicated that the reliability of the subscales for introversion/hopelessness, impulsivity, sensation seeking, and anxiety sensitivity was comparable with those in previous studies. Confirmatory factor analysis supported the four-factor model of the original theoretical model of SURPS. However, no significant difference was found between each subscale score and the self-reported substance use behavior. Possible explanations on such findings are discussed. Despite the fact that the sensitivity of the SURPS in differentiating substance users and nonusers among Hong Kong adolescents is ...
TL;DR: This paper explored the underlying structure of cultural competence for preservice teachers through exploratory factor a.k.a. cultural competence in the context of pre-service teachers, and found that cultural competence is a function of the learner's ability to adapt to the role of the teacher.
Abstract: The ongoing conceptual controversy of cultural competence for preservice teachers led to the current study that explored the underlying structure of cultural competence through exploratory factor a...
TL;DR: The Social Skills Improvement System (SSIS) as discussed by the authors is designed to assist in the screening and classification of students (ages 8 to 18) who are suspected of having significant social skills deficits.
Abstract: The Social Skills Improvement System (SSIS; Gresham & Elliot, 2008) is designed to assist in the screening and classification of students (ages 8 to 18) who are suspected of having significant social skills deficits, and to offer support in the development of interventions for those found to display significant social skills impairments. Although the SSIS is a revision of the widely-used Social Skills Rating System (SSRS; Gresham & Elliott, 1990), the strengths of the SSRS are retained in the SSIS. Improvements include updated norms, four additional subscales, greater overlap across forms, validity scales, improved psychometric properties, Spanish versions of forms, and direct links to intervention. The focus of the current review is the SSIS Rating Scales (SSIS-RS), a component of the SSIS. In addition to the SSIS-RS, the SSIS includes the Performance Screening Guide (intended for universal assessment [Tier 1]; Elliot & Gresham, 2007a), the Classwide Intervention Program Teacher’s Guide (Elliot & Gresham, 2007b), and the Intervention Guide (Elliott & Gresham, 2009).
TL;DR: This article examined the cross-cultural equivalence of a newly developed questionnaire, the Emotion Awareness Questionnaire (EAQ30) that assesses emotional awareness of children through self-report.
Abstract: The main aim of the present study was to examine the cross-cultural equivalence of a newly developed questionnaire, the Emotion Awareness Questionnaire (EAQ30) that assesses emotional awareness of children through self-report. Participants were recruited in three countries: the Netherlands (N = 665), Spain (N = 464), and Belgium (N = 707), reflecting three languages: Dutch, Spanish, and French respectively. The results of multigroup confirmatory factor analysis showed that the 6-factor structure of the EAQ30 is similar in all three samples. These results were confirmed by the high coefficients of congruence that indicated factorial invariance across samples. Comparison between the three samples showed minor cultural differences. These findings support the factorial and construct validity of the EAQ30 and suggest that the EAQ30 accurately captures the structure of children’s emotional dispositions, regardless of cultural differences.
TL;DR: The relationship between direct and indirect measurements of social skills and social problem behaviors for preschool children at risk for Attention Deficit hyperactivity disorder (ADHD) was examined in this article.
Abstract: The relationship between direct and indirect measurements of social skills and social problem behaviors for preschool children at risk for attention deficit hyperactivity disorder (ADHD) was examined. Participants included 137 preschool children, aged 3 to 5 years, at risk for ADHD, who were participating in a larger study examining the effects of early intervention for young children. Teachers rated the social skills and social problems of the participants. Direct observation data of participants were also collected at preschool during free play. Results support previous research on social skills assessment and suggest that indirect and direct measures may not be measuring the same aspect of social skills. Thus, a variety of evaluation tools are necessary to comprehensively assess the social skills of preschool children with social challenges.
TL;DR: In this paper, a detailed analysis of the psychometric properties of the MTEBI using Bayesian item response theory is provided, including local dependence between item pairs, psychometric quality of the items, validity of the scoring procedure, and measurement accuracy for teachers with different efficacy levels.
Abstract: The mathematics teaching efficacy beliefs of preservice elementary teachers have been the subject of several studies. A widely used measure in these studies is the Mathematics Teaching Efficacy Beliefs Instrument (MTEBI). The present study provides a detailed analysis of the psychometric properties of the MTEBI using Bayesian item response theory. We discuss local dependence between item pairs, psychometric quality of the items, validity of the scoring procedure, and measurement accuracy for teachers with different efficacy levels. Our findings suggest that in its present form, the test reliability of the MTEBI may not be as high as assumed to date. The scale, wording, and placement of the items need revision. Moreover, additional items need to be constructed to measure below average levels of efficacy more accurately. Ordering the items according to difficulty, we describe the structure of mathematics teaching efficacy beliefs and draw some implications for mathematics teacher educators.
TL;DR: Goldstein et al. as mentioned in this paper proposed the Autism Spectrum Rating Scales (ASRS) to measure behaviors in children between the ages of 2 and 18 that are associated with disorders on the autism spectrum as rated by parents/caregivers and/or teachers.
Abstract: Goldstein, S., & Naglieri, J. A. (2009). ASRS: Autism Spectrum Rating Scales. Toronto, Ontario, Canada: Multi-Health Systems. The Autism Spectrum Rating Scales (ASRS) are designed to measure behaviors in children between the ages of 2 and 18 that are associated with disorders on the autism spectrum as rated by parents/caregivers and/or teachers. The rating scales include items related to behaviors associated with Autism, Asperger’s Disorder, and Pervasive Developmental Disorder—Not Otherwise Specified (PDD-NOS). The ASRS are a norm-referenced tool based on the diagnostic criteria of ASDs in the Diagnostic and Statistical Manual of Mental Disorders (4th ed.), Text Revision (DSM-IV-TR; American Psychiatric Association, 2000).
TL;DR: The Preschool Behavioral and Emotional Rating Scale (PreBERS) as discussed by the authors is a 42-item family member or school personnel-completed rating scale designed to measure the behavioral and emotional strengths of preschool children ages 3-0 to 5-11.
Abstract: The Preschool Behavioral and Emotional Rating Scale (PreBERS) is a 42-item family member– or school personnel–completed rating scale designed to measure the behavioral and emotional strengths of preschool children ages 3-0 to 5-11. According to the manual, results can be used to identify preschoolers with limited behavioral and emotional strengths, plan interventions, and monitor progress. A unique aspect of the PreBERS is that it relies entirely on strengthbased assessment rather than a deficit approach. Overall results are described as the Strength Index, which is composed of four subscales: Emotional Regulation (ER), School Readiness (SR), Social Confidence (SC), and Family Involvement (FI). Results can be described as a quotient (M = 100; SD = 15), percentile, or with a qualitative description (e.g., below average). Subscales can be described as scaled scores (M = 10; SD = 3), percentiles, or with qualitative descriptions. Higher scores are indicative of more strengths. Seven open-ended questions are included to obtain additional information on the child’s personal, family, and community resources and strengths. Completion of the PreBERS requires about 10 minutes. Materials include the Examiner’s Manual and a Summary/Response Form.
TL;DR: The 2009 Forest Grove School District v. T.A. as mentioned in this paper case could have significant implications for school psychology practice, as the Court ruled that the parents of a student with a disability were entitled to private school tuition reimbursement, even though the student had not been identified with any disability or previously provided with special education services.
Abstract: The 2009 Forest Grove School District v. T.A. United States Supreme Court case could have significant implications for school psychology practice. The Court ruled that the parents of a student with a disability were entitled to private school tuition reimbursement even though T.A. had not been identified with a disability or previously provided with special education services. The district multidisciplinary team had not identified T.A.’s disability in part because of a limited psychoeducational evaluation that did not assess him in “all areas of suspected disability” which is a requirement of the Individuals with Disabilities Education Act (IDEA; 20 U.S.C. § 1414(b)(3)(B)). Dissatisfied with the team’s decision, the parents sought an independent comprehensive evaluation of T.A.’s psychological processes which revealed the full nature of his disability and needs. Although T.A. had been unresponsive to interventions in his home school, this comprehensive psychoeducational evaluation led to more targeted int...
TL;DR: Several instruments have been developed to gauge student perceptions of their classroom environment as greater attention has been given to the relationship between student achievement and student perception of their environment as mentioned in this paper,... ).
Abstract: Several instruments have been developed to gauge student perceptions of their classroom environment as greater attention has been given to the relationship between student achievement and student p...
TL;DR: In this paper, a double-checking procedure was established at a psychology training clinic in an effort to improve intelligence test scoring accuracy, which significantly changed IQ scores, increasing their accuracy.
Abstract: The rigors of standardized testing make for numerous opportunities for examiner error, including simple computational mistakes in scoring. Although experts recommend that test scoring be double-checked, the extent to which independent double-checking would reduce scoring errors is not known. A double-checking procedure was established at a psychology training clinic in an effort to improve intelligence test scoring accuracy. The authors examined 50 archival protocols to determine the effects of independent double-checking on Wechsler Intelligence Scale for Children—Third Edition scores. As expected, these procedures significantly changed IQ scores, increasing their accuracy. Anonymous trainee feedback supported the utility of these efforts and endorsed it as making a useful contribution to professional development. Recommendations for instituting such procedures in training settings are discussed, and a sample checklist to guide double-checking is provided.