Showing papers in "Journal of Experimental Education in 2016"
TL;DR: Using 1,000 replications of 12 conditions with varied Level 1 and Level 2 sample sizes, the author compared parameter estimates, standard errors, and statistical significance using various alternative procedures to indicate that several acceptable procedures can be used in lieu of or together with multilevel modeling.
Abstract: Multilevel modeling has grown in use over the years as a way to deal with the nonindependent nature of observations found in clustered data. However, other alternatives to multilevel modeling are available that can account for observations nested within clusters, including the use of Taylor series linearization for variance estimation, the design effect adjusted standard errors approach, and fixed effects modeling. Using 1,000 replications of 12 conditions with varied Level 1 and Level 2 sample sizes, the author compared parameter estimates, standard errors, and statistical significance using various alternative procedures. Results indicate that several acceptable procedures can be used in lieu of or together with multilevel modeling, depending on the type of research question asked and the number of clusters under investigation. Guidelines for applied researchers are discussed.
200 citations
TL;DR: In this article, the educational utility of teaching in students' preferred ways was evaluated, and a pilot test was conducted to evaluate the effectiveness of autonomy-supportive teaching to enhance students' autonomy and conceptual learning.
Abstract: We tested the educational utility of “teaching in students' preferred ways” as a new autonomy-supportive way of teaching to enhance students' autonomy and conceptual learning. A pilot test first differentiated preferred versus nonpreferred ways of teaching. In the main study, a hired teacher who was blind to the purpose of the study taught 63 college-age participants in small groups the same 48-minute lesson in one of these two different ways, and we assessed participants' perceived autonomy support, autonomy-need satisfaction, engagement (self-report and rater scored), and conceptual learning (self-report and rater scored). Multilevel analyses showed that participants randomly assigned to receive a preferred way of teaching perceived the teacher as more autonomy supportive and showed significantly greater autonomy-need satisfaction, engagement, and conceptual learning. Mediation analyses using multilevel modeling for clustered data showed that this way of teaching enhanced conceptual learning because it ...
192 citations
TL;DR: In this paper, the psychological adjustment of ethnic minority college students may be linked to a sense of school belonging and ethnic identity, two constructs related to individuals feeling like they belong to a larger group.
Abstract: This study considers how the psychological adjustment of ethnic minority college students may be linked to a sense of school belonging and ethnic identity, two constructs related to individuals feeling like they belong to a larger group. Using self-reports from 311 undergraduates from ethnic minority backgrounds, school belonging was found to be negatively associated with depressive symptoms, and positively associated with perceived self-worth, scholastic competence, and social acceptance, while ethnic identity was only found to be positively associated with self-worth. Furthermore, the interaction between ethnic identity and school belonging was significantly associated with self-worth, suggesting that in the absence of a sense of belonging at school, stronger ethnic identity was linked to higher self-worth. College students reported the lowest levels of self-worth when they were neither connected to their college nor connected to their ethnic group.
95 citations
TL;DR: This paper investigated the extent to which differences in implicit and explicit math-language gender stereotypes, and essentialist beliefs among preservice teachers affect tracking recommendations for math/science- versus language-oriented secondary schools.
Abstract: This study investigated the extent to which differences in implicit and explicit math–language gender stereotypes, and essentialist beliefs among preservice teachers affect tracking recommendations for math/science- versus language-oriented secondary schools. Consistent with expectations, the results suggest that student's gender influences preservice teachers’ school career recommendations: Boys are more likely to be recommended for a math/science-oriented secondary school, whereas girls are more likely recommended for a language-oriented school. Both implicit math–language gender stereotypes and beliefs in genetic determinism (reflecting essentialist beliefs) predicted the stereotypicality of school career recommendations, whereas explicit measures did not. The results suggest that more closely investigating factors contributing to stereotypical behavior in teachers might help to minimize biased actions and decisions in the educational context.
82 citations
TL;DR: The Test of Relational Reasoning (TORR) as mentioned in this paper has been used to evaluate students' ability to reason relationally, and its performance predicted students' performance on SAT verbal and math problems.
Abstract: Relational reasoning is the foundational cognitive ability to discern meaningful patterns within an informational stream, but its reliable and valid measurement remains problematic. In this investigation, the measurement of relational reasoning unfolded in three stages. Stage 1 entailed the establishment of a research-based conceptualization of the construct and the development of a corresponding Test of Relational Reasoning (TORR). Stage 2 focused on the reliability and validity of data from the TORR. Analyses showed the data from the TORR to be reliable indicators of students' ability to reason relationally, and TORR performance predicted students' performance on SAT verbal and math problems. Stage 3 examined the underlying structure of the construct through Confirmatory Factor Analysis (CFA). Of the three CFA models tested, models with dedicated factors for analogical, anomalous, antinomous, and antithetical reasoning were deemed the best fit for the data.
60 citations
TL;DR: The authors examined the performance of random effect binary outcome multilevel models under varying methods of estimation, level-1 and level-2 sample size, outcome prevalence, variance component sizes, and number of predictors using SAS software.
Abstract: The design of research studies utilizing binary multilevel models must necessarily incorporate knowledge of multiple factors, including estimation method, variance component size, or number of predictors, in addition to sample sizes. This Monte Carlo study examined the performance of random effect binary outcome multilevel models under varying methods of estimation, level-1 and level-2 sample size, outcome prevalence, variance component sizes, and number of predictors using SAS software. Mean estimates of statistical power were influenced primarily by sample sizes at both levels. In addition, confidence interval coverage and width and the likelihood of nonpositive definite random effect covariance matrices were impacted by variance component size and estimation method. The interactions of these and other factors with various model performance outcomes are explored.
57 citations
TL;DR: The authors examined the moderating effects of a situational factor (text type) and an individual factor (i.e., subject-matter knowledge) on the relation between depth of processing and performance.
Abstract: This study examines the moderating effects of a situational factor (i.e., text type) and an individual factor (i.e., subject-matter knowledge) on the relation between depth of processing and performance. One-hundred and fifty-one undergraduates completed measures of subject-matter knowledge, read either an expository or persuasive text about the existence of extraterrestrials while thinking aloud, and then completed a passage recall task and an open-ended task. Results indicated that the relation between depth of processing and the open-ended tasks was moderated by the type of text participants read (i.e., expository or persuasive). Moreover, there was a significant interaction between the passage recall measure and open-ended task for depth of processing and type of text.
50 citations
TL;DR: This article examined preservice teachers' implicit attitudes toward students with special educational needs (SEN) from different ethnic backgrounds and explicit attitudes toward inclusive education and found that implicit and explicit evaluations could bias teachers' interactions with students.
Abstract: Drawing on social cognition frameworks, we experimentally examined preservice teachers' implicit attitudes toward students with special educational needs (SEN) from different ethnic backgrounds and preservice teachers' explicit attitudes toward inclusive education. Preservice teachers (N = 46) completed an evaluative priming task and questionnaires. Results showed indifferent implicit attitudes toward students with SEN with immigrant backgrounds and positive implicit attitudes toward those without immigrant backgrounds. Furthermore, participants reported a high motivation to act without prejudice toward minorities but held less favorable explicit attitudes toward inclusion of students with SEN, especially students with behavioral problems. Differential patterns of implicit and explicit evaluations could bias teachers' interactions with students. Findings are discussed with respect to implications for educational practice and research.
41 citations
TL;DR: In this article, exploratory structural equation modeling is applied to investigate the factor structure of multidimensional self-concept instruments, and the results supported the superiority of exploratory SEM compared with confirmatory factor analyses for both instruments.
Abstract: This study is a substantive-methodological synergy in which exploratory structural equation modeling is applied to investigate the factor structure of multidimensional self-concept instruments. On the basis of a sample of German students (N = 1958) who completed the Self-Description Questionnaire I and the Self-Perception Profile for Children, the results supported the superiority of exploratory structural equation modeling compared with confirmatory factor analyses for both instruments. Exploratory structural equation modeling resulted in lower factor correlations and substantively meaningful cross-loadings. The authors also proposed and contrasted 3 mechanisms for testing grade-related differences in the differentiation of self-concept facets and found no evidence of increased differentiation between Grades 3 to 6.
34 citations
TL;DR: In this paper, the authors determined the impact of different types of refutation texts on adolescent readers' conceptual change learning in science and found that learning effects were optimized when the correct conceptions were presented in terms of a corroborated consensus among multiple sources.
Abstract: A refutation text is designed to promote conceptual change by explicitly acknowledging commonly held misconceptions about a topic, directly refuting them, and providing an accurate explanation. In this study, we determined the impact of different types of refutation texts on adolescent readers’ conceptual change learning in science. Specifically, we manipulated the way the correct conception was justified and explained following work in epistemic cognition. Three different types of justification were compared to a control condition: justification by authority, justification by multiple sources, and justification by personal opinion. The findings showed that learning effects were optimized when the correct conceptions were presented in terms of a corroborated consensus among multiple sources.
34 citations
TL;DR: The authors empirically scrutinize Muis, Bendixen, and Haerle's (2006) Theory of Integrated Domains in Epistemology framework and find that students' beliefs across domains are somewhat related but still unique to that particular domain.
Abstract: The purpose of this study was to empirically scrutinize Muis, Bendixen, and Haerle's (2006) Theory of Integrated Domains in Epistemology framework. Secondary, college, undergraduate, and graduate students completed self-reports designed to measure their domain-specific and domain-general epistemic beliefs for mathematics, psychology, and general knowledge, respectively. Following completion of the questionnaires, students participated in an interview that further probed their epistemic beliefs to better understand the nature of their beliefs. Results from our study suggest students’ beliefs across domains are somewhat related but still unique to that particular domain. Moreover, analysis of the interviews revealed that students espouse general knowledge beliefs and domain-specific beliefs. Interestingly, students expressed absolutist beliefs about mathematics, but were multiplist in their stances toward psychology and general knowledge. When asked to provide examples that came to mind when reporting their...
TL;DR: This paper examined the ecological validity of using observation-based scores to evaluate individual teachers' effectiveness, mirroring their use by school administrators, and found that teachers' instructional quality was significantly less effective than their emotional supportiveness and classroom organization.
Abstract: The authors examined the ecological validity of using observation-based scores to evaluate individual teachers’ effectiveness, mirroring their use by school administrators. Using the Classroom Assessment Scoring System, the authors asked (a) how similar are teachers’ emotional support, classroom organization, and instructional support scores from each other; and (b) to what extent are these scores stable from day-to-day? Using a time-series, single-case methodology, the authors examined patterns of eight Kindergarten teachers’ performance in all lessons within a 4–5 week, thematically coherent science unit. Thus, the opportunity for identifying within-teacher stability was maximized. Teachers' instructional quality was significantly less effective than their emotional supportiveness and classroom organization. Furthermore, teachers’ effectiveness ratings varied markedly from one lesson to the next, for all three dimensions.
TL;DR: The authors evaluated the impact of covariance matrices at the second and third levels of the three-level model on the treatment effect estimate and found that covariance does not result in less biased between-study variance estimates.
Abstract: The impact of misspecifying covariance matrices at the second and third levels of the three-level model is evaluated. Results indicate that ignoring existing covariance has no effect on the treatment effect estimate. In addition, the between-case variance estimates are unbiased when covariance is either modeled or ignored. If the research interest lies in the between-study variance estimate, including at least 30 studies is warranted. Modeling covariance does not result in less biased between-study variance estimates as the between-study covariance estimate is biased. When the research interest lies in the between-case covariance, the model including covariance results in unbiased between-case variance estimates. The three-level model appears to be less appropriate for estimating between-study variance if fewer than 30 studies are included.
TL;DR: In this paper, reflections on the rationale behind self-made errors are assumed to enhance knowledge acquisition, and students in the first condition group demonstrated a greater procedural knowledge at the post-test and at the follow-up test.
Abstract: Thus far, it is unclear how students can learn most effectively from their own errors. In this study, reflections on the rationale behind self-made errors are assumed to enhance knowledge acquisition. In a field experiment with pre/post/follow-up design, the authors practiced fractions with 174 seventh- and eighth-grade students who were randomly assigned to one of two conditions: The students reflected on either the rationale behind their own errors or on the correct solution corresponding to their own errors. Students in the first condition group demonstrated a greater procedural knowledge at the posttest and at the follow-up test. Furthermore, at the follow-up test, these students demonstrated a higher conceptual knowledge. The implications for theory and school instruction are discussed.
TL;DR: This article examined the concordance between preservice teachers' responsibilities and their intended instructional practices and found that responsibility for student motivation negatively predicted performance goal-oriented practices; responsibility for teaching positively predicted rationales and negatively predicted extrinsic rewards.
Abstract: During their education, preservice teachers begin to assume professional responsibilities and gain pedagogical knowledge. However, the question remains whether preservice teachers intend to use instructional practices that are effective in meeting their assumed responsibilities. Thus, we examined the concordance between preservice teachers' responsibilities and their intended instructional practices. At the start of the semester there was no concordance. At the end of the semester there was high concordance in two instances: Responsibility for student motivation negatively predicted performance goal-oriented practices; responsibility for teaching positively predicted rationales and negatively predicted extrinsic rewards. Instructional practices associated with responsibility for achievement and motivation were not concordant. We discuss how preservice teachers understand these responsibilities and their readiness to adopt evidence-based practices.
TL;DR: The authors investigate the effects of discrepancies between teachers' perceptions of students' motivation and students' reports of their motivation on math and English grades and identify possible gender and ethnic differences in student motivation.
Abstract: The aims of this study were to investigate the effects of discrepancies between teachers' perceptions of students' motivation and students' reports of their motivation on math and English grades and to identify possible gender and ethnic differences. Participants included 215 low-income, ethnic-minority students and their teachers in academically struggling schools. Discrepancy in motivation ratings accounted for a significant amount of variation in final grades. This teacher-student discrepancy effect was larger than both the effect of students' recent standardized test scores and the effect of self-efficacy in both subjects. Girls were found to attract positive teacher bias, but their grades were not differentially affected. Differences in teacher-student discrepancy effects between African American and Latino students were not found.
TL;DR: This paper found that adolescents' reading comprehension is dually influenced by a text's readability and cohesion, and that adolescents in Group D significantly outperformed students in Group C (g = 0.78) on comprehension test items targeting factual recall and inferences of causal content.
Abstract: To better understand dimensions of text complexity and their effect on the comprehension of adolescents, 103 high school seniors were randomly assigned to 4 groups. Each group read versions of the same 2 informational passages and answered comprehension test items targeting factual recall and inferences of causal content. Group A passages had a challenging readability level and high cohesion; Group B passages had an easier readability and low cohesion; Group C passages had a challenging readability level and low cohesion; and Group D passages had an easier readability and high cohesion. Students in Group D significantly outperformed students in Group C (g = 0.78). Although the effect sizes of comparisons among all groups ranged from g = 0.13 to 0.73, no other comparisons were statistically significant. Results indicate that adolescents’ reading comprehension is dually influenced by a text's readability and cohesion. Implications for matching readers to instructional text are discussed.
TL;DR: In this article, the authors examined the natural combinations of achievement goals (mastery, performanceapproach, performance-avoidance) among 3rd grade students and how clusters differed in self-, teacher-, and peer-reported adjustment variables.
Abstract: Person-centered analyses of achievement goals have been scarce in studies of elementary school children. In this investigation, the authors examined the natural combinations of achievement goals (mastery, performance-approach, performance-avoidance) among 3rd grade students (N = 195) and how clusters differed in self-, teacher-, and peer-reported adjustment variables. Cluster analysis revealed four groups of students: mastery (above average in mastery goals, below average in performance-approach and performance-avoidance goals), multi-goal (above average in all three goals), avoidant (above average in performance-avoidance goals, below average in mastery and performance-approach goals), and low motivation (below average in all three goals). Clusters differed in self-reported academic self-efficacy and perceptions of teacher support, teacher-reported academic competence, and peer nominations of social status. Mastery students had the most adaptive profiles; low motivation, the least adaptive. Avoidant boys...
TL;DR: In this article, the authors used a crossover design, where students took three tests individually without feedback and then took the same test again, following the process of team-based testing (TBT), in teams in which the members reached consensus for each question and answered until they were correct.
Abstract: In two experiments, 90 undergraduates took six tests as part of an educational psychology course. Using a crossover design, students took three tests individually without feedback and then took the same test again, following the process of team-based testing (TBT), in teams in which the members reached consensus for each question and answered until they were correct. Students took the other three tests individually with feedback. All students were individually tested over a portion of this content two weeks later and again after two months. Independent samples t tests revealed that TBT students scored higher when retested two months later than those who took the test individually. Finally, three-fourths of the students reported that they enjoyed TBT more than individual testing. Although TBT requires more class time to administer, it appears to be beneficial for long-term student learning.
TL;DR: In this paper, the authors examined whether structuring (SS) versus problematising scaffolds (PS) differently affect reciprocal peer-tutoring (RPT) groups' adoption of particular regulation skills, deep-level regulation, and tutee-initiated regulation.
Abstract: The study examines whether structuring (SS) versus problematising scaffolds (PS) differently affect reciprocal peer-tutoring (RPT) groups’ adoption of particular regulation skills, deep-level regulation, and tutee-initiated regulation. A quasi-experimental design involving two experimental groups (SS versus PS condition) was adopted. The first, third, and sixth RPT-session of eight randomly selected RPT-groups (four from the SS condition, four from the PS condition) were videotaped (48hr). Mixed ANOVAs were conducted to investigate the impact of both scaffold types on RPT-groups’ metacognitive regulation. The results indicate that neither scaffold type encourages RPT-groups into a balanced adoption of, or initiative for, regulation skills and a deep-level approach. Nevertheless, the PS condition significantly outperforms the SS condition in evoking deep-level monitoring, tutee-initiated orientation, and tutee-initiated monitoring.
TL;DR: In this paper, the authors investigated the effectiveness of problem-based learning compared to direct instruction at fostering content learning, particularly for novices, and examined how students' goal orientations may moderate learning in PBL and lecture/discussion environments.
Abstract: Conflicting research results have stirred controversy over the effectiveness of problem-based learning (PBL) compared to direct instruction at fostering content learning, particularly for novices. We addressed this by investigating effectiveness with respect to recognition learning and transfer and conducting an aptitude-treatment interaction analysis. We also examined how students' goal orientations may moderate learning in PBL and lecture/discussion environments. Participants included 197 preservice teachers in an undergraduate educational psychology course. Controlling for GPA, students in the PBL condition performed significantly better than students in the lecture/discussion condition on a transfer measure (p <.001, partial η2 =.271) and equally on a recognition learning measure (p =.530). We also found an aptitude-treatment interaction for recognition learning (p =.012, partial η2 =.034) but not transfer (p =.088). Results of goal orientation on learning outcomes were inconclusive.
TL;DR: In this article, the authors describe two studies testing the efficacy of a commercial one-on-one cognitive training program (LearningRx) and its computer-based version (Brainskills) in laboratory and school settings.
Abstract: This article describes two studies testing the efficacy of a commercial one-on-one cognitive training program (LearningRx) and its computer-based version (Brainskills) in laboratory and school settings. Study 1 tested Brainskills in a laboratory setting with 322 middle school students. Paired t-tests revealed significant gains on all cognitive measures and math performance after 3 weeks of training. Study 2, a randomized control study, included 225 high school students randomly assigned to one of three conditions: LearningRx, Brainskills, or study hall (control) in a school setting for a 15-week training period. Univariate ANCOVAs revealed significantly higher scores for the treatment groups compared with controls on working memory, logic and reasoning, and three of four math attitude measures but not for math performance. Implications for school-based interventions are discussed.
TL;DR: In this paper, the authors developed a statistical model that best explains variability in the number of school days suspended, which is a count variable that may be zero-inflated and overdispersed relative to a Poisson model.
Abstract: The purpose of this article is to develop a statistical model that best explains variability in the number of school days suspended. Number of school days suspended is a count variable that may be zero-inflated and overdispersed relative to a Poisson model. Four models were examined: Poisson, negative binomial, Poisson hurdle, and negative binomial hurdle. Additionally, the probability of a student being suspended for at least 1 day was modeled using a binomial logistic regression model. Of the count models considered, the negative binomial hurdle model had the best fit. Modeling the probability of a student being suspended for at least 1 day using a binomial logistic regression model with interactions fit both the training and test data and had adequate fit. Findings here suggest that both the negative binomial hurdle and the binomial logistic regression models should be considered when modeling school suspensions.
TL;DR: In this paper, three experienced coders used Ames' clue set as a foundation for a system to classify context clues in children's text and found that the adjustments to Ames' system resulted in 15 clues that can be used to capture the majority of clues found in children narrative and expository text.
Abstract: This study aimed to determine which types of context clues exist in children's texts and whether it is possible for experts to identify reliably those clues. Three experienced coders used Ames’ clue set as a foundation for a system to classify context clues in children's text. Findings showed that the adjustments to Ames’ system resulted in 15 clues that can be used to capture the majority of clues found in children narrative and expository text. This study adds to the limited empirically validated systems available for classifying different context clues. A usable classification of context clues has application to reading research that considers or controls for text factors as well as teaching reading strategies in K–12 classrooms.
TL;DR: The effectiveness of two methods of improving connections—elaborative interrogation and diagrams—in written lessons about posterior probability are examined and implications for instruction in probabilistic reasoning are discussed.
Abstract: To learn from a text, students must make meaningful connections among related ideas in that text. This study examined the effectiveness of two methods of improving connections—elaborative interrogation and diagrams—in written lessons about posterior probability. Undergraduate students (N = 198) read a lesson in one of three questioning conditions (read twice, embedded questioning, and elaborative interrogation) and one of three diagram conditions (text only, diagram without redundant text, and diagram with redundant text). Elaborative interrogation negatively affected learning from the lesson, relative to reading the lesson twice. One possible explanation for this finding is that the quality of answers to the elaborative interrogations was poor. When the lesson was read twice, diagrams helped learning from the lesson relative to text only. Implications of these findings for instruction in probabilistic reasoning are discussed.
TL;DR: This article examined the immediate and sustained effects of three training conditions on both spelling performance and spelling consciousness of 72 third-grade low- and high-skilled spellers, and found that the strategy-instruction condition was more effective for spelling performance, and more active for spelling consciousness pertaining to loan words than the no-correction condition.
Abstract: This study examined the immediate and sustained effects of three training conditions on both spelling performance and spelling consciousness of 72 third-grade low- and high-skilled spellers. Spellers were assigned to a strategy-instruction, self-correction, or no-correction condition. The role of spelling ability and word characteristic were also taken into account. Regarding the immediate effects, the strategy-instruction condition was more effective for spelling performance, and more effective for spelling consciousness pertaining to loan words than the no-correction condition. Regarding the sustained effects on spelling performance and spelling consciousness, the positive effect of the strategy-instruction condition faded out after training. The four training sessions were insufficient for establishing long-lasting effects.
TL;DR: The authors compared the performance of two methods for combining p values in the context of a MANOVA, with the typical default for dealing with missing data: listwise deletion, and found that when data are missing at random, the new methods maintained the nominal Type I error rate and had power comparable to the complete data condition.
Abstract: Multivariate analysis of variance (MANOVA) is widely used in educational research to compare means on multiple dependent variables across groups. Researchers faced with the problem of missing data often use multiple imputation of values in place of the missing observations. This study compares the performance of 2 methods for combining p values in the context of a MANOVA, with the typical default for dealing with missing data: listwise deletion. When data are missing at random, the new methods maintained the nominal Type I error rate and had power comparable to the complete data condition. When 40% of the data were missing completely at random, the Type I error rates for the new methods were inflated, but not for lower percents.
TL;DR: The results showed that specifying the most complex within-subject V-C structure with the use of LRT, ΔAIC, and ΔBIC achieved the highest recovery rate of the true mean trajectory.
Abstract: This study investigated the optimal strategy for model specification search under the latent growth modeling (LGM) framework, specifically on searching for the correct polynomial mean or average growth model when there is no a priori hypothesized model in the absence of theory. In this simulation study, the effectiveness of different starting models on the search of the true mean growth model was investigated in terms of the mean and within-subject variance-covariance (V-C) structure model. The results showed that specifying the most complex (i.e., unstructured) within-subject V-C structure with the use of LRT, ΔAIC, and ΔBIC achieved the highest recovery rate (>85%) of the true mean trajectory. Implications of the findings and limitations are discussed.
TL;DR: In this paper, a second-order factor that accounts for the interrelationship among three first-order factors: mathematical beliefs, mathematical disposition, and mathematical cognition was used to compare and rank student quantitative literacy by gender and class standing.
Abstract: Quantitative literacy (QLT) represents an underlying higher-order construct that accounts for a person's willingness to engage in quantitative situations in everyday life. The purpose of this study is to retest the construct validity of a model of quantitative literacy (Wilkins, 2010). In this model, QLT represents a second-order factor that accounts for the interrelationship among three first-order factors: mathematical beliefs, mathematical disposition, and mathematical cognition. Using data from two samples of undergraduate students (n = 186 and n = 184), a series of confirmatory factor analyses were conducted. Results supported the hierarchical three-factor structure and confirmed its factorial invariance across multiple groups of students. Latent QLT scores from the model were used to compare and rank student QLT by gender and class standing.
TL;DR: In this paper, the effect of complex structure on dimensionality assessment in compensatory multidimensional item response models using DETECT- and NOHARM-based methods was investigated via the accuracy of identifying the correct number of dimensions and the ability to accurately recover item groupings using a simple matching similarity (SM) coefficient.
Abstract: This study investigated the effect of complex structure on dimensionality assessment in compensatory multidimensional item response models using DETECT- and NOHARM-based methods. The performance was evaluated via the accuracy of identifying the correct number of dimensions and the ability to accurately recover item groupings using a simple matching similarity (SM) coefficient. The DETECT-based methods yielded higher proportion correct than the NOHARM-based methods in two- and three-dimensional conditions, especially when correlations were ≤.60, data exhibited ≤30% complexity, and sample size was 1,000. As the complexity increased and the sample size decreased, the performance of the methods typically diminished. The NOHARM-based methods were either equally successful or better in recovering item groupings than the DETECT-based methods and were mostly affected by complexity levels. The DETECT-based methods were affected largely by the test length, such that with the increase of the number of items, SM coef...