Daniel McGinn

Bio: Daniel McGinn is an academic researcher from Harvard University. The author has contributed to research in topics: Functor & Structure (category theory). The author has an hindex of 5, co-authored 6 publications receiving 92 citations. Previous affiliations of Daniel McGinn include University of Wisconsin-Madison.

Validating Arguments for Observational Instruments: Attending to Multiple Sources of Variation

Abstract: Measurement scholars have recently constructed validity arguments in support of a variety of educational assessments, including classroom observation instruments. In this article, we note that users must examine the robustness of validity arguments to variation in the implementation of these instruments. We illustrate how such an analysis might be used to assess a validity argument constructed for the Mathematical Quality of Instruction instrument, focusing in particular on the effects of varying the rater pool, subject matter content, observation procedure, and district context. Variation in the subject matter content of lessons did not affect rater agreement with master scores, but the evaluation of other portions of the validity argument varied according to the composition of the rater pool, observation procedure, and district context. These results demonstrate the need for conducting such analyses, especially for classroom observation instruments that are subject to multiple sources of variation.

Approximate measurement invariance in cross-classified rater-mediated assessments.

Abstract: An important assumption underlying meaningful comparisons of scores in rater-mediated assessments is that measurement is commensurate across raters. When raters differentially apply the standards established by an instrument, scores from different raters are on fundamentally different scales and no longer preserve a common meaning and basis for comparison. In this study, we developed a method to accommodate measurement noninvariance across raters when measurements are cross-classified within two distinct hierarchical units. We conceptualized random item effects cross-classified graded response models and used random discrimination and threshold effects to test, calibrate, and account for measurement noninvariance among raters. By leveraging empirical estimates of rater-specific deviations in the discrimination and threshold parameters, the proposed method allows us to identify noninvariant items and empirically estimate and directly adjust for this noninvariance within a cross-classified framework. Within the context of teaching evaluations, the results of a case study suggested substantial noninvariance across raters and that establishing an approximately invariant scale through random item effects improves model fit and predictive validity.

Mathematical content knowledge and knowledge for teaching: exploring their distinguishability and contribution to student learning

Abstract: During the last three decades, scholars have proposed several conceptual structures to represent teacher knowledge. A common denominator in this work is the assumption that disciplinary knowledge and the knowledge needed for teaching are distinct. However, empirical findings on the distinguishability of these two knowledge components, and their relationship with student outcomes, are mixed. In this replication and extension study, we explore these issues, drawing on evidence from a multi-year study of over 200 fourth- and fifth-grade US teachers. Exploratory and confirmatory factor analyses of these data suggested a single dimension for teacher knowledge. Value-added models predicting student test outcomes on both state tests and a test with cognitively challenging tasks revealed that teacher knowledge positively predicts student achievement gains. We consider the implications of these findings for teacher selection and education.

Kripke submodels and universal sentences

Abstract: We define two notions for intuitionistic predicate logic: that of a submodel of a Kripke model, and that of a universal sentence We then prove a corresponding preservation theorem If a Kripke model is viewed as a functor from a small category to the category of all classical models with (homo)morphisms between them, then we define a submodel of a Kripke model to be a restriction of the original Kripke model to a subcategory of its domain, where every node in the subcategory is mapped to a classical submodel of the corresponding classical model in the range of the original Kripke model We call a sentence universal if it is built inductively from atoms (including ⊤ and ⊥) using ∧, ∨, ∀, and →, with the restriction that antecedents of → must be atomic We prove that an intuitionistic theory is axiomatized by universal sentences if and only if it is preserved under Kripke submodels We also prove the following analogue of a classical model-consistency theorem: The universal fragment of a theory Γ is contained in the universal fragment of a theory Δ if and only if every rooted Kripke model of Δ is strongly equivalent to a submodel of a rooted Kripke model of Γ Our notions of Kripke submodel and universal sentence are natural in the sense that in the presence of the rule of excluded middle, they collapse to the classical notions of submodel and universal sentence (© 2007 WILEY-VCH Verlag GmbH & Co KGaA, Weinheim)

Quantifier Elimination for a Class of Intuitionistic Theories

Abstract: From classical, Fraisse-homogeneous, (≤ ω)-categorical theories over finite relational languages (which we refer to as JRS theories), we construct intuitionistic theories that are complete, prove negations of classical tautologies, and admit quantifier elimination. The technique we use considers Kripke models as functors from a small category to the category of L-structures with morphisms, rather than the usual interpretation wherein the frame of a Kripke model is a partial order. While one can always “unravel” a functor Kripke model to obtain a partial order Kripke model with the same intuitionistic theory, our technique is perhaps an easier way to consider a Kripke model that includes a single classical node structure and all of the endomorphisms of that classical JRS structure. We also determine the intuitionistic universal fragments of these theories, in accordance with the hierarchy of intuitionistic formulas put forth in [9] and expounded on by Fleischmann in [11]. This portion of the thesis (Chapter 1) is the result of joint work with Ben Ellison, Jonathan Fleischmann, and Wim Ruitenburg, as published (up to minor structural changes) in [10]. Given a classical JRS theory, we determine axiomatizations of the corresponding intuitionistic theory in Chapter 2. We first do so by axiomatizing properties apparent from the behavior of the model, and discuss improvements to that axiom system. We then present another axiomatization, this time by axiomatizing the properties of quantifier elimination. We discuss improvements to this system, and show how this system and various subsystems thereof are equivalent to our first axiomatization and corresponding subsystems thereof.

Standards for educational and psychological testing

Abstract: Pedagosko i psiholosko testiranje i procjenjivanje spadaju među najvažnije doprinose znanosti o ponasanju nasem drustvu i pružaju temeljna i znacajna poboljsanja u odnosu na ranije postupke. Iako se ne može ustvrditi da su svi testovi dovoljno usavrseni niti da su sva testiranja razborita i korisna, postoji velika kolicina informacija koje ukazuju na ucinkovitost kvalitetnih instrumenata u onim situacijama u kojima je njihovo koristenje potkrijepljeno validacijskim podacima. Pravilna upotreba testova može dovesti do boljih odluka o pojedincima i programima nego sto bi to bio slucaj bez njihovog koristenja, a također i ukazati na put za siri i pravedniji pristup obrazovanju i zaposljavanju. Međutim, losa upotreba testova može dovesti do zamjetne stete nanesene ispitanicima i drugim sudionicima u procesu donosenja odluka na temelju testovnih podataka. Cilj Standarda je promoviranje kvalitetne i eticne upotrebe testova te uspostavljanje osnovice za ocjenu kvalitete postupaka testiranja. Svrha objavljivanja Standarda je uspostavljanje kriterija za evaluaciju testova, provedbe testiranja i posljedica upotrebe testova. Iako bi evaluacija prikladnosti testa ili njegove primjene trebala ovisiti prvenstveno o strucnim misljenjima, Standardi pružaju okvir koji osigurava obuhvacanje svih relevantnih pitanja. Bilo bi poželjno da svi autori, sponzori, nakladnici i korisnici profesionalnih testova usvoje Standarde te da poticu druge da ih također prihvate.

An Interaction-Based Approach to Enhancing Secondary School Instruction and Student Achievement.

Abstract: Improving teaching quality is widely recognized as critical to addressing deficiencies in secondary school education, yet the field has struggled to identify rigorously evaluated teacher-development approaches that can produce reliable gains in student achievement. A randomized controlled trial of My Teaching Partner–Secondary—a Web-mediated approach focused on improving teacher-student interactions in the classroom—examined the efficacy of the approach in improving teacher quality and student achievement with 78 secondary school teachers and 2237 students. The intervention produced substantial gains in measured student achievement in the year following its completion, equivalent to moving the average student from the 50th to the 59th percentile in achievement test scores. Gains appeared to be mediated by changes in teacher-student interaction qualities targeted by the intervention.

Editorial: Measurement Invariance.

Abstract: Multi-item surveys are frequently used to study scores on latent factors, like human values, attitudes, and behavior. Such studies often include a comparison, between specific groups of individuals or residents of different countries, either at one or multiple points in time (i.e., a cross-sectional or a longitudinal comparison or both). If latent factor means are to be meaningfully compared, the measurement structures of the latent factor and their survey items should be stable, that is “invariant.” As proposed by Mellenbergh (1989), “measurement invariance” (MI) requires that the association between the items (or test scores) and the latent factors (or latent traits) of individuals should not depend on group membership or measurement occasion (i.e., time). In other words, if item scores are (approximately) multivariate normally distributed, conditional on the latent factor scores, the expected values, the covariances between items, and the unexplained variance unrelated to the latent factors should be equal across groups. Many studies examining MI of survey scales have shown that the MI assumption is very hard to meet. In particular, strict forms of MI rarely hold. With “strict” we refer to a situation in which measurement parameters are exactly the same across groups or measurement occasions, that is an enforcement of zero tolerance with respect to deviations between groups or measurement occasions. Often, researchers just ignore MI issues and compare latent factor means across groups or measurement occasions even though the psychometric basis for such a practice does not hold. However, when a strict form of MI is not established and one must conclude that respondents attach different meanings to survey items, this makes it impossible to make valid comparisons between latent factor means. As such, the potential bias caused by measurement non-invariance obstructs the comparison of latent factor means (if strict MI does not hold) or regression coefficients (if less strict forms of MI do not hold). Traditionally, MI is tested for in a multiple group confirmatory factor analysis (MGCFA) with groups defined by unordered categorical (i.e., nominal) between-subject variables. In MGCFA, MI is tested at each constraint of the latent factor model using a series of nested (latent) factor models. This traditional way of testing for MI originated with Joreskog (1971), who was the first scholar to thoroughly discuss the invariance of latent factor (or measurement) structures. Additionally, Sorbom (1974, 1978) pioneered the specification and estimation of latent factor means using a multi-group SEM approach in LISREL (Joreskog and Sorbom, 1996). Following these contributions the multi-group specification of latent factor structures has become widespread in all major SEM software programs (e.g., AMOS Arbuckle, 2006, EQS Bender and Wu, 1995, LAVAAN Rosseel, 2012, Mplus Muthen and Muthen, 2013, STATA STATA, 2015, and OpenMx Boker et al., 2011). Shortly thereafter, Byrne et al. (1989) introduced the distinction between full and partial MI. Although their introduction was of great value, the first formal treatment of different forms of MI and their consequences for the validity of multi-group/multi-time comparisons is attributable to Meredith (1993). So far, a tremendous amount of papers dealing with MI have been published. The literature on MI published in the 20th century is nicely summarized by Vandenberg and Lance (2000). Noteworthy is also the overview of applications in cross-cultural studies provided by Davidov et al. (2014), as well as a recent book by Millsap (2011) containing a general systematic treatment of the topic of MI. The traditional MGCFA approach to MI-testing is described by, for example, Byrne (2004), Chen et al. (2005), Gregorich (2006), van de Schoot et al. (2012), Vandenberg (2002) and Wicherts and Dolan (2010). Researchers entering the field of MI are recommended to first consult Meredith (1993) and Millsap (2011) before reading other valuable academic works. Recent developments in statistics have provided new analytical tools for assessing MI. The aim of this special issue is to provide a forum for a discussion of MI, covering some crucial “themes”: (1) ways to assess and deal with measurement non-invariance; (2) Bayesian and IRT methods employing the concept of approximate MI; and (3) new or adjusted approaches for testing MI to fit increasingly complex statistical models and specific characteristics of survey data.