Although the belief has been expressed that performance assessments are intrinsically more fair than multiple-choice measures, some forms of performance assessment may in fact be more likely than conventional tests to tap construct-irrelevant factors. The assessment of differential item functioning (DIF) can be helpful in investigating the effect on subpopulations of the introduction of performance tasks. In this study, two extensions of the Mantel-Haenszel (MH; 1959) procedure that may be useful in assessing DIP in performance measures were explored. The test of conditional association proposed by Mantel (1963) seems promising as a test of DIF for pofytomous items when the primary interest is in the between-group difference in item means, conditional on some measure of ability. The generalized statistic proposed by Mantel and Haenszel may be more useful than Mantel's test when the entire response distributions of the groups are of interest. Simulation results showed that, for both inferential procedures, the studied item should be included in the matching variable, as in the dichotomous case. Descriptive statistics that index the magnitude of DIP were also investigated.

Assessment of Differential Item Functioning for Performance Tasks

Liu and Agresti (1996) proposed a Mantel and Haenszel-type (1959) estimator of a common odds ratio for several 2 × J tables, where the J columns are ordinal levels of a response variable. This article applies the Liu-Agresti estimator to the case of assessing differential item functioning (DIF) in items having an ordinal response variable. A simulation study was conducted to investigate the accuracy of the Liu-Agresti estimator in relation to other statistical DIF detection procedures. The results of the simulation study indicate that the Liu-Agresti estimator is a viable alternative to other DIF detection statistics.

Applying the Liu-Agresti Estimator of the Cumulative Common Odds Ratio to DIF Detection in Polytomous Items

Many statistics used in the assessment of differential item functioning (DIF) in polytomous items yield a single item-level index of measurement invariance that collapses information across all response options of the polytomous item. Utilizing a single item-level index of DIF can, however, be misleading if the magnitude or direction of the DIF changes across the steps underlying the polytomous response process. A more comprehensive approach to examining measurement invariance in polytomous item formats is to examine invariance at the level of each step of the polytomous item, a framework described in this article as differential step functioning (DSF). This article proposes a nonparametric DSF estimator that is based on the Mantel-Haenszel common odds ratio estimator (Mantel & Haenszel, 1959), which is frequently implemented in the detection of DIF in dichotomous items. A simulation study demonstrated that when the level of DSF varied in magnitude or sign across the steps underlying the polytomous response options, the DSF-based approach typically provided a more powerful and accurate test of measurement invariance than did corresponding item-level DIF estimators.

Assessing Differential Step Functioning in Polytomous Items Using a Common Odds Ratio Estimator.

Context

Over nearly two decades, a wealth of literature describing the various capabilities, uses and adaptations of virtual microscopy (VM) has been published. Many studies have investigated the effects on and benefits to student learning of VM compared with optical microscopy (OM).



Objectives

This study statistically aggregated the findings of multiple comparative studies through a meta-analysis in order to summarise and substantiate the pedagogical efficacy of teaching with VM.



Methods

Using predefined eligibility criteria, teams of paired researchers screened the titles and abstracts of VM studies retrieved from seven different databases. After two rounds of screening, numerical and thematic data were extracted from the eligible studies for analysis. A summary effect size and estimate of heterogeneity were calculated to determine the effects of VM on learner performance and the amount of variance between studies, respectively. Trends in student perceptions were also analysed and reported.



Results

Of the 725 records screened, 72 studies underwent full-text review. In total, 12 studies were viable for meta-analysis and additional studies were reviewed to extract themes relating to learners’ perceptions of VM. The meta-analysis detected a small yet significant positive effect on learner performance (standardised mean difference 0.28, 95% confidence interval 0.09–0.47; p = 0.003), indicating that learners experience marked knowledge gains when exposed to VM over OM. Variation among studies was evident as high heterogeneity was reported. An analysis of trends in learner perceptions noted that respondents favoured VM over OM by a large margin.



Conclusions

Although many individual studies have reported non-significant findings in comparisons of VM and OM, the enhanced power afforded by meta-analysis revealed that the pedagogical approach of VM is modestly superior to that of OM and is preferred by learners.

Meta-analysis and review of learner performance and preference: virtual versus optical microscopy.

Mantel-Haenszel methods comprise a highly flexible methodology for assessing the degree of association between two categorical variables, whether they are nominal or ordinal, while controlling for ...

Generalized Mantel-Haenszel Methods for Differential Item Functioning Detection

A conditional test for the homogeneity of the odds ratios in L2 × K (K > 2) tables is developed based on a multiple hypergeometric distribution. The test is a generalization of the Breslow-Day test (1980) and is useful in sparse tables. For computational feasibility, the conditional maximum likelihood estimator is replaced by a consistent but inefficient estimator in the test statistic. If this is the case, it is shown that the asymptotic distribution is not chi squared, as has been noticed by Tarone (1985) in the case of 2 × 2 tables. Following Tarone we develop a correction method so that the statistic follows an asymptotic chi-squared distribution. An algorithm is developed for calculating the homogeneity test with a generalized Mantel-Haenszel estimator.

Homogeneity Test with a Generalized Mantel-Haenszel Estimator for L2 × K Contingency Tables

A projection method is introduced to generalize the Mantel-Haenszel estimator for estimating the common odds ratios in K 2 × J (J > 2) tables. The method provides estimators in an explicit form and needs no iterative computation. It is shown that the generalized Mantel-Haenszel estimator proposed by Greenland, and Yanagawa and Fujii is obtained by this method as a special case. Furthermore, selecting appropriate weight, it is shown that the method leads to an estimator that is asymptotically equivalent to the generalized Mantel-Haenszel estimator proposed by Liang. A dually consistent estimator of the variance of the estimator is proposed. The characteristics of these estimators have been evaluated by Monte Carlo means.

Projection-Method Mantel-Haenszel Estimator for K 2 × J Tables

Generalization of Mantel-Haenszel procedure for 2 x J (J > 2) tables is reviewed. Included are generalized Mantel-Haenszel tests, estimators for a common odds ratio, and generalized Breslow-Day test for the homogeneity of odds ratios across the strata.-Environ Health Perspect 102(Suppl 8): 57-60 (1994)

Generalized Mantel-Haenszel procedures for 2 x J tables.

This paper considers the faivariate survival model that given. Z=z (a common random variable), X and Y independently follow the weibull distribution with hazard functions β1tr and β2tr, respectively, where β1, β2 and r are unknown parameters. We propose a test for testing H0: β1 = β2 v.s. H1: β1 ≠ β2. The test is the optimal C(α)-test. The power of the test is compared with the test proposed by Tapan (1988) by simulation study. Also we examine the behaviors of the tests using practical data from reliability.

Test for the Equality of Marginal Distributions on Positively Dependent Bivariate Survival Data

This paper reviews dependence models for bivariate survival data, classifying them into the four groups: the shock model, the Freund model, the Clayton model, and the mixture model. The paper then concentrates on the mixture model, discussing the testing problem for the equality of marginal distributions under the Weibull type baseline hazard assumption. The new test proposed recently by Fujii is introduced, and its characteristic is studied with respect to the test proposed by Nayak and the sign test by simulation study.

Yoshinori Fujii

Papers

Homogeneity Test with a Generalized Mantel-Haenszel Estimator for L2 × K Contingency Tables

Projection-Method Mantel-Haenszel Estimator for K 2 × J Tables

Generalized Mantel-Haenszel procedures for 2 x J tables.

Test for the Equality of Marginal Distributions on Positively Dependent Bivariate Survival Data

Models for association in bivariate survival data.