Preface Chapter 1 The challenge Chapter 2 The nature of the evidence: A synthesis of meta-analyses Chapter 3 The argument: Visible teaching and visible learning Chapter 4: The contributions from the student Chapter 5 The contributions from the home Chapter 6 The contributions from the school Chapter 7 The contributions from the teacher Chapter 8 The contributions from the curricula Chapter 9 The contributions from teaching approaches - I Chapter 10 The contributions from teaching approaches - II Chapter 11: Bringing it all together Appendix A: The 800 meta-analyses Appendix B: The meta-analyses by rank order References

Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement

An account is given of a recently devised method of prediction based on sample reuse techniques. It is most useful in low structure data paradigms that involve minimal assumptions. A series of applications demonstrating the technique is presented.

The Predictive Sample Reuse Method with Applications

This discussion paper argues that both the use of Cronbach’s alpha as a reliability estimate and as a measure of internal consistency suffer from major problems. First, alpha always has a value, which cannot be equal to the test score’s reliability given the interitem covariance matrix and the usual assumptions about measurement error. Second, in practice, alpha is used more often as a measure of the test’s internal consistency than as an estimate of reliability. However, it can be shown easily that alpha is unrelated to the internal structure of the test. It is further discussed that statistics based on a single test administration do not convey much information about the accuracy of individuals’ test performance. The paper ends with a list of conclusions about the usefulness of alpha.

/pdf/on-the-use-the-misuse-and-the-very-limited-usefulness-of-3b1f8w7ttm.pdf

On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha

https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/j.2517-6161.1972.tb00885.x

Bayes Estimates for the Linear Model

SUMMARY Two simple estimators are derived here for the means of the random effect model by means of predictive sample reuse. They are applied to two sets of data in the literature and compared with several other procedures. The mixed model is also discussed.

/pdf/a-predictive-approach-to-the-random-effect-model-2l1b0ysgjd.pdf

A predictive approach to the random effect model

Statistical methods for educational and psychological research

Let Σ
 x
 be the (population) dispersion matrix, assumed well-estimated, of a set of non-homogeneous item scores. Finding the greatest lower bound for the reliability of the total of these scores is shown to be equivalent to minimizing the trace of Σ
 x
 by reducing the diagonal elements while keeping the matrix non-negative definite. Using this approach, Guttman's bounds are reviewed, a method is established to determine whether his λ4 (maximum split-half coefficient alpha) is the greatest lower bound in any instance, and three new bounds are discussed. A geometric representation, which sheds light on many of the bounds, is described.

Lower bounds for the reliability of the total score on a test composed of non-homogeneous items: I: Algebraic lower bounds

Finding the greatest lower bound for the reliability of the total score on a test comprisingn non-homogenous items with dispersion matrix Σ
 x
 is equivalent to maximizing the trace of a diagonal matrix Σ
 E
 with elements θ
 I
 , subject to Σ
 E
 and Σ
 T
 =Σ
 x
 − Σ
 E
 being non-negative definite. The casesn=2 andn=3 are solved explicity. A computer search in the space of the θ
 i
 is developed for the general case. When Guttman's λ4 (maximum split-half coefficient alpha) is not the g.l.b., the maximizing set of θ
 i
 makes the rank of Σ
 T
 less thann − 1. Numerical examples of various bounds are given.

Lower bounds for the reliability of the total score on a test composed of non-homogeneous items: II: A search procedure to locate the greatest lower bound

An improved Bayesian method, due to Lindley, for the simultaneous estimation of multiple regressions in m groups is studied by applying the method to test data. Evidence is found to support the belief that in many testing applications the collateral information obtained from each subset of m − 1 colleges will be useful for estimating the regression in the mth college, especially when the sample sizes are small. Using a 25 per cent sample, the Bayesian prediction equations achieved an average 9.7 per cent reduction in mean square error, as compared with the within-group least squares equations, when cross-validated with a later sample. More importantly, the mean square error for the Bayesian equations based on the 25 per cent sample was only barely greater than that for the least squares equations based on the full sample data. Thus the main virtue of the method is that it permits predictions to be made separately for relevant subpopulations (e.g. male-female) where sample sizes would otherwise be too small to achieve an acceptable degree of accuracy.

ESTIMATING MULTIPLE REGRESSIONS IN m GROUPS: A CROSS‐VALIDATION STUDY

In many applications, it is desirable to estimate binomial proportions inm groups where it is anticipated that these proportions are similar but not identical. Following a general approach due to Lindley, a Bayesian Model II aposteriori modal estimate is derived that estimates the inverse sine transform of each proportion by a weighted average of the inverse sine transform of the observed proportion in the individual group and the average of the estimated values. Comparison with a classical method due to Jackson spotlights some desirable features of Model II analyses. The simplicity of the present formulation makes it possible to study the behavior of the Bayesian Model II approach more closely than in more complex formulations. Also, it is possible to estimate the amount of gain afforded by the Model II analyses.

Paul H. Jackson

Papers

Statistical methods for educational and psychological research

Lower bounds for the reliability of the total score on a test composed of non-homogeneous items: I: Algebraic lower bounds

Lower bounds for the reliability of the total score on a test composed of non-homogeneous items: II: A search procedure to locate the greatest lower bound

ESTIMATING MULTIPLE REGRESSIONS IN m GROUPS: A CROSS‐VALIDATION STUDY

The estimation of proportions inm groups