/pdf/educational-testing-service-fjdwd2v4ls.pdf

Educational Testing Service

Multiple matrix sampling is a psychometric procedure in which a set of test-items is subdivided randomly into subtests of items with each subtest administered to different subgroups of examinees selected at random from the examinee population. Although each examinee rec, only a proportion of the complete set of items, the statistical model employed permits the researcher to estimate the mean, variance and frequency distribution of test scores which would have been obtained by testing all examinees on all items. Contained herein is a detailed description of multiple matrix sampling. The topics covered range from an introductory discussion to the listing with expanded writeup of the computer program used to analyze the data. Throughout this Report an attempt has been made to keep the practitioner clearly in mind.

/pdf/principles-and-procedures-of-multiple-matrix-sampling-5588vnoosw.pdf

Principles and procedures of multiple matrix sampling.

The demand for sophisticated and compre­ hensive evaluation of educational efforts is affecting thinking about measurement techniques and strategies. Evaluation for many years has been equated with a process of determining whether specified objectives are attained, but current evaluation models focus on a larger number of phenomena. The objectives of an educational effort are still an important component of evaluation, but the current models are more inclusive. Evaluation theorists indicate that evaluation should attend to outcomes other than specified objectives, to inputs or antecedent conäitions, and to processes or transactions. The implementation of an input-process-outcome evaluation plan raises important measurement problems. The inclusion of the many variables in a comprehensive evaluation requires a massive amount of measurement and classification. There are also problems associated with obtaining valid and reliable measurement and classifications of a great many variables, including many not traditionally considered in evaluation methodology.

6: Measurement Techniques In Evaluation

sampling&dquo; may be considered as one term for a class of operations called matrix sampling. Types of matrix sampling may include the sampling of items from a population, persons from a population (finite population sampling), or sampled matrices reflecting different combinations of persons and items. Osbum (1967) has recently shown a relationship of matrix or item sampling to the concept of unmatched data samples in the terminology of generalizability developed by Cronbach, Rajaratnam, and Gleser (1963). Studies dealing with item sampling have been reported by Cook

The Estimation of Mean Achievement Scores for Schools by the Item-Sampling Technique

A universe defined test is a test constructed and administered in such a way that an examinee’s score on the test provides an unbiased estimate of his score on some explicitly defined universe of item content (Osburn, 1968). Two general requirements for test construction are implied in the above definition. The first is that all items which could possibly appear in the test should be specified in advance; this implies that for a specified content area it is possible to clearly define a content population of items. Secondly, the items in a particular test should be selected by random sampling or stratified random sampling from the universe (or population) of items defining the content area. The idea of random sampling from a universe of test items is not new. Psychometric theorists have used this idea extensively in building reliability and other testing models. The point is that few test constructors have seriously attempted to implement the model in actual practice. Osburn (1968), however, has defined a procedure for implementing in practice the universe defined test concept. The focus of the present paper is not an elaboration of the implications of the universe defined test for test construction

Computer-Aided Item Sampling for Achievement Testing: A Description of a Computer Program Implementing the Universe Defined Test Concept.

THE theory of generslizability distinguishes between experiments in which items are matched and experiments in which items are unmatched (Cronbach, Rajaratnam and Gleser, 1963). These writers have shown that the unmatched item design leads to different reliability formulas for estimating the generalizability of a test. This paper shows that the unmatched item experiment has certain advantages over the usual matched item experiment depending upon the objectives of the study. Items are said to be matched if for every item in the test there is an zpr for every person in the sample of examinees. Items are said to be unmatched when the items are selected randomIy and independently for each person. Most studies using psychological tests have used the matched item design, and it might seem that unmatched item designs would be something to avoid. This is definitely not the case. It turns out that unmatched item designs have important applications in test experiments as will be shown below. Suppose tha t we are sampling from a population of persons, P, and a population of items, I , and we wish to generalize over both populations. Consider the following experiments.

H. G. Osburn

Papers

A Note on Design of Test Experiments