Showing papers in "Educational and Psychological Measurement in 1969"

A Procedure for Sample-Free Item Analysis

Abstract: Omt purpose is to describe in detail a convenient procedure for performing a new kind of item analysis. This new item analysis is different in a vital way from that described in textbooks like Gulliksen’s Theory of Mental Tests and used in computing programs like TSSA2. The difference is that (a) test calibrations are independent of the sample of persons used to estimate item parameters, and (b) person measurements, the transformation of test scores into estimates of person ability, are independent of the selection of items used to obtain test scores.

Measures of Achieving Tendency

Abstract: MEHRABIAN (1968) reported findings relating to scales of achievement constructed for males and females. The scales were designed to discriminate high achievers (i.e., individuals whose motive to achieve is stronger than their motive to avoid failure) from low achievers (i.e., individuals whose motive to avoid failure is stronger than their motive to achieve). The present study provides additional validity data relating to revised versions of these scales. In this study the revised scales, which consisted of 26 items each, were administered to subjects along with measures of affiliation, achievement, social desirability, test anxiety, dogmatism, and neuroticism. In addition, a series of questions were administered to provide information about developmental antecedents of high versus low achieving tendencies and socialization patterns. The following hypotheses were based on existing findings in the area of achievement, such as those reviewed by Byrne (1966). It was hypothesized that high achievers exhibit less test anxiety,

Note on a Criterion for the Number of Common Factors.

Abstract: LINN (1968) has published the results of a, Monte Carlo approach to the number of factors problem involving the use of random normal deviates as additional variables in correlational matrices from which principal components are extracted. Horn (1966) has made use of an independent component analysis of random normal deviates, parallel to the analysis of the &dquo;real&dquo; variables, in order to correct the Kaiser criterion (see Horn) which assumes population values of the correlations, for capitalization on chance in the sample. This note briefly presents some results obtained from another, and more promising, variation on their procedures. Linn found that his mean square ratios, analogous to F-ratios with data from the latent roots of real variables in the numerator and of random variables in the denomination, fluctuated widely

Capitalization on Chance in Rotation of Factors.

Abstract: FACTOR analysis is used for two different theoretical purposes: to search for structure among correlated measures for purposes of theory construction, and to test hypotheses, derived perhaps from previous factor analyses, about measures. For purposes of the first sort a variety of computer programs for both oblique and orthogonal rotation of factors is available. Visually guided rotations are also still used. For purposes of the second sort visually guided rotations are also used, but either orthogonal or oblique rotations by computer to a target matrix is a typical procedure. The question arises with respect to either of these theoretical purposes, and to either visually guided or computer rotations, as to the extent to which the investigator can capitalize on chance. In multiple regression, for example, the investigator capitalizes on chance to an extent which is a function of the number of variables

Expected Reliability as a Function of Choices Per Item

Abstract: BY making several reasonable assumptions, it is possible to estimate the reliability that should be expected in an objective test as a function of the number of choices per item. These estimates for a 100 item objective test are presented in Table 1. They indicate an appreciable increase in reliability when the number of choices per item is increased from two to three; a smaller increase in going to four choice items; and still smaller increases beyond that point. They indicate, too, that a 100 item true-false test yielding a reliability coefficient of .74 is a reasonably good test, as true-false tests go, whereas a reasonably good 100 item test made up of four-choice items ought to yield a reliability coefficient of .86. The same assumptions can be used to answer a different question: how many two-choice, three-choice, etc. items must be included in

Package: a System of Computer Routines for the Analysis of Correlational Data:

Abstract: IN this period of large capacity, high-speed computers, there can be found at almost every computer installation a program library containing an assortment of routines to analyze correlational data. However, few offer a program system composed of several, linked routines. And usually either the system is not transportable because some of its elements are written in machine or assembly code or the system consists of a complex group of control cards that prevent the novice from fully utilizing it. Moreover, modifying programs within the system or expanding the program selection requires a large investment of time, money, and energy. The present writers believed a system of correlational analysis programs could be developed that was (1) transportable from installation to installation, (2) made up of a relatively simple set of

An Alternative Factor Analytic Solution for Wechsler's Intelligence Scales.

Abstract: FOR a factor analytic solution to offer maximal promise of predictive utility, Peterson (1960, 1965) maintains that it must display at least two properties: descriptive efficiency and statistical invariance. He has demonstrated a relationship between these two properties in the domain of personality, using data drawn from ratings and questionnaires. Over many comparisons of alternative solutions, those with only a few broad factors consistently showed greater invariance than those with many narrow ones. The purpose of the present study was to determine whether this relationship also holds in the intellectual domain, as sampled by Wechsler’s (1949, 1955, 1967) scales. Cohen’s (1957a, 1957b, 1959) factor analytic studies of the standardization data for seven age groups on the WAIS and the WISC have become classics in the field. He factored the subtest intercor-

Overall Measures of Self-Actualization Derived From the Personal Orientation Inventory

Abstract: THE Personal Orientation Inventory (POI) was designed by Everett L. Shostrom (1964, 1966) as a group test for measuring self-actualization. Its validity is based primarily upon clinically judged self-actualized vs. non-self-actualized subjects (Shostrom, 1966). Other studies report the test’s ability to differentiate between such things as preand post-sensitivity training (Shostrom, 1964), stages of psychotherapy involvement (Shostrom and Knapp, 1966), levels of performance on a neuroticism inventory (Knapp, 1965), and achieving vs. underachieving college freshmen (LeMay and Damm, 1968). The test consists of 150 two-choice (paired opposites) comparative value judgments which can be broken down into the following scales: Inner Directed (I), Time Competent (Tc), Self-Actualizing Value (SAV), Existentiality (Ex), Feeling Reactivity (Fr), Spontaneity (S), Self Regard (Sr) , Self Acceptance (Sa), Nature of Man (Nc), Synergy (Sy), Acceptance of Aggression (A), and Capacity for Intimate Contact (C). Although differential profile patterns have been demonstrated to allow for predictive judgments, it has not yet been demonstrated which of a number of alternative methods may provide the best

Effects of Promised Reward and Threatened Penalty on Performance of a Multiple-Choice Vocabulary Test

Abstract: IN research on multiple-choice tests some attention has been paid to the effect on test performance of instructions about guessing. The focus of this attention has been on instructions of two types. The first encourages the examinee to guess when he does not know an answer and informs him that his score will be the number of correct answers. The second encourages the examinee not to guess by telling him there is a penalty for wrong answers and that his score will be the number of correct answers minus some fraction of the number of wrong answers. Research comparing the effects of these two types of instructions on test behavior has been reported by Blommers and Lindquist (1965), Keislar (1953), Michael, Stewart, Douglass, and Rainwater (1963), Ruch and De Graff (1926), and Swineford and Miller (1953). A consistent finding of these studies is that instructions containing the threat of a penalty for wrong answers reduces the number of questions attempted to a level significantly below the number attempted under instructions to guess unknown answers. Associated with this result is the observation that the number of

Generality of Risk Taking on Objective Examinations.

Abstract: RISK taking on objective examinations (RTOOE) is defined as guessing when the examinee is aware that there is a penalty for incorrect responses (Slakter, 1967b). There is evidence that RTOOE is related to behaviors such as dominance-submission (Votaw, 1936), maladjustment (Sheriffs and Boomer, 1954), vocational choice (Ziller, 1957b), and perception of risk in military situations (Torrance and Ziller, 1957). Since RTOOE measures can be obtained from Ss ostensibly taking achievement or aptitude tests, it would appear that RTOOE provides psychologists with a potentially useful disguised measure of risk taking. In addition, there are indications that Ss low in RTOOE tend to be penalized on their test score (Sherriffs and Boomer, 1954; Slakter, 1968; Votaw, 1936). These studies have demonstrated that when Ss who are low in RTOOE are forced to respond to all test items, their average test score will increase even though the usual penalty for guessing is applied. Since these findings provide evidence that RTOOE confounds the achievement or aptitude being measured by the examination, RTOOE becomes of interest to individuals concerned with educational measurement.

The Development and Evaluation of Several Programmed Testing Methods.

Abstract: Seven programmed test procedures were developed and investigated using existing data on 190 verbal-type items for 4,885 students. Tests were scored as if the student had taken items in the order implied by the test. Shortened conventional tests were also scored for comparative purposes. The criteria against which these procedures were evaluated were the 190-item total test score and four outside criterion tests. Against the criterion of reproducing the 190-item total test score in the cross-validation sample the programmed tests were found to he only slightly superior to the shortened conventional tests. However, the programmed tests had correlations with the outside criterion tests that were substantially higher than the corresponding shortened conventional tests. It was estimated that a test which was parallel to the 190-item total test would have to be 3.36 times as long as the best programmed test to have an equal median correlation with the outside criterion tests.

The Scrambled Sentence Test: A Group Measure of Hostility.

Abstract: THIS paper describes the development of a disguised group test of hostility. It was derived from Watson, Pritzker, and Madison’s (1955.) individual test, which was administered by projecting on a screen a set of four words arranged in a scrambled order. The subject was asked to assemble a three-word sentence from this set, his response being scored as either &dquo;hostile&dquo; or &dquo;neutral.&dquo; For example, if from the scrambled set &dquo;Shoot I’ll you ask,&dquo; the subject chose &dquo;I’ll shoot you,&dquo; his response would be &dquo;hostile&dquo;; if he constructed the sentence &dquo;I’ll ask you,&dquo; his response would be &dquo;neutral.&dquo; The entire test consisted of 60 sets of scrambled words, each set presented one at a time, in rapid succession. All responses were sound recorded, and scored at the conclusion of the test; the subject’s total score was the number of hostile sentences he had assembled. Assuming that their test favored the expression of &dquo;repressed&dquo; impulses, Watson and his colleagues hypothesized that neurotics would reveal more hostility than &dquo;normal&dquo; individuals. The hypothesis was confirmed: the mean hostility score of neurotic patients was significantly higher than that of a control group of patients who had not sought treatment. Subsequently, this scrambled sentence task was adapted for use in several experimental studies of operant conditioning (Anderson, 1958; Scott, 1958), and also has been employed in a number of

Item Factor Structure of the Adjective Check List

Abstract: As one of the frequently used instruments for assessment of self-concept through self-description, the Adjective Check List (ACL) has recognized merit in comparison with other similar instruments. Existing scales, as developed by Gough and Heilbrun (1965), provide data regarding behavioral tendencies that may be useful for research as well as for diagnostic and counseling purposes (e.g., Heilbrun, 1960, 1961; MacKinnon, 1963; Parker, 1967). It is becoming clear in psychological research that a thorough investigation of dimensions of test behavior related to such variables as the self-concept is essential (Foa, 1961; Loehlin, 1961; Briar and Bieri, 1963; Scarr, 1966; Parker and Megargee, 1967). Interest has not been lacking, but empirical analysis of the item factor structure of most commonly used personality measurement instruments has not been accomplished because limitations of existing computation equipment have made the task prohibitive. Consequently, beyond a few studies such as Comrey’s item factor analyses of subscales of the Minnesota Multiphasic Personality Inventory (e.g., Comrey, 1957, 1958), other work with instruments containing many items as focused mainly on analyses of the factor structure of subscale scores themselves (c.f., Scarr,

Accuracy of Self-Reported College Grade Averages and Characteristics of Non and Discrepant Reporters

Abstract: on officially recorded grades. The present study was not intentionally designed to bear upon this question. Relevant data had been collected in the process of other research, and since there was found to be a sparsity of appropriate research on the accuracy of grade reporting, it was decided to investigate this question. It is recognized that these data may have limited generalizability to other institutions or curricula.

Restrictions on the Possible Values of r12 Given r13 and r23.

Abstract: IN the final stages of preparing the manuscript of their new statistics textbook for the printer, Glass and Stanley (1969) discovered that they had offered without proof a formula for the restriction on the possible values of r12, the Pearsonian coefficient of correlation, in terms of rI3 and r23. The investigators searched a number of textbooks in psychology and education to determine whether proofs appear there. None were found. The basic proof seems due to Yule (1897). A proof is more readily available in Yule’s statistics textbook (e.g., Yule and Kendall, 1950, p. 301). The investigators offer herewith a simple algebraic version that makes clear the nonmystical basis for these limits. Let us begin with rI2’3, the first-order partial coefficient of correlation for the correlation of variables 1 and 2 when variable 3 is &dquo;partialed out.&dquo; The familiar formula (e.g., see Hays, 1963, pp. 574-576) is

FORTAP: a FORTRAN Test Analysis Package:

Abstract: THE FORTAP program is the fifth generation in a family of test analysis programs which had its inception as a program on the UNIVAC 1103 in 1959 (Baker, 1959). Although the basic conceptualization of the program did not change materially, each succeeding generation incorporated additional features and capabilities. Since the first four generations were written in machine language for various computers, their availability was restricted. In order to make the program available to a wider range of researchers, it was decided to write the fifth generation program in FORTRAN. The change in programming language made it necessary to study the structure of the program, and as a result of

The Prediction of Creative Achievement from a Biographical Inventory

Abstract: IN a recent review article, Freeberg (1967) summarized the literature dealing with the use of biographical information to predict student achievement. Since biographical questionnaires seem to measure a wide variety of factors-both intellectual and nonintellectual, Freeberg concluded that they perform at their best and surpass other predictors when complex criteria are used, such as creative achievement. Although there have been numerous studies concerned with the relationship of biographical data and academic success in terms of grades (Asher and Grey, 1940; Myers, 1952; Schaefer, 1963), relatively few biographical studies (Anastasi and Schaefer, 1969; Schaefer and Anastasi, 1968) have focussed on the prediction of student creative achievement.

Semantic Differential Ratings and the Rank-Ordering of Values.

Abstract: minal value is a belief about an end state of existence (e.g., a comfortable life, an exciting life, a sense of accomplishment), and an instrumental value is a belief about a mode of conduct (e.g., ambitious, broadminded, capable). In order to measure the relative importance of values to individuals, Rokeach selected 18 terminal and 18 instrumental values for use in two separate rankorder perference scales. Subjects were instructed to rank the values in order of their importance in their lives. The ranking technique has the advantage of forcing the individual to generate a value system. This is important if one hypothesizes that behavior is determined by the relative (rather than the absolute) importance of a person’s values. However, questions may be raised about the reliability and validity of such a technique. Penner, Homant, and Rokeach (1968) have found that the reliability of Rokeach’s value scales is comparable to that obtained by a more laborious paired-comparison method of measuring those

Undergraduate Ability Factors in Relationship to Graduate Performance.

Abstract: THE present study investigated the relationships among a series of criteria which can be employed in considering admission of students in graduate psychology programs. Further, the validity of these criteria for predicting graduate level performance was assessed. The following set of variables subsumes most of the criteria which are employed in considering candidates for graduate work in psychology: the Graduate Record Examination (GRE) Aptitude Tests and the GRE Advanced Test in Psychology; the Miller Analogies Test (MAT) ; undergraduate grade point average; grade point average in the junior and senior years of undergraduate work; increase in the grade point average in the last two years over the first two years of undergraduate work; number of mathematics and logic courses taken; rating of the department which the student attended as an undergraduate; sex of the student; amount of research experience of the student as an undergraduate; an average assessment of the overall promise of the student as a graduate student ; and a rating of the student’s research in contrast to service orientation-the latter three being assessed from letters of recommendation and/or biographical statements of the student. There is research evidence relating scores on the MAT and the GRE, as well as undergraduate grade point average, to graduate school performance. The evidence relating to the MAT obtained in studies by Cureton, Cureton and Bishop (1949), Kelley and Fiske

Volunteer Effects on Experimental Outcomes.

Abstract: THERE is a growing literature which shows that volunteers for participation in behavioral research differ from nonvolunteers on several important dimensions.3 Volunteers appear to be better educated and to occupy higher status positions than nonvolunteers. Volunteers also seem to be brighter, higher in approval need, less authoritarian, more sociable, more arousal-seeking, more unconventional, more often firstborn, and younger than nonvolunteers. Men seem to volunteer more for unconventional studies; women,

Trait and Evaluative Consistency in Self-Description:

Abstract: THE social desirability scale values (SDSVs) of personality statements can be estimated by having a group of judges rate the statements on a 9-point scale in accordance with standard instructions described by Edwards (1957). On the 9-point rating scale, 1 represents extremely undesirable statements, 5 represents neutral statements, and 9 represents extremely desirable statements. The average rating assigned to a statement is the SDSV of the statement. If subjects are asked to describe themselves in terms of a set of personality statements, the percentage answering True, P (T), to each statement can be obtained. For random or representative sets of statements, P(T) is a linear increasing function of the SDSVs of the statements. The evidence regarding the linear relationship between P(T) and SDSV has been summarized by Edwards (1967), who reports that the typical product-moment correlation between these two variables is about .87. When the SDSV of a personality statement is known, it is possible to define a socially desirable (SD) response to the statement. An SD response is defined as a True response to statements with SDSVs > 5.0 or as a False response to statements with SDSVs

A Distinction Between Judgments of Frequency and of Desirability as Determinants of Response

Abstract: THE research reported in this paper is directed at the solution of two inter-related problems: (a) the identification of the determinants of the largest factor of personality inventories such as the MMPI; and (b) the evaluation of consistencies in judging item properties as possible sources of valid data about characteristics of judges. The first problem is approached by considering, in addition to judged desirability, two alternative connotative properties of items, namely, the judged frequency of occurrence of the trait represented by the item and the judged frequency of endorsement of the item. The second problem, the use of judgmental consistencies as an assessment technique, was approached by evaluating the degree to which individuals judged items differentially under alternative instructional sets.

Comparisons of Varimax Rotations with Rotations to Theoretical Targets

Abstract: IT is well known by those who factor-analyze, and by some of those who do not, that one of the most serious difficulties is the rotation problem. It should be agreed that the aim of those who apply factor analysis for the purpose of discovering scientific constructs in psychology should be to achieve psychologically significant factors, which can be replicated, which fit into systematic psychological theory, and which can be investigated meaningfully by other methods. Only in this way can there be general agreement upon factorially discovered constructs and thus the unambiguous communicability that science requires. As one important step toward this goal, Thurstone proposed his criterion of simple structure. In doing so, Thurstone implied considerable faith in the expectation that whatever collection of empirical variables is reasonably used in a factor analysis, those variables would tend to cluster when represented as vectors in factor space, with areas of greater density, separated by areas of lower density. This general principle has been rather generally accepted, although it can be questioned whether, even when stated in this rough form, the principle applies to all factor-analytic data and is a safe guide as to where psychologically meaningful axes should

Sex Differences in Self-Description on the Adjective Check List

Abstract: designed primarily to measure sex differences with regard to interests and emotional expressivness, the most significant scales of this kind are the Masculinity-feminity (MF) scale of the Minnesota Multiphasic Personality Inventory (MMPI) (Dahlstrom and Welsh, 1960) and the Femininity (Fe) scale of the California, Psychological Inventory (CPI) (Gough, 1957). Another personality assessment technique coming into wider use for research and clinical purposes is the Gough Adjective Check List (ACL). However, although there are a number of existing scales for this inventory (Gough and Heilbrun, 1965), there has been no reported investigation thus far identifying systematically sex differences associated with ACL item endorsement. To help broaden understanding of masculinity and femininity in self-concept and self-description behavior, the present research was directed toward two objectives: (a) to develop empirically an ACL femininity scale through identification of ACL adjectives for which

Multiple Comparisons in a Simplified Rank Method of Scaling.

Abstract: RANK totals that accrue to each object from the paired-comparison frequency matrix can be linearly transformed to yield scale scores for the objects. Tests of significance may be applied to the rank sum differences depending upon the objectives of the scaling process. Where the Bradley-Terry (1952) and Thurstone (1927) procedures yield scales based on a normalizing transformation, the rank scale is a variance-stable scale, as is, e.g., the arcsine transformation of percentages. This is immediately apparent from the fact that any given rank-sum difference has the same significance no matter where the rank totals may be located on the scale. Thus the variances for all scale values are equal, a feature that other methods assume but do not guarantee. Moreover, with a given number of judges and items, scale variances are independent of the nature of the items being scaled. Therefore the technique lends itself to scaling a wide variety of stimuli. Use of multiple-comparison tests of a significance would appear to be an important adjunct to the scaling process for the following

Estimating the Reliability of Peer Ratings

Abstract: PEER ratings or buddy ratings are widely used for measuring certain characteristics of group members. This type of rating has been found to yield significant predictions of various performance criteria (see Hollander, 1957 and Smith, 1967) and is relatively independent of group composition (Gordon and Medland, 1965). The peer rating approach involves all members of a group evaluating one another on some characteristic. Each person’s score

What the Itpa Measures: a Synthesis of Factor Studies of the 1961 Edition:

Abstract: on the conceptualization of disabled school learners and on their programming. While the instrument is a tool for a profile description of a child, it happens also to be frankly based on a differentiated rather than a general concept of human intellect. Do the nine subtests each measure an independent ability? Or is there evidence that they separately sample the functions listed in psycholinguistic theory (levels of representation, processes, and channels of communication) ? The factor analytic approach is one means of determining an answer. There have been (to the present writer’s knowledge) 16 different factor studies employing some or all parts of this scale. Those studies judged of value to answer the basic question were compared for this synthesis. One is an original analysis on data from the McCarthy and Olson (1964) validation report, and another is a re-analysis of the Haring and Ridgway