The Technical Qualities of the Elicited Imitation Subsection of The Assessment of College English, International (ACE-In)
25 Jun 2020-
TL;DR: The authors investigated the technical qualities of the elicited imitation (EI) items used by ACE-In, a locally developed English language proficiency test used in the undergraduate English Academic Purpose Program at Purdue University EI is a controversial language assessment tool that has been utilized and examined for decades.
Abstract: The present study investigated technical qualities of the elicited imitation (EI) items used by the Assessment of College English – International (ACE-In), a locally developed English language proficiency test used in the undergraduate English Academic Purpose Program at Purdue University EI is a controversial language assessment tool that has been utilized and examined for decades The simplicity of the test format and the ease of rating place EI in an advantageous position to be widely implemented in language assessment On the other hand, EI has received a series of critiques, primarily questioning its validity To offer insights into the quality of the EI subsection of the ACE-In and to provide guidance for continued test development and revision, the present study examined the measurement qualities of the items by analyzing the pre- and post-test performance of 100 examines on EI The analyses consist of an item analysis that reports item difficulty, item discrimination, and total score reliability; an examination of pre-post changes in performance that reports a matched pairs t-test and item instructional sensitivity; and an analysis of the correlation patterns between EI scores and TOEFL iBT total and subsection scoresThe results of the item analysis indicated that the current EI task was slightly easy for the intended population, but test items functioned satisfactorily in terms of separating examinees of higher proficiency from those of lower proficiency The EI task was also found to have high internal consistency across forms As for the pre-post changes, a significant pair-wise difference was found between the pre- and post-performance after a semester of instruction However, the results also reported that over half of the items were relatively insensitive to instruction The last stage of the analysis indicated that while EI scores had a significant positive correlation with TOEFL iBT total scores and speaking subsection scores, EI scores were negatively correlated with TOEFL iBT reading subsection scores Findings of the present study provided evidence in favor of the use of EI as a measure of L2 proficiency, especially as a viable alternative to free-response items EI is also argued to provide additional information regarding examinees’ real-time language processing ability that standardized language tests are not intended to measure Although the EI task used by the ACE-In is generally suitable for the targeted population and testing purposes, it can be further improved if test developers increase the number of difficult items and control the contents and the structures of sentence stimuli Examining the technical qualities of test items is fundamental but insufficient to build a validity argument for the test The present EI test can benefit from test validation studies that exceed item analysis Future research that focuses on improving item instructional sensitivity is also recommended
Citations
More filters
Book•
01 Jan 2000
TL;DR: This paper aims to provide a history of second language acquisition research methodology and its applications in the field of interlanguage studies and aims to clarify the role of data analysis in this research.
Abstract: Preface Acknowledgements 1. Introduction 2. Second language acquisition research methodology 3. SLA - Types of data analysis 4. Interlanguage studies: Substantive findings 5. The linguistic environment for language acquisition 6. Explanations for differential success among second language learners 7. Theories in second language acquisition 8. Instructed second language acquisition Bibliography Index
275 citations
Book Chapter•
19 Jun 2009
TL;DR: In the case of implicit learning, learners remain unaware of the learning that has taken place, although it is evident in the behavioral responses they make, and cannot verbalize what they have learned.
Abstract: (1) Implicit learning proceeds without making demands on central attentional resources. As N. Ellis (2008: 125) puts it, ‘generalizations arise from conspiracies of memorized utterances collaborating in productive schematic linguistic productions’. Thus, the resulting knowledge is subsymbolic, reflecting statistical sensitivity to the structure of the learned material. In contrast, explicit learning typically involves memorizing a series of successive facts and thus makes heavy demands on working memory. As a result, it takes place consciously and results in knowledge that is symbolic in nature (i.e. it is represented in explicit form). (2) In the case of implicit learning, learners remain unaware of the learning that has taken place, although it is evident in the behavioral responses they make. Thus, learners cannot verbalize what they have learned. In the case of explicit learning, learners are aware that they have learned something and can verbalize what they have learned.
251 citations
01 Jan 2015
1 citations
References
More filters
Book•
01 Dec 1969TL;DR: The concepts of power analysis are discussed in this paper, where Chi-square Tests for Goodness of Fit and Contingency Tables, t-Test for Means, and Sign Test are used.
Abstract: Contents: Prefaces. The Concepts of Power Analysis. The t-Test for Means. The Significance of a Product Moment rs (subscript s). Differences Between Correlation Coefficients. The Test That a Proportion is .50 and the Sign Test. Differences Between Proportions. Chi-Square Tests for Goodness of Fit and Contingency Tables. The Analysis of Variance and Covariance. Multiple Regression and Correlation Analysis. Set Correlation and Multivariate Methods. Some Issues in Power Analysis. Computational Procedures.
115,069 citations
"The Technical Qualities of the Elic..." refers background in this paper
...To clarify, the effect sizes in fields that are attempting to measure latent traits and lack experimental control are expected to be “small”, but those effects should not be necessarily interpreted as unimportant (Cohen, 1988)....
[...]
49,129 citations
"The Technical Qualities of the Elic..." refers background in this paper
...To clarify, the effect sizes in fields that are attempting to measure latent traits and lack experimental control are expected to be “small”, but those effects should not be necessarily interpreted as unimportant (Cohen, 1988)....
[...]
Journal Article•
TL;DR: The theory of information as discussed by the authors provides a yardstick for calibrating our stimulus materials and for measuring the performance of our subjects and provides a quantitative way of getting at some of these questions.
Abstract: First, the span of absolute judgment and the span of immediate memory impose severe limitations on the amount of information that we are able to receive, process, and remember. By organizing the stimulus input simultaneously into several dimensions and successively into a sequence or chunks, we manage to break (or at least stretch) this informational bottleneck. Second, the process of recoding is a very important one in human psychology and deserves much more explicit attention than it has received. In particular, the kind of linguistic recoding that people do seems to me to be the very lifeblood of the thought processes. Recoding procedures are a constant concern to clinicians, social psychologists, linguists, and anthropologists and yet, probably because recoding is less accessible to experimental manipulation than nonsense syllables or T mazes, the traditional experimental psychologist has contributed little or nothing to their analysis. Nevertheless, experimental techniques can be used, methods of recoding can be specified, behavioral indicants can be found. And I anticipate that we will find a very orderly set of relations describing what now seems an uncharted wilderness of individual differences. Third, the concepts and measures provided by the theory of information provide a quantitative way of getting at some of these questions. The theory provides us with a yardstick for calibrating our stimulus materials and for measuring the performance of our subjects. In the interests of communication I have suppressed the technical details of information measurement and have tried to express the ideas in more familiar terms; I hope this paraphrase will not lead you to think they are not useful in research. Informational concepts have already proved valuable in the study of discrimination and of language; they promise a great deal in the study of learning and memory; and it has even been proposed that they can be useful in the study of concept formation. A lot of questions that seemed fruitless twenty or thirty years ago may now be worth another look. In fact, I feel that my story here must stop just as it begins to get really interesting. And finally, what about the magical number seven? What about the seven wonders of the world, the seven seas, the seven deadly sins, the seven daughters of Atlas in the Pleiades, the seven ages of man, the seven levels of hell, the seven primary colors, the seven notes of the musical scale, and the seven days of the week? What about the seven-point rating scale, the seven categories for absolute judgment, the seven objects in the span of attention, and the seven digits in the span of immediate memory? For the present I propose to withhold judgment. Perhaps there is something deep and profound behind all these sevens, something just calling out for us to discover it. But I suspect that it is only a pernicious, Pythagorean coincidence.
19,835 citations
Book•
01 Jan 1956
TL;DR: The theory provides us with a yardstick for calibrating the authors' stimulus materials and for measuring the performance of their subjects, and the concepts and measures provided by the theory provide a quantitative way of getting at some of these questions.
Abstract: First, the span of absolute judgment and the span of immediate memory impose severe limitations on the amount of information that we are able to receive, process, and remember. By organizing the stimulus input simultaneously into several dimensions and successively into a sequence or chunks, we manage to break (or at least stretch) this informational bottleneck. Second, the process of recoding is a very important one in human psychology and deserves much more explicit attention than it has received. In particular, the kind of linguistic recoding that people do seems to me to be the very lifeblood of the thought processes. Recoding procedures are a constant concern to clinicians, social psychologists, linguists, and anthropologists and yet, probably because recoding is less accessible to experimental manipulation than nonsense syllables or T mazes, the traditional experimental psychologist has contributed little or nothing to their analysis. Nevertheless, experimental techniques can be used, methods of recoding can be specified, behavioral indicants can be found. And I anticipate that we will find a very orderly set of relations describing what now seems an uncharted wilderness of individual differences. Third, the concepts and measures provided by the theory of information provide a quantitative way of getting at some of these questions. The theory provides us with a yardstick for calibrating our stimulus materials and for measuring the performance of our subjects. In the interests of communication I have suppressed the technical details of information measurement and have tried to express the ideas in more familiar terms; I hope this paraphrase will not lead you to think they are not useful in research. Informational concepts have already proved valuable in the study of discrimination and of language; they promise a great deal in the study of learning and memory; and it has even been proposed that they can be useful in the study of concept formation. A lot of questions that seemed fruitless twenty or thirty years ago may now be worth another look. In fact, I feel that my story here must stop just as it begins to get really interesting. And finally, what about the magical number seven? What about the seven wonders of the world, the seven seas, the seven deadly sins, the seven daughters of Atlas in the Pleiades, the seven ages of man, the seven levels of hell, the seven primary colors, the seven notes of the musical scale, and the seven days of the week? What about the seven-point rating scale, the seven categories for absolute judgment, the seven objects in the span of attention, and the seven digits in the span of immediate memory? For the present I propose to withhold judgment. Perhaps there is something deep and profound behind all these sevens, something just calling out for us to discover it. But I suspect that it is only a pernicious, Pythagorean coincidence.
16,902 citations
"The Technical Qualities of the Elic..." refers background in this paper
...To further clarify the roles of STM, it is also necessary to reexamine the discussion of the famous “magical number seven” proposed by Miller (1956)....
[...]
...According to Miller (1956), a “unit,” also referred to as a “chunk”, is the largest meaningful unit of information to a speaker....
[...]
...Miller (1956) presented evidence for the existence of a clear and definite limit to the accuracy of immediate memory....
[...]
Book•
01 Jan 1982
TL;DR: It is concluded that language acquisition occurs best when language is used for the purpose for which it was designed: communication.
Abstract: This text explores the relationship between second language teaching practice and what is known about the process of second language acquisition and summarizes the current state of second language acquisition theory.-- Draws general conclusions about the application of theory to methods and materials and describes the characteristics that effective materials should include.-- Concludes that language acquisition occurs best when language is used for the purpose for which it was designed: communication.
6,737 citations
"The Technical Qualities of the Elic..." refers background in this paper
...Krashen states that “learning does not become acquisition” (p. 83, Krashen, 1982)....
[...]
...Krashen (1982) argued that an adult second language learner employed two approaches to internalize the rules of a target language: one was through language learning and the other was via language acquisition, and both approaches account for the development of linguistic competence....
[...]