Verbal reports as data.
About: This article is published in Psychological Review.The article was published on 1980-05-01. It has received 4365 citations till now.
Citations
More filters
TL;DR: In this article, the results of a multi-year research program to identify the factors associated with variations in subjective workload within and between different types of tasks are reviewed, including task-, behavior-, and subject-related correlates of subjective workload experiences.
Abstract: The results of a multi-year research program to identify the factors associated with variations in subjective workload within and between different types of tasks are reviewed. Subjective evaluations of 10 workload-related factors were obtained from 16 different experiments. The experimental tasks included simple cognitive and manual control tasks, complex laboratory and supervisory control tasks, and aircraft simulation. Task-, behavior-, and subject-related correlates of subjective workload experiences varied as a function of difficulty manipulations within experiments, different sources of workload between experiments, and individual differences in workload definition. A multi-dimensional rating scale is proposed in which information about the magnitude and sources of six workload-related factors are combined to derive a sensitive and reliable estimate of workload.
11,418 citations
TL;DR: The meaning of the terms "method" and "method bias" are explored and whether method biases influence all measures equally are examined, and the evidence of the effects that method biases have on individual measures and on the covariation between different constructs is reviewed.
Abstract: Despite the concern that has been expressed about potential method biases, and the pervasiveness of research settings with the potential to produce them, there is disagreement about whether they really are a problem for researchers in the behavioral sciences. Therefore, the purpose of this review is to explore the current state of knowledge about method biases. First, we explore the meaning of the terms “method” and “method bias” and then we examine whether method biases influence all measures equally. Next, we review the evidence of the effects that method biases have on individual measures and on the covariation between different constructs. Following this, we evaluate the procedural and statistical remedies that have been used to control method biases and provide recommendations for minimizing method bias.
8,719 citations
TL;DR: A theoretical framework is proposed that explains expert performance in terms of acquired characteristics resulting from extended deliberate practice and that limits the role of innate (inherited) characteristics to general levels of activity and emotionality.
Abstract: because observed behavior is the result of interactions between environmental factors and genes during the extended period of development. Therefore, to better understand expert and exceptional performance, we must require that the account specify the different environmental factors that could selectively promote and facilitate the achievement of such performance. In addition, recent research on expert performance and expertise (Chi, Glaser, & Farr, 1988; Ericsson & Smith, 1991a) has shown that important characteristics of experts' superior performance are acquired through experience and that the effect of practice on performance is larger than earlier believed possible. For this reason, an account of exceptional performance must specify the environmental circumstances, such as the duration and structure of activities, and necessary minimal biological attributes that lead to the acquisition of such characteristics and a corresponding level of performance. An account that explains how a majority of individuals can attain a given level of expert performance might seem inherently unable to explain the exceptional performance of only a small number of individuals. However, if such an empirical account could be empirically supported, then the extreme characteristics of experts could be viewed as having been acquired through learning and adaptation, and studies of expert performance could provide unique insights into the possibilities and limits of change in cognitive capacities and bodily functions. In this article we propose a theoretical framework that explains expert performance in terms of acquired characteristics resulting from extended deliberate practice and that limits the role of innate (inherited) characteristics to general levels of activity and emotionality. We provide empirical support from two new studies and from already published evidence on expert performance in many different domains.
7,886 citations
TL;DR: Schmidt as mentioned in this paper presented on the role of consciousness in second language learning at the 1988 Second Language Research Forum (SLRF) held in at the University of Hawai'i, USA.
Abstract: Richard Schmidt presents on the role of consciousness in second language learning at the 1988 Second Language Research Forum (SLRF) held in at the University of Hawai‘i.
4,131 citations
Cites background from "Verbal reports as data."
...These conditions are discussed at length in the verbal report literature (Ericsson and Simon 1980, 1984; Fserch and Kasper 1987), but two are particularly relevant for the present discussion....
[...]
...Because of memory constraints, failure to report retrospectively that something has been noticed does not demonstrate that the event was not registered in conscious awareness at the time of the event (Ericsson and Simon 1980; White 1980)....
[...]
TL;DR: A review of the literature on problem solving and metacognition can be found in this article, where the authors outline and substantiate a broad conceptualization of what it means to think mathematically, summarize the literature relevant to understanding mathematical thinking and problem solving, and point to new directions in research, development and assessment.
Abstract: The goals of this chapter are (1) to outline and substantiate a broad conceptualization of what it means to think mathematically, (2) to summarize the literature relevant to understanding mathematical thinking and problem solving, and (3) to point to new directions in research, development, and assessment consonant with an emerging understanding of mathematical thinking and the goals for instruction outlined here. The use of the phrase “learning to think mathematically” in this chapter’s title is deliberately broad. Although the original charter for this chapter was to review the literature on problem solving and metacognition, the literature itself is somewhat ill defined and poorly grounded. As the literature summary will make clear, problem solving has been used with multiple meanings that range from “working rote exercises” to “doing mathematics as a professional”; metacognition has multiple and almost disjoint meanings (from knowledge about one’s thought processes to self-regulation during problem solving) that make it difficult to use as a concept. This chapter outlines the various meanings that have been ascribed to these terms and discusses their role in mathematical thinking. The discussion will not have the character of a classic literature review, which is typically encyclopedic in its references and telegraphic in its discussions of individual papers or results. It will, instead, be selective and illustrative, with main points illustrated by extended discussions of pertinent examples. Problem solving has, as predicted in the 1980 Yearbook of the National Council of Teachers of Mathematics (Krulik, 1980, p. xiv), been the theme of the 1980s. The decade began with NCTM’s widely heralded statement, in its Agenda for Action, that “problem solving must be the focus of school mathematics” (NCTM, 1980, p. 1). It concluded with the publication of Everybody Counts (National Research Council, 1989) and the Curriculum and Evaluation Standards for School Mathematics (NCTM, 1989), both of which emphasize problem solving. One might infer, then, that there is general acceptance of the idea that the primary goal of mathematics instruction should be to have students become competent problem solvers. Yet, given the multiple interpretations of the term, the goal is hardly clear. Equally unclear is the role that problem solving, once adequately characterized, should play in the larger context of school mathematics. What are the goals for mathematics instruction, and how does problem solving fit within those goals? Such questions are complex. Goals for mathematics instruction depend on one’s conceptualization of what mathematics is, and what it means to understand mathematics. Such conceptualizations vary widely. At one end of the spectrum, mathematical knowledge is seen as a body of facts and procedures dealing with quantities, magnitudes, and forms, and the relationships among them; knowing mathematics is seen as having mastered these facts and procedures. At the other end of the spectrum, mathematics is conceptualized as the “science of patterns,” an (almost) empirical discipline closely akin to the sciences in its emphasis on pattern-seeking on the basis of empirical evidence. The author’s view is that the former perspective trivializes mathematics; that a curriculum based on mastering a corpus of mathematical facts and procedures is severely impoverished—in much the same way that an English curriculum would be considered impoverished if it focused largely, if not exclusively, on issues of grammar. The author characterizes the mathematical enterprise as follows:
2,756 citations
References
More filters
Book•
01 Jun 1972TL;DR: The aim of the book is to advance the understanding of how humans think by putting forth a theory of human problem solving, along with a body of empirical evidence that permits assessment of the theory.
Abstract: : The aim of the book is to advance the understanding of how humans think. It seeks to do so by putting forth a theory of human problem solving, along with a body of empirical evidence that permits assessment of the theory. (Author)
10,770 citations
TL;DR: In this paper, it was shown that people are sometimes unaware of the existence of a stimulus that influenced a response, unaware of its existence, and unaware that the stimulus has affected the response.
Abstract: Evidence is reviewed which suggests that there may be little or no direct introspective access to higher order cognitive processes. Subjects are sometimes (a) unaware of the existence of a stimulus that importantly influenced a response, (b) unaware of the existence of the response, and (c) unaware that the stimulus has affected the response. It is proposed that when people attempt to report on their cognitive processes, that is, on the processes mediating the effects of a stimulus on a response, they do not do so on the basis of any true introspection. Instead, their reports are based on a priori, implicit causal theories, or judgments about the extent to which a particular stimulus is a plausible cause of a given response. This suggests that though people may not be able to observe directly their cognitive processes, they will sometimes be able to report accurately about them. Accurate reports will occur when influential stimuli are salient and are plausible causes of the responses they produce, and will not occur when stimuli are not salient or are not plausible causes.
10,186 citations
TL;DR: A judgmental heuristic in which a person evaluates the frequency of classes or the probability of events by availability, i.e., by the ease with which relevant instances come to mind, is explored.
8,823 citations
8,493 citations