Diagnostic systems of several kinds are used to distinguish between two classes of events, essentially "signals" and "noise". For them, analysis in terms of the "relative operating characteristic" of signal detection theory provides a precise and valid measure of diagnostic accuracy. It is the only measure available that is uninfluenced by decision biases and prior probabilities, and it places the performances of diverse systems on a common, easily interpreted scale. Representative values of this measure are reported here for systems in medical imaging, materials testing, weather forecasting, information retrieval, polygraph lie detection, and aptitude testing. Though the measure itself is sound, the values obtained from tests of diagnostic systems often require qualification because the test data on which they are based are of unsure quality. A common set of problems in testing is faced in all fields. How well these problems are handled, or can be handled in a given field, determines the degree of confidence that can be placed in a measured value of accuracy. Some fields fare much better than others.

https://science.sciencemag.org/content/sci/240/4857/1285.full.pdf

Measuring the accuracy of diagnostic systems

1. Introduction 2. The emergence of learning activity as a historical form of human learning 3. The zone of proximal development as the basic category of expansive research 4. The instruments of expansion 5. Toward an expansive methodology 6. Epilogue.

Learning by expanding: An activity-theoretical approach to developmental research

Confirmation bias, as the term is typically used in the psychological literature, connotes the seeking or interpreting of evidence in ways that are partial to existing beliefs, expectations, or a h...

/pdf/confirmation-bias-a-ubiquitous-phenomenon-in-many-guises-18m0fi8wck.pdf

Confirmation Bias: A Ubiquitous Phenomenon in Many Guises:

In 1974 an article appeared in Science magazine with the dry-sounding title “Judgment Under Uncertainty: Heuristics and Biases” by a pair of psychologists who were not well known outside their discipline of decision theory. In it Amos Tversky and Daniel Kahneman introduced the world to Prospect Theory, which mapped out how humans actually behave when faced with decisions about gains and losses, in contrast to how economists assumed that people behave. Prospect Theory turned Economics on its head by demonstrating through a series of ingenious experiments that people are much more concerned with losses than they are with gains, and that framing a choice from one perspective or the other will result in decisions that are exactly the opposite of each other, even if the outcomes are monetarily the same. Prospect Theory led cognitive psychology in a new direction that began to uncover other human biases in thinking that are probably not learned but are part of our brain’s wiring.

Thinking fast and slow.

A review is presented of the book “Heuristics and Biases: The Psychology of Intuitive Judgment,” edited by Thomas Gilovich, Dale Griffin, and Daniel Kahneman.

Heuristics and Biases: The Psychology of Intuitive Judgment

Null hypothesis significance testing (NHST) is arguably the most widely used approach to hypothesis evaluation among behavioral and social scientists. It is also very controversial. A major concern expressed by critics is that such testing is misunderstood by many of those who use it. Several other objections to its use have also been raised. In this article the author reviews and comments on the claimed misunderstandings as well as on other criticisms of the approach, and he notes arguments that have been advanced in support of NHST. Alternatives and supplements to NHST are considered, as are several related recommendations regarding the interpretation of experimental data. The concluding opinion is that NHST is easily misunderstood and misused but that when applied with good judgment it can be an effective aid to the interpretation of experimental data.

Null hypothesis significance testing : A review of an old and continuing controversy

Attention and Performance Viii

To communicate effectively, people must have a reasonably accurate idea about what specific other people know. An obvious starting point for building a model of what another knows is what one oneself knows, or thinks one knows. This article reviews evidence that people impute their own knowledge to others and that, although this serves them well in general, they often do so uncritically, with the result of erroneously assuming that other people have the same knowledge. Overimputation of one's own knowledge can contribute to communication difficulties. Corrective approaches are considered. A conceptualization of where own-knowledge imputation fits in the process of developing models of other people's knowledge is proposed.

/pdf/how-we-know-and-sometimes-misjudge-what-others-know-imputing-55yhvxl6oq.pdf

Raymond S. Nickerson

Papers

Confirmation Bias: A Ubiquitous Phenomenon in Many Guises:

Null hypothesis significance testing : A review of an old and continuing controversy

Attention and Performance Viii

How We Know—and Sometimes Misjudge—What Others Know: Imputing One's Own Knowledge to Others

The Teaching of Thinking