A representation and interpretation of the area under a receiver operating characteristic (ROC) curve obtained by the "rating" method, or by mathematical predictions based on patient characteristics, is presented. It is shown that in such a setting the area represents the probability that a randomly chosen diseased subject is (correctly) rated or ranked with greater suspicion than a randomly chosen non-diseased subject. Moreover, this probability of a correct ranking is the same quantity that is estimated by the already well-studied nonparametric Wilcoxon statistic. These two relationships are exploited to (a) provide rapid closed-form expressions for the approximate magnitude of the sampling variability, i.e., standard error that one uses to accompany the area under a smoothed ROC curve, (b) guide in determining the size of the sample required to provide a sufficiently reliable estimate of this area, and (c) determine how large sample sizes should be to ensure that one can statistically detect difference...

The meaning and use of the area under a receiver operating characteristic (ROC) curve.

Attention and Effort

Receiver operating characteristic (ROC) curves are used to describe and compare the performance of diagnostic technology and diagnostic algorithms. This paper refines the statistical comparison of the areas under two ROC curves derived from the same set of patients by taking into account the correlation between the areas that is induced by the paired nature of the data. The correspondence between the area under an ROC curve and the Wilcoxon statistic is used and underlying Gaussian distributions (binormal) are assumed to provide a table that converts the observed correlations in paired ratings of images into a correlation between the two ROC areas. This between-area correlation can be used to reduce the standard error (uncertainty) about the observed difference in areas. This correction for pairing, analogous to that used in the paired t-test, can produce a considerable increase in the statistical sensitivity (power) of the comparison. For studies involving multiple readers, this method provides a measure...

/pdf/a-method-of-comparing-the-areas-under-receiver-operating-50lpcksrsi.pdf

A method of comparing the areas under receiver operating characteristic curves derived from the same cases.

The clinical performance of a laboratory test can be described in terms of diagnostic accuracy, or the ability to correctly classify subjects into clinically relevant subgroups. Diagnostic accuracy refers to the quality of the information provided by the classification device and should be distinguished from the usefulness, or actual practical value, of the information. Receiver-operating characteristic (ROC) plots provide a pure index of accuracy by demonstrating the limits of a test's ability to discriminate between alternative states of health over the complete spectrum of operating conditions. Furthermore, ROC plots occupy a central or unifying position in the process of assessing and using diagnostic tools. Once the plot is generated, a user can readily go on to many other activities such as performing quantitative ROC analysis and comparisons of tests, using likelihood ratio to revise the probability of disease in individual subjects, selecting decision thresholds, using logistic-regression analysis, using discriminant-function analysis, or incorporating the tool into a clinical strategy by using decision analysis.

/pdf/receiver-operating-characteristic-roc-plots-a-fundamental-1bsmhsp7j4.pdf

Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine.

Social behavior is ordinarily treated as being under conscious (if not always thoughtful) control. However, considerable evidence now supports the view that social behavior often operates in an implicit or unconscious fashion. The identifying feature of implicit cognition is that past experience influences judgment in a fashion not introspectively known by the actor. The present conclusion--that attitudes, self-esteem, and stereotypes have important implicit modes of operation--extends both the construct validity and predictive usefulness of these major theoretical constructs of social psychology. Methodologically, this review calls for increased use of indirect measures--which are imperative in studies of implicit cognition. The theorized ordinariness of implicit stereotyping is consistent with recent findings of discrimination by people who explicitly disavow prejudice. The finding that implicit cognitive effects are often reduced by focusing judges' attention on their judgment task provides a basis for evaluating applications (such as affirmative action) aimed at reducing such unintended discrimination.

/pdf/implicit-social-cognition-attitudes-self-esteem-and-2j32pp6h9o.pdf

Implicit Social Cognition: Attitudes, Self-Esteem, and Stereotypes.

Book on statistical decision theory and sensory processes in signal detection theory and psychophysics

Signal detection theory and psychophysics

In keeping with our promise earlier in this review, we summarize here the process by which we believe spatial cues are used for localizing a sound source in a free-field listening situation. We believe it entails two parallel processes: 1. The azimuth of the source is determined using differences in interaural time or interaural intensity, whichever is present. Wightman and colleagues (1989) believe the low-frequency temporal information is dominant if both are present. 2. The elevation of the source is determined from spectral shape cues. The received sound spectrum, as modified by the pinna, is in effect compared with a stored set of directional transfer functions. These are actually the spectra of a nearly flat source heard at various elevations. The elevation that corresponds to the best-matching transfer function is selected as the locus of the sound. Pinnae are similar enough between people that certain general rules (e.g. Blauert's boosted bands or Butler's covert peaks) can describe this process. Head motion is probably not a critical part of the localization process, except in cases where time permits a very detailed assessment of location, in which case one tries to localize the source by turning the head toward the putative location. Sound localization is only moderately more precise when the listener points directly toward the source. The process is not analogous to localizing a visual source on the fovea of the retina. Thus, head motion provides only a moderate increase in localization accuracy. Finally, current evidence does not support the view that auditory motion perception is anything more than detection of changes in static location over time.

Sound Localization by Human Listeners

Frequency discrimination was measured for frequencies from 200 to 8000 Hz and for sensation levels from 5 to 80 dB using pulsed sinusoids as stimuli in an adaptive two‐interval force‐choice psychophysical procedure. An analysis of variance indicated significant effects of frequency and sensation level, and of the interaction between frequency and sensation level. The effect of sensation level is greatest at low frequencies and decreases at high frequenices, being quite small at 8000 Hz. The data are used to evaluate the predictions of current theoretical models.

Frequency discrimination as a function of frequency and sensation level

By conventional spike count measures, auditory neurons in the cat's anterior ectosylvian sulcus cortical area are broadly tuned for the location of a sound source. Nevertheless, an artificial neural network was trained to classify the temporal spike patterns of single neurons according to sound location. The spike patterns of 73 percent of single neurons coded sound location with more than twice the chance level of accuracy, and spike patterns consistently carried more information than spike counts alone. In contrast to neurons that are sharply tuned for location, these neurons appear to encode sound locations throughout 360 degrees of azimuth.

/pdf/a-panoramic-code-for-sound-location-by-cortical-neurons-51s0o4f1u9.pdf

A panoramic code for sound location by cortical neurons

Intensity discrimination was measured for pulsed sinusoids of various frequencies (200–8000 Hz) and sensation levels (5–80 dB). The data for all frequencies were fitted by a single function, ΔI/I=0.463 (I/I0)−0.072, where I0 is intensity at threshold, I is the intensity of the tone, and ΔI is the increment needed to obtain 71% correct in a two‐interval forced‐choice adaptive procedure. The form of this function is in good agreement with data reported in comparable studies but differs markedly from the data reported by Riesz [Phys. Rev. 31, 867–875 (1928)]. An analysis of the actual values of ΔI/I reported in the other studies indicates a range larger than would be predicted on the basis of individual differences among observers in this study. The data are also discussed differences among observers in this study. The data are also discussed in terms of the predictions of current theoretical models.

David M. Green

Papers

Signal detection theory and psychophysics

Sound Localization by Human Listeners

Frequency discrimination as a function of frequency and sensation level

A panoramic code for sound location by cortical neurons

Intensity discrimination as a function of frequency and sensation level