https://ccrma.stanford.edu/workshops/mir2009/references/ROCintro.pdf

An introduction to ROC analysis

A review is presented of the book “Heuristics and Biases: The Psychology of Intuitive Judgment,” edited by Thomas Gilovich, Dale Griffin, and Daniel Kahneman.

Heuristics and Biases: The Psychology of Intuitive Judgment

Much research in the last two decades has demon- strated that human responses deviate from the performance deemed normative according to various models of decision mak- ing and rational judgment (e.g., the basic axioms of utility theory). This gap between the normative and the descriptive can be inter- preted as indicating systematic irrationalities in human cognition. However, four alternative interpretations preserve the assumption that human behavior and cognition is largely rational. These posit that the gap is due to (1) performance errors, (2) computational limitations, (3) the wrong norm being applied by the experi- menter, and (4) a different construal of the task by the subject. In the debates about the viability of these alternative explanations, attention has been focused too narrowly on the modal response. In a series of experiments involving most of the classic tasks in the heuristics and biases literature, we have examined the implica- tions of individual differences in performance for each of the four explanations of the normative/descriptive gap. Performance er- rors are a minor factor in the gap; computational limitations un- derlie non-normative responding on several tasks, particularly those that involve some type of cognitive decontextualization. Un- expected patterns of covariance can suggest when the wrong norm is being applied to a task or when an alternative construal of the task should be considered appropriate.

https://www.cambridge.org/core/services/aop-cambridge-core/content/view/S0140525X00003435

Individual differences in reasoning: Implications for the rationality debate?

In this review, we examine the oft-made claim that peer-relationship difficulties in childhood predict serious adjustment problems in later life. The article begins with a framework for conceptualizing and assessing children's peer difficulties and with a discussion of conceptual and methodological issues in longitudinal risk research. Following this, three indexes of problematic peer relationships (acceptance, aggressiveness, and shyness/withdrawal) are evaluated as predictors of three later outcomes (dropping out of school, criminality, and psychcpathology). The relation between peer difficulties and later maladjustment is examined in terms of both the consistency and strength of prediction. A review and analysis of the literature indicates general support for the hypothesis that children with poor peer adjustment are at risk for later life difficulties. Support is clearest for the outcomes of dropping out and criminality. It is also clearest for low acceptance and aggressiveness as predictors, whereas a link between shyness/withdrawal and later maladjustment has not yet been adequately tested. The article concludes with a critical discussion of the implicit models that have guided past research in this area and a set of recommendations for the next generation of research on the risk

Peer relations and later personal adjustment: Are low-accepted children at risk?

Modern theories in cognitive psychology and neuroscience indicate that there are two fundamental ways in which human beings comprehend risk. The analytic system uses algorithms and normative rules, such as probability calculus, formal logic, and risk assessment. It is relatively slow, effortful, and requires conscious control. The experiential system is intuitive, fast, mostly automatic, and not very accessible to conscious awareness. The experiential system enabled human beings to survive during their long period of evolution and remains today the most natural and most common way to respond to risk. It relies on images and associations, linked by experience to emotion and affect (a feeling that something is good or bad). This system represents risk as a feeling that tells us whether it's safe to walk down this dark street or drink this strange-smelling water. Proponents of formal risk analysis tend to view affective responses to risk as irrational. Current wisdom disputes this view. The rational and the experiential systems operate in parallel and each seems to depend on the other for guidance. Studies have demonstrated that analytic reasoning cannot be effective unless it is guided by emotion and affect. Rational decision making requires proper integration of both modes of thought. Both systems have their advantages, biases, and limitations. Now that we are beginning to understand the complex interplay between emotion and reason that is essential to rational behavior, the challenge before us is to think creatively about what this means for managing risk. On the one hand, how do we apply reason to temper the strong emotions engendered by some risk events? On the other hand, how do we infuse needed "doses of feeling" into circumstances where lack of experience may otherwise leave us too "coldly rational"? This article addresses these important questions.

https://papers.ssrn.com/sol3/Delivery.cfm?abstractid=931686

Risk As Analysis and Risk As Feelings: Some Thoughts About Affect, Reason, Risk, and Rationality

A new research instrument, the MacArthur Structured Assessment of Competencies of Criminal Defendants (MacSAC-CD), was administered to three groups of women defendants: those adjudicated incompetent and committed to forensic hospitals for treatment (n = 38), jail inmates receiving mental health treatment (n = 18) who had not been adjudicated incompetent, and randomly selected jail inmates (n = 50). Measures of the competence-related abilities of understanding and reasoning were found to have satisfactory indices of internal consistency (coefficient alpha), and all measures correlated in the expected direction with measures of global psychopathology, psychoticism, and verbal cognitive functioning. Between-group mean scores were all in the expected direction and were statistically significant for four of seven measures. No differences in MacSAC-CD performance were found in comparisons of male and female defendants who had been adjudicated incompetent, nor were differences found in the performance of male and female jail inmates.

http://jaapl.org/content/jaapl/26/2/215.full.pdf

The competence-related abilities of women criminal defendants

The Inclusion of Biological Risk Factors in Violence Risk Assessments

This study is the first to our knowledge to simultaneously measure the predictors of lawyers' satisfaction with their careers and the predictors of lawyers' satisfaction with their lives more broadly. One class of the University of Virginia School of Law was studied between their matriculation in 1987 and their graduation in 1990. All 360 living graduates of this class were contacted in 2007, with a response rate of 72.2 percent. Descriptive information was obtained and empirically validated measures of both career satisfaction and life satisfaction were administered. Respondents were found to have taken many diverse career paths, with most (85 percent) having changed jobs at least once, and half having changed jobs at least twice. Gender differences in the personal and professional lives of respondents were pervasive. Women graduates were far more likely than men to interrupt or forego full-time employment (39 percent vs. 1 percent), mainly in order to care for children, and were also more likely to have a spouse or partner employed full time outside the home (77 percent vs. 24 percent). Working conditions at large private law firms emerged as a significant problem for many respondents; half of those who started their careers in large firms left to go to a different type of employer. Finally, both career satisfaction and life satisfaction were found to be high, with 81 percent of the respondents satisfied with their decision to become a lawyer, and 86 percent satisfied with their lives more broadly.

Lawyers at Mid-Career: A 20-Year Longitudinal Study of Job and Life Satisfaction

The American Sociological Association (“ASA”) filed an amicus brief in Wal-Mart v. Dukes in which the ASA defended the testimony of the plaintiffs’ sociological expert. Unfortunately, the ASA’s portrayal and defense of the method and opinions of this expert do not match the actual method used, and opinions offered, by the expert in the Wal-Mart case. We demonstrate that none of the ASA’s defenses of the expert’s method has merit and that the expert violated basic methodological rules set out by the ASA’s own sources. The opinions to which the expert testified, therefore, lacked a scientific foundation.

https://papers.ssrn.com/sol3/Delivery.cfm/SSRN_ID2558806_code327711.pdf?abstractid=1816752&type=2

The ASA’s Missed Opportunity to Promote Sound Science in Court

When a therapist determines, or pursuant to the standards of his profession should determine, that his patient presents a serious danger of violence to another, he incurs an obligation to use reasonable care to protect the intended victim against such danger. Much has changed in the thirty years since Justice Tobriner wrote these words. I will address two areas of change in this article: change in the science of violence risk assessment, and change in American mental health policy. I will argue that changes in these two areas impact the manner in which a therapist “pursuant to the standards of his [or her] profession should determine” the risk of violence, and the steps that he or she should take in the event that the level of assessed risk exceeds the threshold necessary to trigger “reasonable care to protect the intended victim.” In 1976, when Tarasoff was decided, only two empirical studies existed on the validity of predictions of community violence among people with mental disorders. The results of the studies were such that, in a review cited—dismissively—by the court in Tarasoff, I concluded that “[e]fforts to prophesy the perpetrators of violence appear to be doomed.” What a difference three decades make: the field of violence risk assessment has burgeoned and is now a vast and vibrant area of interdisciplinary scholarship.

John Monahan

Papers

The competence-related abilities of women criminal defendants

The Inclusion of Biological Risk Factors in Violence Risk Assessments

Lawyers at Mid-Career: A 20-Year Longitudinal Study of Job and Life Satisfaction

The ASA’s Missed Opportunity to Promote Sound Science in Court

Tarasoff at Thirty: How Developments in Science and Policy Shape the Common Law