https://ccrma.stanford.edu/workshops/mir2009/references/ROCintro.pdf

An introduction to ROC analysis

A review is presented of the book “Heuristics and Biases: The Psychology of Intuitive Judgment,” edited by Thomas Gilovich, Dale Griffin, and Daniel Kahneman.

Heuristics and Biases: The Psychology of Intuitive Judgment

Much research in the last two decades has demon- strated that human responses deviate from the performance deemed normative according to various models of decision mak- ing and rational judgment (e.g., the basic axioms of utility theory). This gap between the normative and the descriptive can be inter- preted as indicating systematic irrationalities in human cognition. However, four alternative interpretations preserve the assumption that human behavior and cognition is largely rational. These posit that the gap is due to (1) performance errors, (2) computational limitations, (3) the wrong norm being applied by the experi- menter, and (4) a different construal of the task by the subject. In the debates about the viability of these alternative explanations, attention has been focused too narrowly on the modal response. In a series of experiments involving most of the classic tasks in the heuristics and biases literature, we have examined the implica- tions of individual differences in performance for each of the four explanations of the normative/descriptive gap. Performance er- rors are a minor factor in the gap; computational limitations un- derlie non-normative responding on several tasks, particularly those that involve some type of cognitive decontextualization. Un- expected patterns of covariance can suggest when the wrong norm is being applied to a task or when an alternative construal of the task should be considered appropriate.

https://www.cambridge.org/core/services/aop-cambridge-core/content/view/S0140525X00003435

Individual differences in reasoning: Implications for the rationality debate?

In this review, we examine the oft-made claim that peer-relationship difficulties in childhood predict serious adjustment problems in later life. The article begins with a framework for conceptualizing and assessing children's peer difficulties and with a discussion of conceptual and methodological issues in longitudinal risk research. Following this, three indexes of problematic peer relationships (acceptance, aggressiveness, and shyness/withdrawal) are evaluated as predictors of three later outcomes (dropping out of school, criminality, and psychcpathology). The relation between peer difficulties and later maladjustment is examined in terms of both the consistency and strength of prediction. A review and analysis of the literature indicates general support for the hypothesis that children with poor peer adjustment are at risk for later life difficulties. Support is clearest for the outcomes of dropping out and criminality. It is also clearest for low acceptance and aggressiveness as predictors, whereas a link between shyness/withdrawal and later maladjustment has not yet been adequately tested. The article concludes with a critical discussion of the implicit models that have guided past research in this area and a set of recommendations for the next generation of research on the risk

Peer relations and later personal adjustment: Are low-accepted children at risk?

Modern theories in cognitive psychology and neuroscience indicate that there are two fundamental ways in which human beings comprehend risk. The analytic system uses algorithms and normative rules, such as probability calculus, formal logic, and risk assessment. It is relatively slow, effortful, and requires conscious control. The experiential system is intuitive, fast, mostly automatic, and not very accessible to conscious awareness. The experiential system enabled human beings to survive during their long period of evolution and remains today the most natural and most common way to respond to risk. It relies on images and associations, linked by experience to emotion and affect (a feeling that something is good or bad). This system represents risk as a feeling that tells us whether it's safe to walk down this dark street or drink this strange-smelling water. Proponents of formal risk analysis tend to view affective responses to risk as irrational. Current wisdom disputes this view. The rational and the experiential systems operate in parallel and each seems to depend on the other for guidance. Studies have demonstrated that analytic reasoning cannot be effective unless it is guided by emotion and affect. Rational decision making requires proper integration of both modes of thought. Both systems have their advantages, biases, and limitations. Now that we are beginning to understand the complex interplay between emotion and reason that is essential to rational behavior, the challenge before us is to think creatively about what this means for managing risk. On the one hand, how do we apply reason to temper the strong emotions engendered by some risk events? On the other hand, how do we infuse needed "doses of feeling" into circumstances where lack of experience may otherwise leave us too "coldly rational"? This article addresses these important questions.

https://papers.ssrn.com/sol3/Delivery.cfm?abstractid=931686

Risk As Analysis and Risk As Feelings: Some Thoughts About Affect, Reason, Risk, and Rationality

Better decisions through science.

OBJECTIVES: The authors examined Americans' opinions about financial and treatment competence of people with mental health problems, potential for harm to self or others, and the use of legal means to force treatment. METHODS: The 1996 General Social Survey provided interview data with a nationally representative sample (n = 1444). Respondents were given a vignette based on diagnostic criteria for schizophrenia, major depression, alcohol dependence, or drug dependence, or a "control" case. RESULTS: The specific nature of the problem was the most important factor shaping public reaction. Respondents viewed those with "troubles," alcohol dependence, or depression as able to make treatment decisions. Most reported that persons with alcohol or drug problems or schizophrenia cannot manage money and are likely to be violent toward others. Respondents indicated a willingness to coerce individuals into treatment. Respondent and other case characteristics rarely affected opinions. CONCLUSIONS: Americans report gre...

The public's view of the competence, dangerousness, and need for legal coercion of persons with mental health problems.

'...essential reading for those confronted with the ethical and professional dilemmas involved in predicting violent behavior. Lawyers are destined to become familiar with Monahan's book, and mental health professionals will surely want to keep a step ahead.' -- Contemporary Psychology, Vol 27 No 2 '...In summary, Monahan's book is a very readable and succinct one. Often the reader finds himself saying "...well of course, what could be more obvious?" only to reflect for a minute and realize that many clinicians do not give many obvious relevant factors adequate weight in their assessments of dangerousness. Monahan's text is a very positive one which as he puts it, outlines for the clinician: "How to do it (predict violent behavior) and when to do it". Violent behavior must be viewed as the product of a dynamic interaction between an individual and situational/environmental variables.' -- Canadian Journal of Criminology, Vol 24 No 3, July 1982 'Monahan's is an important and well-documented book that deserves to be read by all concerned with the practice and politics of predicting violent behavior.' -- Contemporary Sociology, Vol 12 No 2, March 1983 'Monahan has taken a positive approach to a difficult area. He has delineated what can be done, and how, rather than wringing his hands about the problem.' -- Journal of Security Administration, Vol.9,No. 2, December 1986@Q

Predicting Violent Behavior: An Assessment of Clinical Techniques

This article describes studies designed to inform policy makers and practitioners about factors influencing the validity of violence risk assessment and risk communication. Forensic psychologists and psychiatrists were shown case summaries of patients hospitalized with mental disorder and were asked to judge the likelihood that the patient would harm someone within six months after discharge from the hospital. They also judged whether the patient posed a high risk, medium risk, or low risk of harming someone after discharge. Studies 1 and 2 replicated, with real case summaries as stimuli, the response-scale effects found by Slovic and Monahan (1995). Providing clinicians with response scales allowing more discriminability among smaller probabilities led patients to be judged as posing lower probabilities of committing harmful acts. This format effect was not eliminated by having clinicians judge relative frequencies rather than probabilities or by providing them with instruction in how to make these types of judgments. In addition, frequency scales led to lower mean likelihood judgments than did probability scales, but, at any given level of likelihood, a patient was judged as posing higher risk if that likelihood was derived from a frequency scale (e.g., 10 out of 100) than if it was derived from a probability scale (e.g., 10%). Similarly, communicating a patient's dangerousness as a relative frequency (e.g., 2 out of 10) led to much higher perceived risk than did communicating a comparable probability (e.g., 20%). The different reactions to probability and frequency formats appear to be attributable to the more frightening images evoked by frequencies. Implications for risk assessment and risk communication are discussed.

Violence risk assessment and risk communication: the effects of using actual cases, providing instruction, and employing probability versus frequency formats.

OBJECTIVE: Previous work has suggested that delusions are associated with a higher risk of violence, particularly delusions in which patients believe that people are seeking to harm them or that outside forces are controlling their minds (denoted as “threat/control override” delusions). This study explores the relationship between delusions and violence among patients recently discharged from acute psychiatric hospitalization. METHOD: Data were drawn from the MacArthur Violence Risk Assessment Study, a study of violence in the community that followed 1,136 recently discharged psychiatric patients for 1 year. Interviews at discharge and at five 10-week intervals gathered clinical, historical, situational, and dispositional information, including the presence and nature of delusional thoughts. Violence was ascertained from reports of subjects, collateral informants, and official records. RESULTS: Neither delusions in general nor threat/control override delusions in particular were associated with a higher r...

https://ajp.psychiatryonline.org/doi/pdfplus/10.1176/appi.ajp.157.4.566

John Monahan

Papers

Better decisions through science.

The public's view of the competence, dangerousness, and need for legal coercion of persons with mental health problems.

Predicting Violent Behavior: An Assessment of Clinical Techniques

Violence risk assessment and risk communication: the effects of using actual cases, providing instruction, and employing probability versus frequency formats.

Violence and delusions: data from the MacArthur Violence Risk Assessment Study.