https://ccrma.stanford.edu/workshops/mir2009/references/ROCintro.pdf

An introduction to ROC analysis

A review is presented of the book “Heuristics and Biases: The Psychology of Intuitive Judgment,” edited by Thomas Gilovich, Dale Griffin, and Daniel Kahneman.

Heuristics and Biases: The Psychology of Intuitive Judgment

Much research in the last two decades has demon- strated that human responses deviate from the performance deemed normative according to various models of decision mak- ing and rational judgment (e.g., the basic axioms of utility theory). This gap between the normative and the descriptive can be inter- preted as indicating systematic irrationalities in human cognition. However, four alternative interpretations preserve the assumption that human behavior and cognition is largely rational. These posit that the gap is due to (1) performance errors, (2) computational limitations, (3) the wrong norm being applied by the experi- menter, and (4) a different construal of the task by the subject. In the debates about the viability of these alternative explanations, attention has been focused too narrowly on the modal response. In a series of experiments involving most of the classic tasks in the heuristics and biases literature, we have examined the implica- tions of individual differences in performance for each of the four explanations of the normative/descriptive gap. Performance er- rors are a minor factor in the gap; computational limitations un- derlie non-normative responding on several tasks, particularly those that involve some type of cognitive decontextualization. Un- expected patterns of covariance can suggest when the wrong norm is being applied to a task or when an alternative construal of the task should be considered appropriate.

https://www.cambridge.org/core/services/aop-cambridge-core/content/view/S0140525X00003435

Individual differences in reasoning: Implications for the rationality debate?

In this review, we examine the oft-made claim that peer-relationship difficulties in childhood predict serious adjustment problems in later life. The article begins with a framework for conceptualizing and assessing children's peer difficulties and with a discussion of conceptual and methodological issues in longitudinal risk research. Following this, three indexes of problematic peer relationships (acceptance, aggressiveness, and shyness/withdrawal) are evaluated as predictors of three later outcomes (dropping out of school, criminality, and psychcpathology). The relation between peer difficulties and later maladjustment is examined in terms of both the consistency and strength of prediction. A review and analysis of the literature indicates general support for the hypothesis that children with poor peer adjustment are at risk for later life difficulties. Support is clearest for the outcomes of dropping out and criminality. It is also clearest for low acceptance and aggressiveness as predictors, whereas a link between shyness/withdrawal and later maladjustment has not yet been adequately tested. The article concludes with a critical discussion of the implicit models that have guided past research in this area and a set of recommendations for the next generation of research on the risk

Peer relations and later personal adjustment: Are low-accepted children at risk?

Modern theories in cognitive psychology and neuroscience indicate that there are two fundamental ways in which human beings comprehend risk. The analytic system uses algorithms and normative rules, such as probability calculus, formal logic, and risk assessment. It is relatively slow, effortful, and requires conscious control. The experiential system is intuitive, fast, mostly automatic, and not very accessible to conscious awareness. The experiential system enabled human beings to survive during their long period of evolution and remains today the most natural and most common way to respond to risk. It relies on images and associations, linked by experience to emotion and affect (a feeling that something is good or bad). This system represents risk as a feeling that tells us whether it's safe to walk down this dark street or drink this strange-smelling water. Proponents of formal risk analysis tend to view affective responses to risk as irrational. Current wisdom disputes this view. The rational and the experiential systems operate in parallel and each seems to depend on the other for guidance. Studies have demonstrated that analytic reasoning cannot be effective unless it is guided by emotion and affect. Rational decision making requires proper integration of both modes of thought. Both systems have their advantages, biases, and limitations. Now that we are beginning to understand the complex interplay between emotion and reason that is essential to rational behavior, the challenge before us is to think creatively about what this means for managing risk. On the one hand, how do we apply reason to temper the strong emotions engendered by some risk events? On the other hand, how do we infuse needed "doses of feeling" into circumstances where lack of experience may otherwise leave us too "coldly rational"? This article addresses these important questions.

https://papers.ssrn.com/sol3/Delivery.cfm?abstractid=931686

Risk As Analysis and Risk As Feelings: Some Thoughts About Affect, Reason, Risk, and Rationality

University of VirginiaThis article describes the development of a new clinical instrument for use in assessments of adultcriminal defendants' competence to proceed to adjudication, the MacArthur Competence AssessmentTool-Criminal Adjudication (MacCAT-CA). The MacCAT-CA was derived from a more comprehen-sive research instrument (MacArthur Structured Assessment of Competencies of Criminal Defendants;Hoge, Bonnie, Poythress, Monahan, & Eisenberg, 1997) on the basis of considerations efface validityfor use in legal contexts, psychometric analyses, and advice from mental health experts who reviewedan earlier prototype. This article presents the results from an National Institute of Mental Health-sponsored validation study that investigated the psychometric properties of the MacCAT-CA.At least 25,000 criminal defendants are referred annually forevaluation of their competence to participate in legal proceed-ings (Steadman & Hartstone, 1983). Although a number ofmeasures have been specifically designed to assess defendants'capacities in this area (e.g., Competence Screening Test [CST],Lipsitt, Lelos, & McGarry, 1971; Georgia Court CompetencyTest [GCCT], Wildman et al., 1980; CADCOMP, Barnard etal., 1991; Competency Assessment Instrument [CAI], Labora-tory of Community Psychiatry, 1974; Interdisciplinary FitnessInterview [IFI], Golding, Roesch, & Schreiber, 1984; Fitnessto Stand Trial Interview, Menzies, Webster, Roesch, Jensen, R RobertA. Nicholson, Department of Psychology, University of Tulsa; JohnMonahan and Richard J. Bonnie, School of Law, University of Virginia;Steven K. Hoge, School of Law and School of Medicine, University ofVirginia; Marlene Eisenberg, Institute of Law, Psychiatry, and PublicPolicy, University of Virginia.John F. Edens is now at the Department of Psychology, Sam HoustonState University.This research was supported by the MacArthur Foundation ResearchNetwork on Mental Health and the Law, and by Grant RO1MH54517-O1A1 from the National Institute of Mental Health. We are grateful toTom Feucht-Haviar for his assistance on a pilot test of this measure.Norman G. Poythress, John Monahan, Richard J. Bonnie, and StevenK. Hoge are authors of the MacArthur Competence Assessment Tool-Criminal Adjudication (MacCAT-CA) published by Psychological As-sessment Resources. Randy K. Otto, Norman G. Poythress, Robert A.Nicholson, John F. Edens, John Monahan, Richard J. Bonnie, and StevenK. Hoge are authors of the MacCAT-CA manual, also published byPsychological Assessment Resources.Correspondence concerning this article should be addressed to RandyK. Otto, Department of Mental Health Law and Policy, Florida MentalHealth Institute, University of South Florida, 13301 Bruce B. DownsBoulevard, Tampa, Florida 33612-3899. Electronic mail may be sent toorto@hal.fmhi.usf.edu.Eaves, 1984), each exhibits various limitations (Grisso, 1991;Melton, Petrila, Poythress, & Slobogin, 1997; Nicholson, 1992).Briefly, some measures were designed to serve primarily asscreening instruments (e.g., CST and GCCT) and appear mainlyto be collections of items having various legal content but littleor no underlying conceptual structure. Others (e.g., CAI and IFI)lack standardized administration and criterion-based scoring,relying instead on the judgment of the clinician to determine,on a case-by-case basis, which questions to pose to the defendantand how to rate the responses received. None is designed toprovide quantitative indexes of discrete competence-related abil-ities, nor have interpretative norms based on large, nationalsamples been developed for any of these measures.Given the pivotal role of a defendant's competence, one of themajor research initiatives of the MacArthur Foundation ResearchNetwork on Mental Health and the Law was the developmentof a standardized research instrument for evaluating criminaldefendants' psycholegal abilities related to competence to pro-ceed to adjudication. The MacArthur Structured Assessmentof Competencies of Criminal Defendants (MacSAC-CD; Hoge,Bonnie, Poythress, Monahan, & Eisenberg, 1997) was based onBonnie's (1992, 1993) comprehensive theory of legal compe-tence, and it contains measures of discrete competence-relatedabilities. Items in the MacSAC-CD were modeled after thosecreated by Grisso, Appelbaum, Mulvey, and Fletcher (1995) forthe assessment of competence to consent to treatment (see, moregenerally, Grisso & Appelbaum, 1998). Each measure in theMacSAC-CD involves standardize d administration an crite-rion-based scoring, thus reducing much of the discretionary(and potentially idiosyncratic) scoring that plagues existingmeasures.Hoge et al. (1997) recently described the MacSAC-CDand presented the primary findings from a field study that in-volved both incompetent and competent defendants as researchparticipants. The results of this field study indicated that the

Psychometric Properties of the MacArthur Competence Assessment Tool-Criminal Adjudication

Mental health courts (MHCs) represent an important new development at the interface of the criminal justice and mental health systems. MHCs are criminal courts for persons with mental illness that were in part created to divert this population from jail/prison into community treatment. MHCs are proliferating rapidly despite limited knowledge regarding their characteristics or their efficacy. We surveyed the entire population of adult MHCs in the United States, n = 90. In the past 8 years, MHCs have been created in 34 states, with an aggregate current caseload of 7,560 clients in MHCs nationally. Most courts (92%) reported using jail as a sanction for noncompliance, if only rarely. Further, jail sanction use was significantly predicted by increased judicial supervision and number of felons in the court. Implications for MHCs and social monitoring are discussed.

Patterns of practice in mental health courts: A national survey.

Perceptions of coercion in the admission of voluntary and involuntary psychiatric patients

We administered a semi-structured interview to 157 patients shortly after their admission to a psychiatric hospital. In the first, and open-ended, part of the interview, patients were asked to talk about what had been going on in their lives that led to their coming into the hospital. Then, in a more structured format, they were asked more specific details about who was involved, the patients' relationships with those involved, whether any attempts were made to influence the patient to come into the hospital, and whether such attempts were perceived as fair by the patient. This article presents a qualitative review of the transcripts of a subset of these interviews. It attends specifically to patients' perceptions of the morality of attempts by others--primarily family members, friends and mental health professionals--to influence them to be admitted to the hospital, and of the morality of the process by which these influence attempts resulted in admission. Language: en

Inclusion, motivation, and good faith: the morality of coercion in mental hospital admission

OBJECTIVE: An influential rationale for involuntary hospitalization is that prospective patients who refuse hospitalization at the time it is offered are likely to change their belief about the necessity of hospitalization after receiving hospital treatment. The authors examine how patients changed their evaluations of psychiatric hospitalization following hospital treatment. METHOD: The authors studied 433 patients who were interviewed about their hospitalization within 2 days of their admission to a psychiatric hospital; 267 of these patients were reinterviewed 4–8 weeks following discharge. RESULTS: When reinterviewed at follow-up, 33 (52%) of 64 patients who said at admission that they did not need hospitalization said that, in retrospect, they believed they had needed it. Only 9 (5%) of 198 patients who said at admission that they needed hospitalization shifted to saying that they had not needed it. CONCLUSIONS: Many of the patients who initially judged that they did not need hospitalization revised ...

https://ajp.psychiatryonline.org/doi/pdf/10.1176/ajp.156.9.1385

John Monahan

Papers

Psychometric Properties of the MacArthur Competence Assessment Tool-Criminal Adjudication

Patterns of practice in mental health courts: A national survey.

Perceptions of coercion in the admission of voluntary and involuntary psychiatric patients

Inclusion, motivation, and good faith: the morality of coercion in mental hospital admission

Patients' revisions of their beliefs about the need for hospitalization.