Black box machine learning models are currently being used for high-stakes decision making throughout society, causing problems in healthcare, criminal justice and other domains. Some people hope that creating methods for explaining these black box models will alleviate some of the problems, but trying to explain black box models, rather than creating models that are interpretable in the first place, is likely to perpetuate bad practice and can potentially cause great harm to society. The way forward is to design models that are inherently interpretable. This Perspective clarifies the chasm between explaining black boxes and using inherently interpretable models, outlines several key reasons why explainable black boxes should be avoided in high-stakes decisions, identifies challenges to interpretable machine learning, and provides several example applications where interpretable models could potentially replace black box models in criminal justice, healthcare and computer vision. There has been a recent rise of interest in developing methods for ‘explainable AI’, where models are created to explain how a first ‘black box’ machine learning model arrives at a specific decision. It can be argued that instead efforts should be directed at building inherently interpretable models in the first place, in particular where they are applied in applications that directly affect human lives, such as in healthcare and criminal justice.

Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead

Nos últimos trinta trinta anos, houve profundas mudanças na forma como compreendemos o crime e a justiça criminal. O crime tornou-se um evento simbólico, um verdadeiro teste para a ordem social e para as políticas governamentais, um desafio para a sociedade civil, para a democracia e para os direitos humanos. Segundo David Garland, professor da Faculdade de Direito da New York University, um dos principais autores no campo da Sociologia da Punição e com artigo publicado na Revista de Sociologia e Política , número 13, na modernidade tardia houve uma verdadeira obsessão securitária, direcionando as políticas criminais para um maior rigor em relação às penas e maior intolerância com o criminoso. Há trinta anos, nos EUA e na Inglaterra essa tendência era insuspeita. O livro mostra que os dois países compartilham intrigantes similaridades em suas práticas criminais, a despeito da divisão racial, das desigualdades econômicas e da letalidade violenta que marcam fortemente o cenário americano. Segundo David Garland, encontram-se nos dois países os “mesmos tipos de riscos e inseguranças, a mesma percepção a respeito dos problemas de um controle social não-efetivo, as mesmas críticas da justiça criminal tradicional, e as mesmas ansiedades recorrentes sobre mudança e ordem sociais”1 (GARLAND, 2001, p. 2). O argumento principal da obra é o seguinte: a modernidade tardia, esse distintivo padrão de relações sociais, econômicas e culturais, trouxe consigo um conjunto de riscos, inseguranças e problemas de controle social que deram uma configuração específica às nossas respostas ao crime, ao garantir os altos custos das políticas criminais, o grau máximo de duração das penas e a excessivas taxas de encarceramento.

http://www.scielo.br/pdf/rsocp/n20/n20a15.pdf

The culture of control: crime and social order in contemporary, society

Machine learning is a means to derive artificial intelligence by discovering patterns in existing data. Here, we show that applying machine learning to ordinary human language results in human-like semantic biases. We replicated a spectrum of known biases, as measured by the Implicit Association Test, using a widely used, purely statistical machine-learning model trained on a standard corpus of text from the World Wide Web. Our results indicate that text corpora contain recoverable and accurate imprints of our historic biases, whether morally neutral as toward insects or flowers, problematic as toward race or gender, or even simply veridical, reflecting the status quo distribution of gender with respect to careers or first names. Our methods hold promise for identifying and addressing sources of bias in culture, including technology.

/pdf/semantics-derived-automatically-from-language-corpora-4lzxrze2vb.pdf

Semantics derived automatically from language corpora contain human-like biases

With the widespread use of AI systems and applications in our everyday lives, it is important to take fairness issues into consideration while designing and engineering these types of systems. Such systems can be used in many sensitive environments to make important and life-changing decisions; thus, it is crucial to ensure that the decisions do not reflect discriminatory behavior toward certain groups or populations. We have recently seen work in machine learning, natural language processing, and deep learning that addresses such challenges in different subdomains. With the commercialization of these systems, researchers are becoming aware of the biases that these applications can contain and have attempted to address them. In this survey we investigated different real-world applications that have shown biases in various ways, and we listed different sources of biases that can affect AI applications. We then created a taxonomy for fairness definitions that machine learning researchers have defined in order to avoid the existing bias in AI systems. In addition to that, we examined different domains and subdomains in AI showing what researchers have observed with regard to unfair outcomes in the state-of-the-art methods and how they have tried to address them. There are still many future directions and solutions that can be taken to mitigate the problem of bias in AI systems. We are hoping that this survey will motivate researchers to tackle these issues in the near future by observing existing work in their respective fields.

A Survey on Bias and Fairness in Machine Learning

Recidivism prediction instruments (RPIs) provide decision-makers with an assessment of the likelihood that a criminal defendant will reoffend at a future point in time. Although such instruments are gaining increasing popularity across the country, their use is attracting tremendous controversy. Much of the controversy concerns potential discriminatory bias in the risk assessments that are produced. This article discusses several fairness criteria that have recently been applied to assess the fairness of RPIs. We demonstrate that the criteria cannot all be simultaneously satisfied when recidivism prevalence differs across groups. We then show how disparate impact can arise when an RPI fails to satisfy the criterion of error rate balance.

Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments

Although risk assessment has increasingly been used as a tool to help reform the criminal justice system, some stakeholders are adamantly opposed to using algorithms. The principal concern is that any benefits achieved by safely reducing rates of incarceration will be offset by costs to racial justice claimed to be inherent in the algorithms themselves. But fairness trade-offs are inherent to the task of predicting recidivism, whether the prediction is made by an algorithm or human. Based on a matched sample of 67,784 Black and White federal supervisees assessed with the Post Conviction Risk Assessment, we compared how three alternative strategies for "debiasing" algorithms affect these trade-offs, using arrest for a violent crime as the criterion . These candidate algorithms all strongly predict violent reoffending (areas under the curve = 0.71-72), but vary in their association with race (r = 0.00-0.21) and shift trade-offs between balance in positive predictive value and false-positive rates. Providing algorithms with access to race (rather than omitting race or "blinding" its effects) can maximize calibration and minimize imbalanced error rates. Implications for policymakers with value preferences for efficiency versus equity are discussed.

Using algorithms to address trade‐offs inherent in predicting recidivism

Actuarial and clinical assessment of criminogenic needs: identifying supervision priorities among federal probation officers

Existing research on the activities of police officers has examined a range of police behaviors and decision processes. The present article attempts to delineate factors that influence the daily discretionary choices of officers concerning the activities they undertake on a daily basis. By empirically examining the contribution of officer and community factors on decisions to engage in a variety of typical police activities, this article provides a more complete understanding of factors that influence the work-related decisions of street-level police officers. Using systematic social observation data on both community-oriented and traditional beat officers, multivariate models are developed to explain officers’ daily activities. The findings reveal that neighborhood-level and attitude variables influenced community and traditional police officers differently. Overall, the individual-level attributes of officers did not influence activity choices of either type of officer. These findings are discussed in t...

Explaining Police Officer Discretionary Activity

Research Summary

The Post Conviction Risk Assessment (PCRA) is a correctional assessment tool used by federal probation officers that identifies offenders most likely to commit new crimes and the criminogenic characteristics that, if changed, could reduce the likelihood of recidivism. We explored how changes in offender risk influence the likelihood of recidivism by tracking a population of 64,716 offenders placed on federal supervision with multiple PCRA assessments. In general, offenders scoring in the high-, moderate-, and low/moderate-risk categories at their initial assessment and experiencing decreases in their risk classifications were less likely to recidivate compared with their counterparts whose risk levels remained unchanged or increased. Conversely, increases in offender risk were associated with higher rates of reoffending behavior. Notably, we saw no recidivism reduction for offenders in the lowest risk category if they received decreases in their overall PCRA scores.



Policy Implications

This analysis provides officers with crucial information about how changes in offender risk can influence the likelihood of arrest. Probation officers should consider adjusting downward the amount of time and resources devoted to offenders with decreasing risk levels once those decreases have stabilized. Alternatively, probation officers should pay particular attention and allocate more resources to those offenders reclassified into higher risk categories. Finally, probation officers should be cautious about providing resources to low-risk offenders who do not seem to benefit from efforts aimed at reducing their criminal risk factors.

Does Change in Risk Matter

Offender assessment in corrections has advanced considerably over the last several decades. Currently, it is not uncommon to find correctional professionals using any number of objective standardized assessment instruments. While many of these instruments possess face validity as well as statistical predictive validity, more work is needed to test classification protocol on new populations, and in various correctional environments. The current paper investigates the predictive validity of the Youth Level of Service/Case Management Inventory (YLS/CMI) within an institutional setting. Specifically, the composite score rendered from the YLS/CMI is used to predict institutional misconduct. The YLS/ CMI was found to effectively differentiate between two levels of offender risk/ need, and was significantly related to outcome using several different statistical techniques.

Christopher T. Lowenkamp

Papers

Using algorithms to address trade‐offs inherent in predicting recidivism

Actuarial and clinical assessment of criminogenic needs: identifying supervision priorities among federal probation officers

Explaining Police Officer Discretionary Activity

Does Change in Risk Matter

Predicting institutional misconduct using the Youth Level of Service/ Case Management Inventory