As machine learning systems become ubiquitous, there has been a surge of interest in interpretable machine learning: systems that provide explanation for their outputs. These explanations are often used to qualitatively assess other criteria such as safety or non-discrimination. However, despite the interest in interpretability, there is very little consensus on what interpretable machine learning is and how it should be measured. In this position paper, we first define interpretability and describe when interpretability is needed (and when it is not). Next, we suggest a taxonomy for rigorous evaluation and expose open questions towards a more rigorous science of interpretable machine learning.

Towards A Rigorous Science of Interpretable Machine Learning

Explanation in artificial intelligence: Insights from the social sciences

People learning new concepts can often generalize successfully from just a single example, yet machine learning algorithms typically require tens or hundreds of examples to perform with similar accuracy. People can also use learned concepts in richer ways than conventional algorithms-for action, imagination, and explanation. We present a computational model that captures these human learning abilities for a large class of simple visual concepts: handwritten characters from the world's alphabets. The model represents concepts as simple programs that best explain observed examples under a Bayesian criterion. On a challenging one-shot classification task, the model achieves human-level performance while outperforming recent deep learning approaches. We also present several "visual Turing tests" probing the model's creative generalization abilities, which in many cases are indistinguishable from human behavior.

/pdf/human-level-concept-learning-through-probabilistic-program-59pdk5w1h1.pdf

Human-level concept learning through probabilistic program induction.

The cultural origins of human cognition

Recent progress in artificial intelligence has renewed interest in building systems that learn and think like people. Many advances have come from using deep neural networks trained end-to-end in tasks such as object recognition, video games, and board games, achieving performance that equals or even beats that of humans in some respects. Despite their biological inspiration and performance achievements, these systems differ from human intelligence in crucial ways. We review progress in cognitive science suggesting that truly human-like learning and thinking machines will have to reach beyond current engineering trends in both what they learn and how they learn it. Specifically, we argue that these machines should (1) build causal models of the world that support explanation and understanding, rather than merely solving pattern recognition problems; (2) ground learning in intuitive theories of physics and psychology to support and enrich the knowledge that is learned; and (3) harness compositionality and learning-to-learn to rapidly acquire and generalize knowledge to new tasks and situations. We suggest concrete challenges and promising routes toward these goals that can combine the strengths of recent neural network advances with more structured cognitive models.

/pdf/building-machines-that-learn-and-think-like-people-w8p3qwruiy.pdf

Building machines that learn and think like people.

http://www.fitelson.org/few/few_08/lombrozo_reading.pdf

The structure and function of explanations.

What makes some explanations better than others? This paper explores the roles of simplicity and probability in evaluating competing causal explanations. Four experiments investigate the hypothesis that simpler explanations are judged both better and more likely to be true. In all experiments, simplicity is quantified as the number of causes invoked in an explanation, with fewer causes corresponding to a simpler explanation. Experiment 1 confirms that all else being equal, both simpler and more probable explanations are preferred. Experiments 2 and 3 examine how explanations are evaluated when simplicity and probability compete. The data suggest that simpler explanations are assigned a higher prior probability, with the consequence that disproportionate probabilistic evidence is required before a complex explanation will be favored over a simpler alternative. Moreover, committing to a simple but unlikely explanation can lead to systematic overestimation of the prevalence of the cause invoked in the simple explanation. Finally, Experiment 4 finds that the preference for simpler explanations can be overcome when probability information unambiguously supports a complex explanation over a simpler alternative. Collectively, these findings suggest that simplicity is used as a basis for evaluating explanations and for assigning prior probabilities when unambiguous probability information is absent. More broadly, evaluating explanations may operate as a mechanism for generating estimates of subjective probability.

Simplicity and probability in causal explanation.

Functional explanation and the function of explanation

Recent neuropsychological and imaging data have implicated different brain networks in the processing of different word classes, nouns being linked primarily to posterior, visual object-processing regions and verbs to frontal, motor-processing areas However, as most of these studies have examined words in isolation, the consequences of such anatomically based representational differences, if any, for the processing of these items in sentences remains unclear Additionally, in some languages many words (eg \`drink') are class-ambiguous, ie they can play either role depending on context, and it is not yet known how the brain stores and uses information associated with such lexical items in context We examined these issues by recording event-related potentials (ERPs) in response to unambiguous nouns (eg \`beer'), unambiguous verbs (eg \`eat'), class-ambiguous words and pseudowords used as nouns or verbs within two types of minimally contrastive sentence contexts: noun-predicting (eg \`John wanted THE [target] but …') and verb-predicting (`John wanted TO [target] but …') Our results indicate that the nature of neural processing for nouns and verbs is a function of both the type of stimulus and the role it is playing Even when the context completely specifies their role, word class-ambiguous items differ from unambiguous ones over frontal regions by ~150 ms Moreover, whereas pseudowords elicit larger N400s when used as verbs than when used as nouns, unambiguous nouns and ambiguous words used as nouns elicit more frontocentral negativity than unambiguous verbs and ambiguous words used as verbs, respectively Additionally, unambiguous verbs elicit a left-lateralized, anterior positivity (~200 ms) not observed for any other stimulus type, though only when these items are used appropriately as verbs (ie in verb-predicting contexts) In summary, the pattern of neural activity observed in response to lexical items depends on their general probability of being a verb or a noun and on the particular role they are playing in any given sentence This implicates more than a simple two-way distinction of the brain networks involved in their storage and processing Experience, as well as context during on-line language processing, clearly shapes the neural representations of nouns and verbs, such that there is no single neural marker of word class Our results further suggest that the presence and nature of the word class-based dissociations observed after brain damage are similarly likely to be a function of both the type of stimulus and the context in which it occurs, and thus must be assessed accordingly

/pdf/brain-responses-to-nouns-verbs-and-class-ambiguous-words-in-53wztqkusa.pdf

Brain responses to nouns, verbs and class-ambiguous words in context

Everyday cognition reveals a sophisticated capacity to seek, generate, and evaluate explanations for the social and physical worlds around us. Why are we so driven to explain, and what accounts for our systematic explanatory preferences? This chapter reviews evidence from cognitive psychology and cognitive development concerning the structure and function of explanations, with a focus on the role of explanations in learning and inference. The findings highlight the value of understanding explanation and abductive inference both as phenomena in their own right and for the insights they provide concerning foundational aspects of human cognition, such as representation, learning, and inference.

Tania Lombrozo

Papers

The structure and function of explanations.

Simplicity and probability in causal explanation.

Functional explanation and the function of explanation

Brain responses to nouns, verbs and class-ambiguous words in context

Explanation and Abductive Inference