Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Co-Planar Stereotaxic Atlas of the Human Brain—3-Dimensional Proportional System: An Approach to Cerebral Imaging, J. Talairach, P. Tournoux. Georg Thieme Verlag, New York (1988), 122 pp., 130 figs. DM 268

I have developed "tennis elbow" from lugging this book around the past four weeks, but it is worth the pain, the effort, and the aspirin. It is also worth the (relatively speaking) bargain price. Including appendixes, this book contains 894 pages of text. The entire panorama of the neural sciences is surveyed and examined, and it is comprehensive in its scope, from genomes to social behaviors. The editors explicitly state that the book is designed as "an introductory text for students of biology, behavior, and medicine," but it is hard to imagine any audience, interested in any fragment of neuroscience at any level of sophistication, that would not enjoy this book. The editors have done a masterful job of weaving together the biologic, the behavioral, and the clinical sciences into a single tapestry in which everyone from the molecular biologist to the practicing psychiatrist can find and appreciate his or

Principles of Neural Science

How to Do Things With Words

Our actions and choices sometimes seem driven by our “needs”, sometimes by our “wants”, and sometimes by both. Needing is related to deprivation of something biologically significant, and wanting is linked to reward prediction and dopamine and usually has more power on behavioral activation than need states alone. Why sometimes there is an association and sometimes a dissociation between needing and wanting remains largely unknown. This paper aims to clarify the relations between needing and wanting, from the perspective of a normative framework to understand brain and cognition; namely, active inference. In this approach, needing is related to a deviation from preferred states that living things tend to occupy in order to reduce their surprise, while wanting is related to the precision over policies that lead to preferred (e.g., rewarding) states. Through a series of simulations, we demonstrate the interplay between needing and wanting systems. Specifically, our simulations show that when need states increase, the tendency to occupy preferred states is enhanced, independently of wanting (or reward prediction), therefore showing a dissociation between needing and wanting. Furthermore, the simulations show that when need states increase, the value of cues that signal reward achievement and the precision of the policies that lead to preferred states increase, suggesting that needs can amplify the value of a reward and its wanting. Taken together, our model and simulations help clarify the directional and underlying influence of need states separately from wanting and reward prediction; and how the same underlying influence of need amplifies wanting, i.e. increases the precision of reward cues that lead to the preferred state.

/pdf/a-computational-account-of-needing-and-wanting-2jc2m1k8.pdf

A computational account of needing and wanting

Beyond simple laboratory studies: developing sophisticated models to study rich behavior

Sequential sampling models of choice, such as the drift-diffusion model (DDM), are frequently fit to empirical data to account for a variety of effects related to choice accuracy/consistency and response time (RT). Sometimes, these models include extensions that can also account for choice confidence. However, no model in this class is able to account for the phenomenon of choice-induced preference change. Studies have reported choice-induced preference change for many decades, and the principle findings are robust: decision-makers tend to rate options higher after they choose them and lower after they reject them. This spreading of alternatives (SoA) in terms of their rated values is fundamentally incompatible with traditional sequential sampling models, which consider the rated values of the options to be stationary throughout choice deliberation. Here, we propose a simple modification of the basic DDM that allows the drift rate to vary across deliberation time depending on which attributes are attended to at which points in time. Critically, the model assumes that initial ratings are based only on the more salient attributes of the individual options, and that more attributes will be considered when decision-makers must choose between options with different salient attributes. We show that this model can account for SoA (in addition to choice consistency and RT), as well as all previously reported relationships between SoA and choice difficulty, attribute disparity, and RT.

Choice-Induced Preference Change under a Sequential Sampling Model Framework

Goals distort the representation of space

Flexible navigation relies on a cognitive map of space, thought to be implemented by hippocampal place cells: neurons that exhibit location-specific firing. In connected environments, optimal navigation requires keeping track of one’s location and of the available connections between subspaces. We examined whether the dorsal CA1 place cells of rats encode environmental connectivity in four geometrically-identical boxes arranged in a square. Rats moved between boxes by pushing saloon-type doors that could be locked in one or both directions. While rats demonstrated knowledge of environmental connectivity, their place cells did not respond to connectivity changes, nor did they represent doorways differently from other locations. Importantly, place cells coded the space in a global frame, expressing minimal repetitive fields despite the repetitive geometry (global coding). These results suggest that CA1 place cells provide a spatial map that does not explicitly include connectivity.

Giovanni Pezzulo

Papers

A computational account of needing and wanting

Beyond simple laboratory studies: developing sophisticated models to study rich behavior

Choice-Induced Preference Change under a Sequential Sampling Model Framework

Goals distort the representation of space

Hippocampal place cells encode global location but not changes in environmental connectivity in a 4-room navigation task