Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

▪ Abstract The prefrontal cortex has long been suspected to play an important role in cognitive control, in the ability to orchestrate thought and action in accordance with internal goals. Its neural basis, however, has remained a mystery. Here, we propose that cognitive control stems from the active maintenance of patterns of activity in the prefrontal cortex that represent goals and the means to achieve them. They provide bias signals to other brain structures whose net effect is to guide the flow of activity along neural pathways that establish the proper mappings between inputs, internal states, and outputs needed to perform a given task. We review neurophysiological, neurobiological, neuroimaging, and computational studies that support this theory and discuss its implications as well as further issues to be addressed

/pdf/an-integrative-theory-of-prefrontal-cortex-function-2j9unprn5f.pdf

An integrative theory of prefrontal cortex function

Publisher Summary This chapter focuses on the modern notion of short-term memory, called working memory. Working memory refers to the temporary maintenance of information that was just experienced or just retrieved from long-term memory but no longer exists in the external environment. These internal representations are short-lived, but can be maintained for longer periods of time through active rehearsal strategies, and can be subjected to various operations that manipulate the information in such a way that makes it useful for goal-directed behavior. Working memory is a system that is critically important in cognition and seems necessary in the course of performing many other cognitive functions, such as reasoning, language comprehension, planning, and spatial processing. This chapter demonstrates the functional importance of dopamine to working memory function in several ways. Elucidation of the cognitive and neural mechanisms underlying human working memory is an important focus of cognitive neuroscience and neurology for much of the past decade. One conclusion that arises from research is that working memory, a faculty that enables temporary storage and manipulation of information in the service of behavioral goals, can be viewed as neither a unitary, nor a dedicated system. Data from numerous neuropsychological and neurophysiological studies in animals and humans demonstrates that a network of brain regions, including the prefrontal cortex, is critical for the active maintenance of internal representations.

Chapter 11 Working memory

Co-Planar Stereotaxic Atlas of the Human Brain—3-Dimensional Proportional System: An Approach to Cerebral Imaging, J. Talairach, P. Tournoux. Georg Thieme Verlag, New York (1988), 122 pp., 130 figs. DM 268

Despite the concern that has been expressed about potential method biases, and the pervasiveness of research settings with the potential to produce them, there is disagreement about whether they really are a problem for researchers in the behavioral sciences. Therefore, the purpose of this review is to explore the current state of knowledge about method biases. First, we explore the meaning of the terms “method” and “method bias” and then we examine whether method biases influence all measures equally. Next, we review the evidence of the effects that method biases have on individual measures and on the covariation between different constructs. Following this, we evaluate the procedural and statistical remedies that have been used to control method biases and provide recommendations for minimizing method bias.

Sources of Method Bias in Social Science Research and Recommendations on How to Control It

This article considers the role of the hippocampus in memory function. A central thesis is that work with rats, monkeys, and humans--which has sometimes seemed to proceed independently in 3 separate literatures--is now largely in agreement about the function of the hippocampus and related structures. A biological perspective is presented, which proposes multiple memory systems with different functions and distinct anatomical organizations. The hippocampus (together with anatomically related structures) is essential for a specific kind of memory, here termed declarative memory (similar terms include explicit and relational). Declarative memory is contrasted with a heterogeneous collection of nondeclarative (implicit) memory abilities that do not require the hippocampus (skills and habits, simple conditioning, and the phenomenon of priming). The hippocampus is needed temporarily to bind together distributed sites in neocortex that together represent a whole memory.

Memory and the hippocampus: A synthesis from findings with rats, monkeys, and humans.

Studies of human amnesia and studies of an animal model of human amnesia in the monkey have identified the anatomical components of the brain system for memory in the medial temporal lobe and have illuminated its function. This neural system consists of the hippocampus and adjacent, anatomically related cortex, including entorhinal, perirhinal, and parahippocampal cortices. These structures, presumably by virtue of their widespread and reciprocal connections with neocortex, are essential for establishing long-term memory for facts and events (declarative memory). The medial temporal lobe memory system is needed to bind together the distributed storage sites in neocortex that represent a whole memory. However, the role of this system is only temporary. As time passes after learning, memory stored in neocortex gradually becomes independent of medial temporal lobe structures.

/pdf/the-medial-temporal-lobe-memory-system-20orw9c62s.pdf

The medial temporal lobe memory system

The medial temporal lobe includes a system of anatomically related structures that are essential for declarative memory (conscious memory for facts and events). The system consists of the hippocampal region (CA fields, dentate gyrus, and subicular complex) and the adjacent perirhinal, entorhinal, and parahippocampal cortices. Here, we review findings from humans, monkeys, and rodents that illuminate the function of these structures. Our analysis draws on studies of human memory impairment and animal models of memory impairment, as well as neurophysiological and neuroimaging data, to show that this system (a) is principally concerned with memory, (b) operates with neocortex to establish and maintain long-term memory, and (c) ultimately, through a process of consolidation, becomes independent of long-term memory, though questions remain about the role of perirhinal and parahippocampal cortices in this process and about spatial memory in rodents. Data from neurophysiology, neuroimaging, and neuroanatomy point to a division of labor within the medial temporal lobe. However, the available data do not support simple dichotomies between the functions of the hippocampus and the adjacent medial temporal cortex, such as associative versus nonassociative memory, episodic versus semantic memory, and recollection versus familiarity.

/pdf/the-medial-temporal-lobe-34yqyq0q6b.pdf

The medial temporal lobe

Amnesic patients acquired a mirror-reading skill at a rate equivalent to that of matched control subjects and retained it for at least 3 months. The results indicate that the class of preserved learning skills in amnesia is broader than previously reported. Amnesia seems to spare information that is based on rules or procedures, as contrasted with information that is data-based or declarative--"knowing how rather than "knowing that." The results support the hypothesis that such a distinction is honored by the nervous system.

https://science.sciencemag.org/content/sci/210/4466/207.full.pdf

Preserved learning and retention of pattern-analyzing skill in amnesia: dissociation of knowing how and knowing that

Definitions: from synapses to behaviour Memory as synaptic change Memory and the developing nervous system Modulation of memory Localised and distributed memory storage The Penfield studies Searching for engrams: simple learning Searching for engrams: complex learning Memory determined by information processing Short-term and long-term memory Identifying a neural system with memory functions Amnesia and the functional organization of memory Prefrontal cortex Memory and brain: a beginning.

Larry R. Squire

Papers

Memory and the hippocampus: A synthesis from findings with rats, monkeys, and humans.

The medial temporal lobe memory system

The medial temporal lobe

Preserved learning and retention of pattern-analyzing skill in amnesia: dissociation of knowing how and knowing that

Memory and Brain