Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

http://science.sciencemag.org/content/sci/201/4359/899.2.full.pdf

Phenomenology of perception.

/pdf/asymptotic-analysis-of-periodic-structures-14wjonu4ba.pdf

Asymptotic Analysis of Periodic Structures

The goals of this chapter are (1) to outline and substantiate a broad conceptualization of what it means to think mathematically, (2) to summarize the literature relevant to understanding mathematical thinking and problem solving, and (3) to point to new directions in research, development, and assessment consonant with an emerging understanding of mathematical thinking and the goals for instruction outlined here. The use of the phrase “learning to think mathematically” in this chapter’s title is deliberately broad. Although the original charter for this chapter was to review the literature on problem solving and metacognition, the literature itself is somewhat ill defined and poorly grounded. As the literature summary will make clear, problem solving has been used with multiple meanings that range from “working rote exercises” to “doing mathematics as a professional”; metacognition has multiple and almost disjoint meanings (from knowledge about one’s thought processes to self-regulation during problem solving) that make it difficult to use as a concept. This chapter outlines the various meanings that have been ascribed to these terms and discusses their role in mathematical thinking. The discussion will not have the character of a classic literature review, which is typically encyclopedic in its references and telegraphic in its discussions of individual papers or results. It will, instead, be selective and illustrative, with main points illustrated by extended discussions of pertinent examples. Problem solving has, as predicted in the 1980 Yearbook of the National Council of Teachers of Mathematics (Krulik, 1980, p. xiv), been the theme of the 1980s. The decade began with NCTM’s widely heralded statement, in its Agenda for Action, that “problem solving must be the focus of school mathematics” (NCTM, 1980, p. 1). It concluded with the publication of Everybody Counts (National Research Council, 1989) and the Curriculum and Evaluation Standards for School Mathematics (NCTM, 1989), both of which emphasize problem solving. One might infer, then, that there is general acceptance of the idea that the primary goal of mathematics instruction should be to have students become competent problem solvers. Yet, given the multiple interpretations of the term, the goal is hardly clear. Equally unclear is the role that problem solving, once adequately characterized, should play in the larger context of school mathematics. What are the goals for mathematics instruction, and how does problem solving fit within those goals? Such questions are complex. Goals for mathematics instruction depend on one’s conceptualization of what mathematics is, and what it means to understand mathematics. Such conceptualizations vary widely. At one end of the spectrum, mathematical knowledge is seen as a body of facts and procedures dealing with quantities, magnitudes, and forms, and the relationships among them; knowing mathematics is seen as having mastered these facts and procedures. At the other end of the spectrum, mathematics is conceptualized as the “science of patterns,” an (almost) empirical discipline closely akin to the sciences in its emphasis on pattern-seeking on the basis of empirical evidence. The author’s view is that the former perspective trivializes mathematics; that a curriculum based on mastering a corpus of mathematical facts and procedures is severely impoverished—in much the same way that an English curriculum would be considered impoverished if it focused largely, if not exclusively, on issues of grammar. The author characterizes the mathematical enterprise as follows:

Learning to Think Mathematically: Problem Solving, Metacognition, and Sense Making in Mathematics (Reprint):

A novel method is proposed for performing multilabel, interactive image segmentation. Given a small number of pixels with user-defined (or predefined) labels, one can analytically and quickly determine the probability that a random walker starting at each unlabeled pixel will first reach one of the prelabeled pixels. By assigning each pixel to the label for which the greatest probability is calculated, a high-quality image segmentation may be obtained. Theoretical properties of this algorithm are developed along with the corresponding connections to discrete potential theory and electrical circuits. This algorithm is formulated in discrete space (i.e., on a graph) using combinatorial analogues of standard operators and principles from continuous potential theory, allowing it to be applied in arbitrary dimension on arbitrary graphs

/pdf/random-walks-for-image-segmentation-4ppfcpqq4u.pdf

Random Walks for Image Segmentation

Winner of the 1983 National Book Award! \"...a perfectly marvelous book about the Queen of Sciences, from which one will get a real feeling for what mathematicians do and who they are. The exposition is clear and full of wit and humor...\" - The New Yorker (1983 National Book Award edition) Mathematics has been a human activity for thousands of years. Yet only a few people from the vast population of users are professional mathematicians, who create, teach, foster, and apply it in a variety of situations. The authors of this book believe that it should be possible for these professional mathematicians to explain to non-professionals what they do, what they say they are doing, and why the world should support them at it. They also believe that mathematics should be taught to non-mathematics majors in such a way as to instill an appreciation of the power and beauty of mathematics. Many people from around the world have told the authors that they have done precisely that with the first edition and they have encouraged publication of this revised edition complete with exercises for helping students to demonstrate their understanding. This edition of the book should find a new generation of general readers and students who would like to know what mathematics is all about. It will prove invaluable as a course text for a general mathematics appreciation course, one in which the student can combine an appreciation for the esthetics with some satisfying and revealing applications. The text is ideal for 1) a GE course for Liberal Arts students 2) a Capstone course for perspective teachers 3) a writing course for mathematics teachers. A wealth of customizable online course materials for the book can be obtained from Elena Anne Marchisotto (elena.marchisotto@csun.edu) upon request.

The Mathematical Experience.

Winner of the 1983 National Book Award! "...a perfectly marvelous book about the Queen of Sciences, from which one will get a real feeling for what mathematicians do and who they are. The exposition is clear and full of wit and humor..." - The New Yorker (1983 National Book Award edition) Mathematics has been a human activity for thousands of years. Yet only a few people from the vast population of users are professional mathematicians, who create, teach, foster, and apply it in a variety of situations. The authors of this book believe that it should be possible for these professional mathematicians to explain to non-professionals what they do, what they say they are doing, and why the world should support them at it. They also believe that mathematics should be taught to non-mathematics majors in such a way as to instill an appreciation of the power and beauty of mathematics. Many people from around the world have told the authors that they have done precisely that with the first edition and they have encouraged publication of this revised edition complete with exercises for helping students to demonstrate their understanding. This edition of the book should find a new generation of general readers and students who would like to know what mathematics is all about. It will prove invaluable as a course text for a general mathematics appreciation course, one in which the student can combine an appreciation for the esthetics with some satisfying and revealing applications. The text is ideal for 1) a GE course for Liberal Arts students 2) a Capstone course for perspective teachers 3) a writing course for mathematics teachers. A wealth of customizable online course materials for the book can be obtained from Elena Anne Marchisotto (elena.marchisotto@csun.edu) upon request.

The Mathematical Experience

Most philosophers of mathematics treat it as isolated, timeless, ahistorical, inhuman. Reuben Hersh argues the contrary, that mathematics must be understood as a human activity, a social phenomenon, part of human culture, historically evolved, and intelligible only in a social context. Hersh pulls the screen back to reveal mathematics as seen by professionals, debunking many mathematical myths, and demonstrating how the "humanist" idea of the nature of mathematics more closely resembles how mathematicians actually work. At the heart of his book is a fascinating historical account of the mainstream of philosophy--ranging from Pythagoras, Descartes, and Spinoza, to Bertrand Russell, David Hilbert, and Rudolph Carnap--followed by the mavericks who saw mathematics as a human artifact, including Aristotle, Locke, Hume, Mill, and Lakatos.What is Mathematics, Really? reflects an insider's view of mathematical life, and will be hotly debated by anyone with an interest in mathematics or the philosophy of science.

What Is Mathematics, Really?

What is Mathematics, Really?

In mathematical research, the purpose of proof is to convince. The test of whether something is a proof is whether it convinces qualified judges. In the classroom, on the other hand, the purpose of proof is to explain. Enlightened use of proofs in the mathematics classroom aims to stimulate the students' understanding, not to meet abstract standards of “rigor” or “honesty.”

Reuben Hersh

Papers

The Mathematical Experience.

The Mathematical Experience

What Is Mathematics, Really?

What is Mathematics, Really?

Proving Is Convincing and Explaining.