Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

Computational properties of use of biological organisms or to the construction of computers can emerge as collective properties of systems having a large number of simple equivalent components (or neurons). The physical meaning of content-addressable memory is described by an appropriate phase space flow of the state of a system. A model of such a system is given, based on aspects of neurobiology but readily adapted to integrated circuits. The collective properties of this model produce a content-addressable memory which correctly yields an entire memory from any subpart of sufficient size. The algorithm for the time evolution of the state of the system is based on asynchronous parallel processing. Additional emergent collective properties include some capacity for generalization, familiarity recognition, categorization, error correction, and time sequence retention. The collective properties are only weakly sensitive to details of the modeling or the failure of individual devices.

https://www.pnas.org/content/pnas/79/8/2554.full.pdf

Neural networks and physical systems with emergent collective computational abilities

Many areas of science depend on exploratory data analysis and visualization. The need to analyze large amounts of multivariate data raises the fundamental problem of dimensionality reduction: how to discover compact representations of high-dimensional data. Here, we introduce locally linear embedding (LLE), an unsupervised learning algorithm that computes low-dimensional, neighborhood-preserving embeddings of high-dimensional inputs. Unlike clustering methods for local dimensionality reduction, LLE maps its inputs into a single global coordinate system of lower dimensionality, and its optimizations do not involve local minima. By exploiting the local symmetries of linear reconstructions, LLE is able to learn the global structure of nonlinear manifolds, such as those generated by images of faces or documents of text.

/pdf/nonlinear-dimensionality-reduction-by-locally-linear-vmiazg4oee.pdf

Nonlinear dimensionality reduction by locally linear embedding.

https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

Clustering is the unsupervised classification of patterns (observations, data items, or feature vectors) into groups (clusters). The clustering problem has been addressed in many contexts and by researchers in many disciplines; this reflects its broad appeal and usefulness as one of the steps in exploratory data analysis. However, clustering is a difficult problem combinatorially, and differences in assumptions and contexts in different communities has made the transfer of useful generic concepts and methodologies slow to occur. This paper presents an overview of pattern clustering methods from a statistical pattern recognition perspective, with a goal of providing useful advice and references to fundamental concepts accessible to the broad community of clustering practitioners. We present a taxonomy of clustering techniques, and identify cross-cutting themes and recent advances. We also describe some important applications of clustering algorithms such as image segmentation, object recognition, and information retrieval.

/pdf/data-clustering-a-review-4c452f5cab.pdf

Data clustering: a review

This chapter contains a review of certain programmed searching methods which might properly be called associative or content-addressable, because an entry (record, document, file, associated word or data, etc.) can be accessed directly on the basis of a keyword or a specified part of the entry. It is to be noted that all of these methods are connected with the use of generalpurpose computers, and the entries are thereby stored in standard randomaccess (addressable) memories. The methods described in Sects. 2.2–7 are generally known under the name hash coding. Al ternative names used for the same techniques are hashing, hash addressing, hash-table method, scatter-storage techniques, key trans formation, key-to-addresstrans formation, computed-entry table method, randomization, random addressing, scrambling, direct-access method, and possibly others. Another content-addressable organization of data named TRIE is described in Sect. 2.8.

Content Addressing by Software

Aivot ja tietokoneet

If the dimensionality of an application problem is not very high (say, a few dozen inputs and outputs at most) and computing need not be performed in real time, most ANN algorithms can be implemented by pure software, especially if contemporary workstation computers are available. However, especially in real-time pattern recognition or robotics applications one might need special co-processor boards or even “neurocomputers.” For really large problems, e.g. in the preprocessing stages for more complicated computer vision, special hardware networks may have to be developed. Such ANN circuits are sometimes necessary to miniaturize devices and make them cheap, for instance in medical applications or consumer electronics.

Hardware for SOM

The basic neural-network literature may already be familiar to a wide circle of scientists and engineers. Nonetheless, the true meaning of several concepts, never explicitly defined, may have escaped the attention of many readers. This chapter is therefore devoted to all those who always wanted to know about these definitions but were afraid to ask. This glossary is by no means exhaustive: it only serves the reading of this book, the literature references given, and other collateral texts. The definitions have been meant to be more explanatory than distinctive. Among alternative explanations, only those that are relevant to neurophysiology or ANNs have been given. The partial vocabularies picked up from artificial-intelligence techniques, biophysics, bionics, image analysis, linguistics, optics, pattern-recognition techniques, phonetics, psychology, and medical science are extremely concise. Mathematical concepts have also been explained in more detail in Chap. 1.

Glossary of “Neural” Terms

http://ieeexplore.ieee.org/iel6/8370/26352/01171876.pdf

Teuvo Kohonen

Papers

Content Addressing by Software

Aivot ja tietokoneet

Hardware for SOM

Glossary of “Neural” Terms

The block diagram of the recognition system is