Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.

Deep Learning

Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

From the Publisher:
This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition. After introducing the basic concepts, the book examines techniques for modelling probability density functions and the properties and merits of the multi-layer perceptron and radial basis function network models. Also covered are various forms of error functions, principal algorithms for error function minimalization, learning and generalization in neural networks, and Bayesian techniques and their applications. Designed as a text, with over 100 exercises, this fully up-to-date work will benefit anyone involved in the fields of neural computation and pattern recognition.

/pdf/neural-networks-for-pattern-recognition-34puigiau1.pdf

Neural networks for pattern recognition

This paper proposes a paradigm for managing the dynamic aspects of organizational knowledge creating processes. Its central theme is that organizational knowledge is created through a continuous dialogue between tacit and explicit knowledge. The nature of this dialogue is examined and four patterns of interaction involving tacit and explicit knowledge are identified. It is argued that while new knowledge is developed by individuals, organizations play a critical role in articulating and amplifying that knowledge. A theoretical framework is developed which provides an analytical perspective on the constituent dimensions of knowledge creation. This framework is then applied in two operational models for facilitating the dynamic creation of appropriate organizational knowledge.

/pdf/a-dynamic-theory-of-organizational-knowledge-creation-22t9dxu55o.pdf

A Dynamic Theory of Organizational Knowledge Creation

https://www2.econ.iastate.edu/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf

Deep learning in neural networks

We present here some aspects of our simulation of the function of the reticular core of the central nervous system that we believe may be useful in the creation of the fully automated diagnostic clinic. We shall not be concerned with the anatomy or the physiology of the actual reticular core because we know that it has served all vertebrates with little evolution of its intrinsic structure. We conceive that the reticular core commits the whole organism to one of a small number of incompatible modes of behavior. It achieves this with a number of relatively undifferentiated neurons, about three-quarters of a million in a frog and perhaps two million in a man, a number which is estimated to be less than the number of afferent peripheral neurons, or bipolars, and greater than the number of efferent peripheral neurons, or motoneurons. Jointly, these sample all signals from the bipolars and all signals ascending or descending in the neuraxis, including those from all other parts of the nervous system, each neuron receiving, perhaps, a thousand of these selected almost at random. The axon of each divides into an ascending and a descending branch, often having in addition

Information processing in the central nervous system with special reference to the reticular system

Conclusions1. On the basis of these results, it may be concluded that epinephrine stimulates neuronal aggregates in the cerebral cortex. 2. The local effect is marked by an increase in duration and intensity of the period of over-excitation subsequent to a stimulus eliciting movement, and it is characterized by the appearance of after-discharges of cortex and muscle after stimuli just sufficient to produce movement.

Local effects of epinephrine on the cerebral cortex.

Excitability of cerebral cortex in infant macaca mulatta

Information in the head

Reticular formation of central nervous system in vertebrates described as behavior controlling circuit of interconnected modules, proposing hybrid computer method for operational scheme

Warren S. McCulloch

Papers

Information processing in the central nervous system with special reference to the reticular system

Local effects of epinephrine on the cerebral cortex.

Excitability of cerebral cortex in infant macaca mulatta

Information in the head

An embodiment of some vertebrate command and control principles