Random forests are a combination of tree predictors such that each tree depends on the values of a random vector sampled independently and with the same distribution for all trees in the forest. The generalization error for forests converges a.s. to a limit as the number of trees in the forest becomes large. The generalization error of a forest of tree classifiers depends on the strength of the individual trees in the forest and the correlation between them. Using a random selection of features to split each node yields error rates that compare favorably to Adaboost (Y. Freund & R. Schapire, Machine Learning: Proceedings of the Thirteenth International conference, aaa, 148–156), but are more robust with respect to noise. Internal estimates monitor error, strength, and correlation and these are used to show the response to increasing the number of features used in the splitting. Internal estimates are also used to measure variable importance. These ideas are also applicable to regression.

Random Forests

Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.

Gradient-based learning applied to document recognition

We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

/pdf/going-deeper-with-convolutions-1yobw2o2ds.pdf

Going deeper with convolutions

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.

Deep Learning

Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

Exogenous state variables and rewards can slow down reinforcement learning by injecting uncontrolled variation into the reward signal. We formalize exogenous state variables and rewards and identify conditions under which an MDP with exogenous state can be decomposed into an exogenous Markov Reward Process involving only the exogenous state+reward and an endogenous Markov Decision Process defined with respect to only the endogenous rewards. We also derive a variance-covariance condition under which Monte Carlo policy evaluation on the endogenous MDP is accelerated compared to using the full MDP. Similar speedups are likely to carry over to all RL algorithms. We develop two algorithms for discovering the exogenous variables and test them on several MDPs. Results show that the algorithms are practical and can significantly speed up reinforcement learning.

Discovering and Removing Exogenous State Variables and Rewards for Reinforcement Learning

From April 14 -- 20, 2007, the Dagstuhl Seminar 07161 ``Probabilistic, Logical and Relational Learning - A Further Synthesis'' was held
in the International Conference and Research Center (IBFI),
Schloss Dagstuhl.
During the seminar, several participants presented their current
research, and ongoing work and open problems were discussed. Abstracts of
the presentations given during the seminar as well as abstracts of
seminar results and ideas are put together in this paper. The first section
describes the seminar topics and goals in general.
Links to extended abstracts or full papers are provided, if available.

07161 Abstracts Collection -- Probabilistic, Logical and Relational Learning - A Further Synthesis.

As the field of machine learning has grown and matured, the number of active, senior researchers in the field has grown as well. Our journal faced a problem: If we added all of these excellent scientists to the editorial board, it would soon become large and unwieldy. Hence, at the 1993 meeting of the editorial board, it was decided to develop a system of rotating 3-year terms, so that over time a wide range of researchers in machine learning would have the opportunity to serve on the editorial board and contribute to the quality and success of the journal. To implement this rotating-term system, currently-serving editorial board members were randomly assigned to one of three groups with terms ending December 31, 1993, December 31, 1994, and December 31, 1995. I want to take this opportunity to recognize and thank those board members whose terms expired December 31, 1993:

/pdf/editorial-new-editorial-board-members-1yayos1re1.pdf

Editorial: New Editorial Board Members

Machine learning methods provide a way for artificial intelligence systems to learn from experience. This article describes four threads of machine learning research supported and guided by the Defense Advanced Research Projects Agency — probabilistic modeling for speech recognition, probabilistic relational models, the integration of multiple machine learning approaches into a task-specific system, and neural network technology. These threads illustrate the Defense Advanced Research Projects Agency way of creating timely advances in a field.

DARPA’s Role in Machine Learning

Biologists collect and analyze phenomic (e.g., anatomical or non-genomic) data to discover relationships among species in the Tree of Life. The domain is seeking to modernize this very time-consuming and largely manual process. We have developed an approach to detect and localize object parts in standardized images of bat skulls. This approach has been further developed for unannotated images by leveraging knowledge learned from a few annotated images. The key challenge is that the unlabeled images show bat skulls of "unknown" species that may have types, total numbers, and layouts of the teeth that differ from the "known" species appearing in the labeled images. Our method begins by matching the unlabeled images to the labeled ones. This allows a transfer of tooth annotations to the unlabeled images. We then learn a tree parts model on the transferred annotations, and apply this model to detect and label teeth in the unlabeled images. Our evaluation demonstrates good performance, which is close to our upper bound performance by the fully supervised model.

/pdf/zero-shot-learning-and-detection-of-teeth-in-images-of-bat-j48wrykxht.pdf

Thomas G. Dietterich

Papers

Discovering and Removing Exogenous State Variables and Rewards for Reinforcement Learning

07161 Abstracts Collection -- Probabilistic, Logical and Relational Learning - A Further Synthesis.

Editorial: New Editorial Board Members

DARPA’s Role in Machine Learning

Zero-Shot Learning and Detection of Teeth in Images of Bat Skulls