We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. The method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and/or parameters. The method is also appropriate for non-stationary objectives and problems with very noisy and/or sparse gradients. The hyper-parameters have intuitive interpretations and typically require little tuning. Some connections to related algorithms, on which Adam was inspired, are discussed. We also analyze the theoretical convergence properties of the algorithm and provide a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework. Empirical results demonstrate that Adam works well in practice and compares favorably to other stochastic optimization methods. Finally, we discuss AdaMax, a variant of Adam based on the infinity norm.

Adam: A Method for Stochastic Optimization

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.

Deep Learning

Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability. The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

/pdf/reinforcement-learning-an-introduction-rzxgej9p17.pdf

Reinforcement Learning: An Introduction

Many problems of recent interest in statistics and machine learning can be posed in the framework of convex optimization. Due to the explosion in size and complexity of modern datasets, it is increasingly important to be able to solve problems with a very large number of features or training examples. As a result, both the decentralized collection or storage of these datasets as well as accompanying distributed solution methods are either necessary or at least highly desirable. In this review, we argue that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas. The method was developed in the 1970s, with roots in the 1950s, and is equivalent or closely related to many other algorithms, such as dual decomposition, the method of multipliers, Douglas–Rachford splitting, Spingarn's method of partial inverses, Dykstra's alternating projections, Bregman iterative algorithms for l1 problems, proximal methods, and others. After briefly surveying the theory and history of the algorithm, we discuss applications to a wide variety of statistical and machine learning problems of recent interest, including the lasso, sparse logistic regression, basis pursuit, covariance selection, support vector machines, and many others. We also discuss general distributed optimization, extensions to the nonconvex setting, and efficient implementation, including some details on distributed MPI and Hadoop MapReduce implementations.

/pdf/distributed-optimization-and-statistical-learning-via-the-8s9gtddw0d.pdf

Distributed Optimization and Statistical Learning Via the Alternating Direction Method of Multipliers

We consider two approaches to robust estimation for the Box–Cox power-transformation model. One approach maximizes weighted, modified likelihoods. A second approach bounds a measure of gross-error sensitivity. Among our primary concerns is the performance of these estimators on actual data. In examples that we study, there seem to be only minor differences between these two robust estimators, but they behave rather differently than the maximum likelihood estimator or estimators that bound only the influence of the residuals. These examples show that model selection, determination of the transformation parameter, and outlier identification are fundamentally interconnected.

Transformations in Regression: A Robust Analysis

The method of maximum tapered likelihood has been proposed as a way to quickly estimate covariance parameters for stationary Gaussian random fields. We show that under a useful asymptotic regime, maximum tapered likelihood estimators are consistent and asymptotically normal for covariance models in common use. We then formalize the notion of tapered quasi-Bayesian estimators and show that they too are consistent and asymptotically normal. We also present asymptotic confidence intervals for both types of estimators and show via simulation that they accurately reflect sampling variability, even at modest sample sizes. Proofs, an example, and detailed derivations are provided in the supplementary materials, available online.

https://stat.duke.edu/~bs128/papers/taper-bayes_02-20-2011.pdf

Tapered Covariance: Bayesian Estimation and Asymptotics

This article describes a fully automated bandwidth selection method for additive models that is applicable to the widely used backfitting algorithm of Buja, Hastie, and Tibshirani. The proposed plug-in estimator is an extension of the univariate local linear regression estimator of Ruppert, Sheather, and Wand and is shown to achieve the same Op (n –2/7) relative convergence rate for bivariate additive models. If more than two covariates are present, theoretical justification of the method requires independence of the covariates, but simulation experiments show that in practice the method is very robust to violations of this assumption. The proposed bandwidth selection method is compared to cross-validation through simulation experiments. Its practical behavior is demonstrated on a real dataset.

A Fully Automated Bandwidth Selection Method for Fitting Additive Models

In weighted least squares, it is typical that the weights are unknown and must be estimated. Most packages provide standard errors assuming that the weights are known. This is fine for sufficiently large sample sizes, but what about for small-to-moderate sample sizes? The investigation of this article into the effect of estimating weights proceeds under the assumption typical in practice—that one has a parametric model for the variance function. In this context, generalized least squares consists of (a) an initial estimate of the regression parameter, (b) a method for estimating the variance function, and (c) the number of iterations in reweighted least squares. By means of expansions for the covariance, it is shown that each of (a)—(c) can matter in problems of small to moderate size. A few conclusions may be of practical interest. First, estimated standard errors assuming that the weights are known can be too small in practice. The investigation indicates that a simple bootstrap operation resul...

The Effect of Estimating Weights in Weighted Least Squares

/pdf/a-fully-automated-bandwidth-selection-method-for-fitting-1a5q4e67cl.pdf

David Ruppert

Papers

Transformations in Regression: A Robust Analysis

Tapered Covariance: Bayesian Estimation and Asymptotics

A Fully Automated Bandwidth Selection Method for Fitting Additive Models

The Effect of Estimating Weights in Weighted Least Squares

A Fully Automated Bandwidth Selection Method for Fitting Additive Models