The performance of stochastic gradient descent (SGD) depends critically on how learning rates are tuned and decreased over time. We propose a method to automatically adjust multiple learning rates so as to minimize the expected error at any one time. The method relies on local gradient variations across samples. In our approach, learning rates can increase as well as decrease, making it suitable for non-stationary problems. Using a number of convex and non-convex learning tasks, we show that the resulting algorithm matches the performance of SGD or other adaptive approaches with their best settings obtained through systematic search, and effectively removes the need for learning rate tuning.

/pdf/no-more-pesky-learning-rates-38no9ntzuf.pdf

No more pesky learning rates

We describe the Fourier basis, a linear value function approximation scheme based on the Fourier series. We empirically demonstrate that it performs well compared to radial basis functions and the polynomial basis, the two most popular fixed bases for linear value function approximation, and is competitive with learned proto-value functions.

/pdf/value-function-approximation-in-reinforcement-learning-using-1mpte0vpid.pdf

Value function approximation in reinforcement learning using the fourier basis

A survey on the application of recurrent neural networks to statistical language modeling

A new connectionist model (named RASHNL) accounts for many "irrational" phenomena found in nonmetric multiple-cue probability learning, wherein people learn to utilize a number of discrete-valued cues that are partially valid indicators of categorical outcomes. Phenomena accounted for include cue competition, effects of cue salience, utilization of configural information, decreased learning when information is introduced after a delay, and effects of base rates. Experiments 1 and 2 replicate previous experiments on cue competition and cue salience, and fits of the model provide parameter values for making qualitatively correct predictions for many other situations. The model also makes 2 new predictions, confirmed in Experiments 3 and 4. The model formalizes 3 explanatory principles: rapidly shifting attention with learned shifts, decreasing learning rates, and graded similarity in exemplar representation.

A model of probabilistic category learning.

Gain adaptation algorithms for neural networks typically adjust learning rates by monitoring the correlation between successive gradients. Here we discuss the limitations of this approach, and develop an alternative by extending Sutton''s work on linear systems to the general, nonlinear case. The resulting online algorithms are computationally little more expensive than other acceleration techniques, do not assume statistical independence between successive training patterns, and do not require an arbitrary smoothing parameter. In our benchmark experiments, they consistently outperform other acceleration methods, and show remarkable robustness when faced with non-i.i.d. sampling of the input space.

/pdf/local-gain-adaptation-in-stochastic-gradient-descent-3en7hdu2p9.pdf

Local Gain Adaptation in Stochastic Gradient Descent

Optimization is an important operation in many domains of science and technology. Local optimization techniques typically employ some form of iterative procedure, based on derivatives of the function to be optimized (objective function). These techniques normally involve parameters that must be set by the user, often by trial and error. Those parameters can have a strong influence on the convergence speed of the optimization. In several cases, a significant speed advantage could be gained if one could vary these parameters during the optimization, to reflect the local characteristics of the function being optimized. Some parameter adaptation methods have been proposed for this purpose, for deterministic optimization situations. For stochastic (also called on-line) optimization situations, there appears to be no simple and effective parameter adaptation method. This paper proposes a new method for parameter adaptation in stochastic optimization. The method is applicable to a wide range of objective functions, as well as to a large set of local optimization techniques. We present the derivation of the method, details of its application to gradient descent and to some of its variants, and examples of its use in the gradient optimization of several functions, as well as in the training of a multilayer perceptron by on-line backpropagation. Introduction Optimization is an operation that is often used in several different domains of science and technology. It normally consists of maximizing or minimizing a given function (called objective function ), that is chosen to represent the quality of a given system. The system may be physical, (mechanical, chemical, etc.), a mathematical model, a computer program, etc., or even a mixture of several of these.

Alexander Plakhov

Papers

Parameter adaptation in stochastic optimization