Pattern Recognition and Adaptive Control
TL;DR: This work has shown that when the state of a control system is represented as a pattern, learning to make the control decisions actually becomes the same as learning to classify the patterns.
Abstract: Adaptive or self-optimizing systems utilize feedback principles to achieve automatic performance optimization. These principles have been applied to both control systems and adaptive logic structures. The Adaline (adaptive linear threshold element) is essentially the same as an adaptive sampled-data system with quantized input and output signals. A digital controller made of adaptive neurons comprises a pattern-recognizing control system. When the state of a control system is represented as a pattern, learning to make the control decisions actually becomes the same as learning to classify the patterns.
Citations
More filters
Proceedings Article•
19 Jun 2016TL;DR: In this paper, the authors present a benchmark suite of continuous control tasks, including classic tasks like cart-pole swing-up, tasks with high state and action dimensionality such as 3D humanoid locomotion, and tasks with partial observations.
Abstract: Recently, researchers have made significant progress combining the advances in deep learning for learning feature representations with reinforcement learning. Some notable examples include training agents to play Atari games based on raw pixel data and to acquire advanced manipulation skills using raw sensory inputs. However, it has been difficult to quantify progress in the domain of continuous control due to the lack of a commonly adopted benchmark. In this work, we present a benchmark suite of continuous control tasks, including classic tasks like cart-pole swing-up, tasks with very high state and action dimensionality such as 3D humanoid locomotion, tasks with partial observations, and tasks with hierarchical structure. We report novel findings based on the systematic evaluation of a range of implemented reinforcement learning algorithms. Both the benchmark and reference implementations are released at https://github.com/rllab/rllab in order to facilitate experimental reproducibility and to encourage adoption by other researchers.
1,038 citations
Posted Content•
TL;DR: In this article, the authors present a benchmark suite of continuous control tasks, including classic tasks like cart-pole swing-up, tasks with high state and action dimensionality such as 3D humanoid locomotion, and tasks with partial observations.
Abstract: Recently, researchers have made significant progress combining the advances in deep learning for learning feature representations with reinforcement learning. Some notable examples include training agents to play Atari games based on raw pixel data and to acquire advanced manipulation skills using raw sensory inputs. However, it has been difficult to quantify progress in the domain of continuous control due to the lack of a commonly adopted benchmark. In this work, we present a benchmark suite of continuous control tasks, including classic tasks like cart-pole swing-up, tasks with very high state and action dimensionality such as 3D humanoid locomotion, tasks with partial observations, and tasks with hierarchical structure. We report novel findings based on the systematic evaluation of a range of implemented reinforcement learning algorithms. Both the benchmark and reference implementations are released at this https URL in order to facilitate experimental reproducibility and to encourage adoption by other researchers.
521 citations
01 Feb 1987
TL;DR: Early ideas which primarily attempt to compensate for gain variations and more general methods like gain scheduling, model reference adaptive control, and self-tuning regulators are reviewed.
Abstract: Adaptive control is now finding its way into the marketplace after many years of effort. This paper reviews some ideas used to design adaptive control systems. It covers early ideas which primarily attempt to compensate for gain variations and more general methods like gain scheduling, model reference adaptive control, and self-tuning regulators. It is shown that adaptive control laws can be obtained using stochastic control theory. Techniques for analyzing adaptive systems are discussed. This covers stability and convergence analysis. Issues of importance for applications like parameterization, tuning, and tracking, as well as different ways of using adaptive control are also discussed. An overview of applications which includes feasibility studies as well as products based on adaptive techniques concludes the paper.
233 citations
01 Jan 1988
TL;DR: Adaptive control is now finding its way into the marketplace after many years of effort as discussed by the authors, and it is shown that adaptive control laws can be obtained using stochastic control theory.
Abstract: Adaptive control is now finding its way into the marketplace after many years of effort. This paper reviews some ideas used to design adaptive control systems. It covers early ideas which primarily attempt to compensate for gain variations and more general methods like gain scheduling, model reference adaptive control, and self-tuning regulators. It is shown that adaptive control laws can be obtained using stochastic control theory. Techniques for analyzing adaptive systems are discussed. This covers stability and convergence analysis. Issues of importance for applications like parameterization, tuning, and tracking, as well as different ways of using adaptive control are also discussed. An overview of applications which includes feasibility studies as well as products based on adaptive techniques concludes the paper.
230 citations
01 Aug 1979
TL;DR: A case study on a hierarchically intelligent controlled prosthesis, summarized herein, establishes the feasibility of the suggested methodologies and the importance of a growing area in control engineering.
Abstract: This is a paper of expository nature reflecting the author's past experiences, his current research efforts, and his aspirations about the future of automatic-control systems. It is not intended to give a quantitative analysis of modern control methodologies, which may be found in the bibliography at the end of the text, but rather emphasize the importance of a growing area in control engineering. Reviewing the classical, optimal, and stochastic control systems, the reader is led into the uncertainties and controversies of adaptive and learning controls. While self-organizing control was proposed for a systematic unification of these most advanced control methodologies, intelligent control--a discipline capable of high-level decision making and task execution--is predicted as the next level of sophistication in the hierarchy of control systems. A case study on a hierarchically intelligent controlled prosthesis, summarized herein, establishes the feasibility of the suggested methodologies. Future applications to other larse scale systems of general or specific scientific interest may prove the importance of such a discipline.
192 citations
References
More filters
01 Jul 1961
TL;DR: In this paper, a machine consisting of a universal non-linear filter, which is a highly adaptable analogue computer, together with a training device is described, where the training device optimizes the output by successive adjustment of the variable coefficients, until it has approached a target function as closely as can be achieved with a polynomial of 94 terms.
Abstract: A machine is described consisting of a universal non-linear filter, which is a highly adaptable analogue computer, together with a training device. The analogue machine has 18 input quantities from which it can compute in about 2 millisec 94 terms of a polynomial, each term containing products and powers of the input quantities, with adjustable coefficients, and can form their sum. The input quantities may be, for instance, 18 past samples of the values of a stochastic variable which is fed into the machine, and the result of the computation is an output function which contains 94 free variables. The training device optimizes the output by successive adjustment of the variable coefficients, until it has approached a target function as closely as can be achieved with a polynomial of 94 terms, by the criterion of the least mean-square error. This is done by repeatedly feeding into the machine a record of the stochastic process, long enough to be representative, and adjusting the variable coefficients, one at a time after each run, by a strategy which ensures that the error will monotonically decrease from run to run.In order to make the machine an optimum filter it is trained on a record of a noisy process, together with a target record which contains the signal only. It is taught as a predictor by taking as the target function a value of the stochastic process advanced by a certain time interval beyond the last value which goes into the input. It is trained as a simulator, for instance of an unknown mechanism, by feeding it with the input of the mechanism to be simulated at one end and presenting it at the other with its output as target function. The machine will then make itself into a model of the device to be simulated and the non-linear transfer function of the device can be read off from the final setting of the coefficients, as nearly as it can be represented by a 94-term polynomial. The machine is not confined to single-input systems.The machine incorporates 80 analogue multipliers of a novel ‘piezomagnetic’ type which, in its present form, can perform over 1000 multiplications per second with an error of 0.5% or less.A few examples of the first test applications of the machine are described.
136 citations
01 Dec 1959
TL;DR: Any stimulus to a system such as described in this paper can be coded into a binary representation by dividing the paper into many small squares and assigning a 1 to a square if the character covers that square, and a 0 to the square if it does not cover it.
Abstract: Any stimulus to a system such as described in this paper can be coded into a binary representation. A character on a piece of paper can be represented in binary form by dividing the paper into many small squares and assigning a 1 to a square if the character covers that square, and a 0 to the square if the character does not cover it.
20 citations