Book•
Introduction to the mathematical theory of control processes
01 Jan 1967-
About: The article was published on 1967-01-01 and is currently open access. It has received 475 citations till now. The article focuses on the topics: Mathematical theory.
Citations
More filters
TL;DR: This article attempts to strengthen the links between the two research communities by providing a survey of work in reinforcement learning for behavior generation in robots by highlighting both key challenges in robot reinforcement learning as well as notable successes.
Abstract: Reinforcement learning offers to robotics a framework and set of tools for the design of sophisticated and hard-to-engineer behaviors. Conversely, the challenges of robotic problems provide both inspiration, impact, and validation for developments in reinforcement learning. The relationship between disciplines has sufficient promise to be likened to that between physics and mathematics. In this article, we attempt to strengthen the links between the two research communities by providing a survey of work in reinforcement learning for behavior generation in robots. We highlight both key challenges in robot reinforcement learning as well as notable successes. We discuss how contributions tamed the complexity of the domain and study the role of algorithms, representations, and prior knowledge in achieving these successes. As a result, a particular focus of our paper lies on the choice between model-based and model-free as well as between value-function-based and policy-search methods. By analyzing a simple problem in some detail we demonstrate how reinforcement learning approaches may be profitably applied, and we note throughout open questions and the tremendous potential for future research.
2,391 citations
Cites background from "Introduction to the mathematical th..."
...This optimization problem can be tackled in two substantially different ways (Bellman, 1967, 1971)....
[...]
01 Jan 2012
TL;DR: A survey of work in reinforcement learning for behavior generation in robots can be found in this article, where the authors highlight key challenges in robot reinforcement learning as well as notable successes and discuss the role of algorithms, representations and prior knowledge in achieving these successes.
Abstract: Reinforcement learning offers to robotics a framework and set of tools for the design of sophisticated and hard-to-engineer behaviors. Conversely, the challenges of robotic problems provide both inspiration, impact, and validation for developments in reinforcement learning. The relationship between disciplines has sufficient promise to be likened to that between physics and mathematics. In this article, we attempt to strengthen the links between the two research communities by providing a survey of work in reinforcement learning for behavior generation in robots. We highlight both key challenges in robot reinforcement learning as well as notable successes. We discuss how contributions tamed the complexity of the domain and study the role of algorithms, representations, and prior knowledge in achieving these successes. As a result, a particular focus of our paper lies on the choice between model-based and model-free as well as between value-function-based and policy-search methods. By analyzing a simple problem in some detail we demonstrate how reinforcement learning approaches may be profitably applied, and we note throughout open questions and the tremendous potential for future research.
1,513 citations
Posted Content•
1,238 citations
TL;DR: It is suggested that competition solves a sensitivity problem that confronts all cellular systems: the noise-saturation dilemma.
Abstract: This article is the first of a series to globally analyse competitive dynamical systems. The article suggests that competition solves a sensitivity problem that confronts all cellular systems: the noise-saturation dilemma. Low energy input patterns can be registered poorly by cells due to their internal noise. High energy input patterns can be registered poorly by cells because their sensitivity approaches zero when all their sites are turned on. How do cells balance between the two equally deadly, but complementary, extremes of noise and saturation? How do cells achieve a Golden Mean?
648 citations
01 Jan 1987
TL;DR: This chapter describes the efforts of cosmologists (and nature?) to form galactic and stellar structures, comments on them from the viewpoint of the homeokinetic principles, and pursues some parallels in the origin of life.
Abstract: 1 This chapter shows application of a consistent set of clear physical principles to describe the beginnings of the universe by the (hot) Big Bang model. The ultimate beginning presents difficulties for the physicist, particularly when it comes to explaining why we find a very inhomogeneous cosmos, instead of a uniform, radiating gas. Stars, globular clusters, galaxies, and clusters of galaxies dot space, but we cannot at present explain their presence without invoking a "deus ex machina." We expect ultimately to rationalize the existence of these local inhomogeneities through gravitational contractions of regions of higher than average mass density that in tum arise out of fluctuations, but the case cannot yet be made completely. Meanwhile, the Big Bang (incomplete) model deals with cosmic evolution in the large, as a balance between gravitational attraction and cosmic expansion. Its relationships are described by two simple equations derived from Einstein's theory of general relativity via local conservations of mass-energy and of momentum. Unfortunately, this model does not by itself produce the observed inhomogeneities. Neither the assumption of thermodynamic fluctuations following a "smooth" beginning nor that ofa chaotic beginning can itself account for stars and galaxies. A mystery remains. -THE EDITOR This chapter describes the efforts of cosmologists (and nature?) to form galactic and stellar structures, comments on them from the viewpoint of the homeokinetic principles (Soodak and Iberall, 1978) discussed elsewhere in this volume, and pursues some parallels in the origin of life. It appears that there are no serious problems in understanding the origin of stars, given the previous formation of a galactic gas mass. A number of reasonable, even likely, paths lead to star formation. On the other hand, the origin of inhomogeneities required to initiate galaxy formation is not yet known. Appendix 1 remarks on some advances in observational and theoretical cosmology made since the original writing of this chapter HARRY SOODAK • Department of Physics, The City College of New York, New York, New York
616 citations