This article surveyed the major results in iterative learning control (ILC) analysis and design over the past two decades. Problems in stability, performance, learning transient behavior, and robustness were discussed along with four design techniques that have emerged as among the most popular. The content of this survey was selected to provide the reader with a broad perspective of the important ideas, potential, and limitations of ILC. Indeed, the maturing field of ILC includes many results and learning algorithms beyond the scope of this survey. Though beginning its third decade of active research, the field of ILC shows no sign of slowing down.

A survey of iterative learning control

Policy search methods can allow robots to learn control policies for a wide range of tasks, but practical applications of policy search often require hand-engineered components for perception, state estimation, and low-level control. In this paper, we aim to answer the following question: does training the perception and control systems jointly end-to-end provide better performance than training each component separately? To this end, we develop a method that can be used to learn policies that map raw image observations directly to torques at the robot's motors. The policies are represented by deep convolutional neural networks (CNNs) with 92,000 parameters, and are trained using a guided policy search method, which transforms policy search into supervised learning, with supervision provided by a simple trajectory-centric reinforcement learning method. We evaluate our method on a range of real-world manipulation tasks that require close coordination between vision and control, such as screwing a cap onto a bottle, and present simulated comparisons to a range of prior policy search methods.

/pdf/end-to-end-training-of-deep-visuomotor-policies-1vqd5uo50d.pdf

End-to-end training of deep visuomotor policies

This tutorial/survey paper: (1) provides a concise point of departure for researchers and practitioners alike wishing to assess the current state of the art in the control and monitoring of civil engineering structures; and (2) provides a link between structural control and other fields of control theory, pointing out both differences and similarities, and points out where future research and application efforts are likely to prove fruitful. The paper consists of the following sections: section 1 is an introduction; section 2 deals with passive energy dissipation; section 3 deals with active control; section 4 deals with hybrid and semiactive control systems; section 5 discusses sensors for structural control; section 6 deals with smart material systems; section 7 deals with health monitoring and damage detection; and section 8 deals with research needs. An extensive list of references is provided in the references section.

Structural control: past, present, and future

From the Publisher:
Fuzzy Modeling for Control addresses fuzzy modeling from the systems and control engineering point of view. It focuses on the selection of appropriate model structures, on the acquisition of dynamic fuzzy models from process measurements (fuzzy identification), and on the design of nonlinear controllers based on fuzzy models. The main features of the presented techniques are illustrated by means of simple examples. In addition, three real-world applications are described. Finally, software tools for building fuzzy models from measurements are available from the author.

Fuzzy Modeling for Control

Reinforcement learning holds the promise of enabling autonomous robots to learn large repertoires of behavioral skills with minimal human intervention. However, robotic applications of reinforcement learning often compromise the autonomy of the learning process in favor of achieving training times that are practical for real physical systems. This typically involves introducing hand-engineered policy representations and human-supplied demonstrations. Deep reinforcement learning alleviates this limitation by training general-purpose neural network policies, but applications of direct deep reinforcement learning algorithms have so far been restricted to simulated settings and relatively simple tasks, due to their apparent high sample complexity. In this paper, we demonstrate that a recent deep reinforcement learning algorithm based on off-policy training of deep Q-functions can scale to complex 3D manipulation tasks and can learn deep neural network policies efficiently enough to train on real physical robots. We demonstrate that the training times can be further reduced by parallelizing the algorithm across multiple robots which pool their policy updates asynchronously. Our experimental evaluation shows that our method can learn a variety of 3D manipulation skills in simulation and a complex door opening skill on real robots without any prior demonstrations or manually designed representations.

Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates

Neural networks for control systems: a survey

A novel technique, directly using artificial neural networks, is proposed for the adaptive control of nonlinear systems. The ability of neural networks to model arbitrary nonlinear functions and their inverses is exploited. The use of nonlinear function inverses raises questions of the existence of the inverse operators. These are investigated and results are given characterising the invertibility of a class of nonlinear dynamical systems. The control structure used is internal model control. It is used to directly incorporate networks modelling the plant and its inverse within the control strategy. The potential of the proposed method is demonstrated by an example.

Neural networks for nonlinear internal model control

Focuses on the use of adaptive connectionist networks for identification and control of nonlinear dynamic systems. Control problems are usually solved using information about the controlled system and a series of procedures to reach the desired control goal. From this point of view it is possible to decompose the control problem into two sub-problems: (1) a representation problem dealing with the way in which the information and dynamics of the system will be represented; and (2) a structural problem which concerns the definition of the architecture and the procedures which use the information on the system in order to reach the desired control goal. The connectionist approach has great potential for control applications since it addresses both problems, providing system representations and control procedures. The authors outline their work in both these areas. >

https://ieeexplore.ieee.org/iel3/1750/4548/00180905.pdf

Connectionist representations and control structures

Comminution is the stage with the largest energy consumption in the mining process. Therefore, several pre-treatments have been proposed to reduce the energy requirements of this stage. This work analyzed the effect of a heating pre-treatment on the Bond index. A conventional heating pre-treatment was applied to a Chilean copper ore. The ore was heated to temperatures from 300 to 600 °C using a conventional furnace, resulting in a reduction of 19% in the Bond work index. Due to the pre-treatment, the mineral cracked in several areas. Microfracture and composition analyses of these areas confirmed that crack generation in the ore is due to the thermal stress produced by the pre-treatment. The fracture analysis explains the reduction in the Bond work index, since crack generation started at similar temperatures to those at which the reduction in the Bond work index was observed. In addition, the analysis also shows that micro-cracks occur between and through different phases, which may have an impact on mineral liberation. These results also show that, under a moderate high temperature, an important reduction in energy consumption can be obtained.

/pdf/heating-pre-treatment-of-copper-ores-and-its-effects-on-the-17vbxqe3.pdf

D. Sbarbaro

Papers

Neural networks for control systems: a survey

Neural networks for nonlinear internal model control

Connectionist representations and control structures

Heating Pre-Treatment of Copper Ores and Its Effects on the Bond Work Index

Designing nonlinear controllers using connectionist networks