https://axon.cs.byu.edu/~martinez/classes/678/Presentations/Yao.pdf

Extreme learning machine: Theory and applications

I have developed "tennis elbow" from lugging this book around the past four weeks, but it is worth the pain, the effort, and the aspirin. It is also worth the (relatively speaking) bargain price. Including appendixes, this book contains 894 pages of text. The entire panorama of the neural sciences is surveyed and examined, and it is comprehensive in its scope, from genomes to social behaviors. The editors explicitly state that the book is designed as "an introductory text for students of biology, behavior, and medicine," but it is hard to imagine any audience, interested in any fragment of neuroscience at any level of sophistication, that would not enjoy this book. The editors have done a masterful job of weaving together the biologic, the behavioral, and the clinical sciences into a single tapestry in which everyone from the molecular biologist to the practicing psychiatrist can find and appreciate his or

Principles of Neural Science

This paper surveys the field of reinforcement learning from a computer-science perspective. It is written to be accessible to researchers familiar with machine learning. Both the historical basis of the field and a broad selection of current work are summarized. Reinforcement learning is the problem faced by an agent that learns behavior through trial-and-error interactions with a dynamic environment. The work described here has a resemblance to work in psychology, but differs considerably in the details and in the use of the word "reinforcement." The paper discusses central issues of reinforcement learning, including trading off exploration and exploitation, establishing the foundations of the field via Markov decision theory, learning from delayed reinforcement, constructing empirical models to accelerate learning, making use of generalization and hierarchy, and coping with hidden state. It concludes with a survey of some implemented systems and an assessment of the practical utility of current methods for reinforcement learning.

/pdf/reinforcement-learning-a-survey-38lmyylg2y.pdf

Reinforcement learning: a survey

Planning algorithms are impacting technical disciplines and industries around the world, including robotics, computer-aided design, manufacturing, computer graphics, aerospace applications, drug design, and protein folding. This coherent and comprehensive book unifies material from several sources, including robotics, control theory, artificial intelligence, and algorithms. The treatment is centered on robot motion planning but integrates material on planning in discrete spaces. A major part of the book is devoted to planning under uncertainty, including decision theory, Markov decision processes, and information spaces, which are the “configuration spaces” of all sensor-based planning problems. The last part of the book delves into planning under differential constraints that arise when automating the motions of virtually any mechanical system. Developed from courses taught by the author, the book is intended for students, engineers, and researchers in robotics, artificial intelligence, and control theory as well as computer graphics, algorithms, and computational biology.

Planning Algorithms: Introductory Material

This paper surveys the field of reinforcement learning from a computer-science perspective. It is written to be accessible to researchers familiar with machine learning. Both the historical basis of the field and a broad selection of current work are summarized. Reinforcement learning is the problem faced by an agent that learns behavior through trial-and-error interactions with a dynamic environment. The work described here has a resemblance to work in psychology, but differs considerably in the details and in the use of the word ``reinforcement.'' The paper discusses central issues of reinforcement learning, including trading off exploration and exploitation, establishing the foundations of the field via Markov decision theory, learning from delayed reinforcement, constructing empirical models to accelerate learning, making use of generalization and hierarchy, and coping with hidden state. It concludes with a survey of some implemented systems and an assessment of the practical utility of current methods for reinforcement learning.

Reinforcement Learning: A Survey

Therapeutic enhancement of humoral immune response to microbial attack is addressed as the stochastic optimal control of a dynamic system. Without therapy, the modeled immune response depends upon the initial concentration of pathogens in a simulated attack. Immune response can be augmented by agents that kill the pathogen directly, that stimulate the production of plasma cells or antibodies, or that enhance organ health. Using a generic mathematical model of immune response to the infection (i.e., of the dynamic state of the system), previous papers demonstrated optimal (open-loop) and neighboring-optimal (closed-loop) control solutions that defeat the pathogen and preserve organ health, given initial conditions that otherwise would be lethal [Optimal Contr. Appl. Methods 23 (2002) 91, Bioinformatics 18 (2002) 1227] . Therapies based on separate and combined application of the agents were derived by minimizing a quadratic cost function that weighted both system response and drug usage, providing implicit control over harmful side effects. Here, we focus on the effects that corrupted or incomplete measurements of the dynamic state may have on neighboring-optimal feedback control. Imperfect measurements degrade the precision of feedback adjustments to therapy; however, optimal state estimation allows the feedback strategy to be implemented with incomplete measurements and minimizes the expected effects of measurement error. Complete observability of the perturbed state for this four state example is provided by measurement of four of the six possible pairs of two variables, either set of three variables, or all four variables. The inclusion of state estimation extends the applicability of optimal control theory for developing new therapeutic protocols to enhance immune response.

Stochastic optimal therapy for enhanced immune response.

Stochastic robustness, a simple technique used to estimate the robustness of linear, time-invariant systems, is applied to a twin-jet transport aircraft control system. Concepts behind stochastic stability robustness are extended to stochastic performance robustness. Stochastic performance robustness measures based on classical design specifications and measures specific to aircraft handling qualities are introduced. Confidence intervals for comparing two control system designs are presented. The application of stochastic performance robustness, the use of confidence intervals, and tradeoffs between performance objectives are demonstrated by means of the twin-jet aircraft example.

Stochastic measures of performance robustness in aircraft control systems

Manual attitude control for Lunar Module employing directional stability, coordinated turn and attitude command

Manual attitude control of the lunar module

Analytical and numerical estimates of the stalling characteristics of a small, single-engine aircraft are compared with flight test results. Analyses include nonlinear simulation and linear stability-and-control evaluation, using aerodynamic and thrust characteristics obtained from a full-scale test in the NASA Langley Research Center 30 X 60-ft Wind Tunnel as well as subscaie model test data. Flight tests include prestall calibration runs, symmetric gradual stalls, and mildly accelerated stalls in the vertical plane. These tests tend to confirm predictions based upon wind-tunnel results, and they indicate areas in which special care must be taken in collecting data for aerodynamic parameter identification.

Stalling Characteristics of a General Aviation Aircraft

Methodologies are presented for the analysis and design of stability augmentation control laws for aircraft in which "hard" displacement and rate limiting are significant. Candidate control laws are derived using the linearquadratic (LQ) regulator. Analytical and computational estimates of the stability limits imposed by control saturation are presented using state trajectories, as well as describing functions and eigenvalue computation. Analysis also includes an investigation of the interaction of the state-space saturation and stability boundaries for various choices of LQ weighting matrices. For minimum energy control, the saturation and stability boundaries are shown to be parallel. In this case, there is a direct relation between the solution to the matrix Riccati equation and the aircraft's open-loop dynamics. V IRTUALLY all aircraft control system actuation mechanisms possess "hard" limits on deflection and rate of deflection, and accounting for these limits is particularly important for aircraft designed with relaxed or negative static stability. Relaxed static stability generally leads to reduced weight and trim drag, both of which enhance the aircraft's performance.1 Relaxed static stability also may lead to increased control activity.2 For such aircraft, consideration must be given to the effects of control saturation on the initiation and, more importantly, the arrestment of dynamic motions, if departure from controlled flight is to be prevented. It may be possible to provide hardware actuation limits large enough so that normal disturbances and command inputs do not force the controls and to their mechanical, hydraulic, or electrical "stops"; however, there is no guarantee that commanded control deflections or rates will not exceed any established limit. Therefore, it is important to define the region within which stability and satisfactory response can be assured. This paper explores the effects of deflection and rate saturation on the stability of a statically unstable aircraft. This paper also reports on the development of methodologies for the analysis and design of stability augmentation control laws, with emphasis on response to initial conditions. (Command response is presented in Ref. 3 is the subject of a separate paper.11) Candidate stability augmentation system (SAS) control laws are derived using optimal control theory, with the linear-quadratic (LQ) regulator taken as the fundamental solution. Analytical and computational estimates of the stability limits imposed by control saturation are presented.

Robert F. Stengel

Papers

Stochastic optimal therapy for enhanced immune response.

Stochastic measures of performance robustness in aircraft control systems

Manual attitude control of the lunar module

Stalling Characteristics of a General Aviation Aircraft

Effects of displacement and rate saturation on the control of statically unstable aircraft