Monte Carlo methods are revolutionizing the on-line analysis of data in fields as diverse as financial modeling, target tracking and computer vision. These methods, appearing under the names of bootstrap filters, condensation, optimal Monte Carlo filters, particle filters and survival of the fittest, have made it possible to solve numerically many complex, non-standard problems that were previously intractable. This book presents the first comprehensive treatment of these techniques, including convergence results and applications to tracking, guidance, automated target recognition, aircraft navigation, robot navigation, econometrics, financial modeling, neural networks, optimal control, optimal filtering, communications, reinforcement learning, signal enhancement, model averaging and selection, computer vision, semiconductor design, population biology, dynamic Bayesian networks, and time series analysis. This will be of great value to students, researchers and practitioners, who have some basic knowledge of probability. Arnaud Doucet received the Ph. D. degree from the University of Paris-XI Orsay in 1997. From 1998 to 2000, he conducted research at the Signal Processing Group of Cambridge University, UK. He is currently an assistant professor at the Department of Electrical Engineering of Melbourne University, Australia. His research interests include Bayesian statistics, dynamic models and Monte Carlo methods. Nando de Freitas obtained a Ph.D. degree in information engineering from Cambridge University in 1999. He is presently a research associate with the artificial intelligence group of the University of California at Berkeley. His main research interests are in Bayesian statistics and the application of on-line and batch Monte Carlo methods to machine learning. Neil Gordon obtained a Ph.D. in Statistics from Imperial College, University of London in 1993. He is with the Pattern and Information Processing group at the Defence Evaluation and Research Agency in the United Kingdom. His research interests are in time series, statistical data analysis, and pattern recognition with a particular emphasis on target tracking and missile guidance.

/pdf/sequential-monte-carlo-methods-in-practice-3kfxeebn8e.pdf

Sequential Monte Carlo methods in practice

Survey of Numerical Methods for Trajectory Optimization

Introduction to linear and nonlinear programming

This paper assumes a requirement for an unmanned multiunit satellite to be assembled in orbit. The requirement to be met is to bring the satellites together so tha t they do not collide but actually rendezvous. The equations of motion of the rendezvous satellite in a relative coordinate system are derived and used to compute a final injection velocity which would effect collision after a time r. The velocity is corrected periodically by a command guidance system and just before impact retrothrust is applied. A terminal infrared homing sj^stem is required to actually accomplish physical contact and joining of the satellites. The first satellite placed in orbit is the "control satellite" and controls all the satellites to be assembled and contains the ccmputer, command guidance equipment, precision orientation equipment, and other features necessary to effect rendezvous. The succeeding satellites contain a propulsion system, a rough at t i tude control system, and a command receiver plus whatever scientific equipment they carry to perform their basic mission. This paper presents the following:

Terminal Guidance System for Satellite Rendezvous

A three-dimensional, neuromusculoskeletal model of the body was combined with dynamic optimization theory to simulate normal walking on level ground. The body was modeled as a 23 degree-of-freedom mechanical linkage, actuated by 54 muscles. The dynamic optimization problem was to calculate the muscle excitation histories, muscle forces, and limb motions subject to minimum metabolic energy expenditure per unit distance traveled. Muscle metabolic energy was calculated by slimming five terms: the basal or resting heat, activation heat, maintenance heat, shortening heat, and the mechanical work done by all the muscles in the model. The gait cycle was assumed to be symmetric; that is, the muscle excitations for the right and left legs and the initial and terminal states in the model were assumed to be equal. Importantly, a tracking problem was not solved. Rather only a set of terminal constraints was placed on the states of the model to enforce repeatability of the gait cycle. Quantitative comparisons of the model predictions with patterns of body-segmental displacements, ground-reaction forces, and muscle activations obtained from experiment show that the simulation reproduces the salient features of normal gait. The simulation results suggest that minimum metabolic energy per unit distance traveled is a valid measure of walking performance.

https://homes.cs.washington.edu/~todorov/courses/amath579/reading/AndersonPandy.pdf

Dynamic optimization of human walking.

Several methods exist for converting optimal control problems into parameter optimization problems Numerical integration, collocation, direct transcription, and differential inclusion are examples of these conversion methods Because of the similarities of these methods, they are categorized in this paper by their numerical integration technique, the order of the integration technique, and the unknowns of the parameter optimization problem The integration techniques are divided into explicit and implicit approaches, and the unknowns are some combination of design parameters, control parameters, and state parameters The method called numerical integration has the controls as unknowns and uses explicit integration Collocation and direct transcription have the controls and states as unknowns and use implicit integration Differential inclusion has the states as unknowns and uses implicit integration (Author)

Conversion of optimal control problems into parameter optimization problems

This paper describes a computational method for solving optimal control problems involving large-scale, nonlinear, dynamical systems. Central to the approach is the idea that any optimal control problem can be converted into a standard nonlinear programming problem by parameterizing each control history using a set of nodal points, which then become the variables in the resulting parameter optimization problem. A key feature of the method is that it dispenses with the need to solve the two-point, boundary-value problem derived from the necessary conditions of optimal control theory. Gradient-based methods for solving such problems do not always converge due to computational errors introduced by the highly nonlinear characteristics of the costate variables. Instead, by converting the optimal control problem into a parameter optimization problem, any number of well-developed and proven nonlinear programming algorithms can be used to compute the near-optimal control trajectories. The utility of the parameter optimization approach for solving general optimal control problems for human movement is demonstrated by applying it to a detailed optimal control model for maximum-height human jumping. The validity of the near-optimal control solution is established by comparing it to a solution of the two-point, boundary-value problem derived on the basis of a bang-bang optimal control algorithm. Quantitative comparisons between model and experiment further show that the parameter optimization solution reproduces the major features of a maximum-height, countermovement jump (i.e., trajectories of body-segmental displacements, vertical and fore-aft ground reaction forces, displacement, velocity, and acceleration of the whole-body center of mass, pattern of lower-extremity muscular activity, jump height, and total ground contact time).

A parameter optimization approach for the optimal control of large-scale musculoskeletal systems.

An integrated sequence stratigraphic study based on outcrop, core and wireline log data documents the combined impact of Cretaceous eustacy and oceanic anoxic events on carbonate shelf morphology and facies distributions in the northern Gulf of Mexico. The diverse facies and abundant data of the Comanche platform serve as a nearly complete global reference section and provide a sensitive record of external processes affecting Cretaceous platform development. Regional cross-sections across the shoreline to shelf-margin profile provide a detailed record of mixed carbonate–siliciclastic strata for the Hauterivian to lower Campanian stages (ca 136 to 80 Ma). The study window on the slowly subsiding passive margin allows the stratigraphic response to external forcing mechanisms to be isolated from regional structural processes. Three second-order supersequences comprised of eight composite sequences are recognized in the Valanginian–Barremian, the Aptian–Albian and the Cenomanian–Campanian. The Valanginian–Barremian supersequence transitioned from a siliciclastic ramp to carbonate rimmed shelf and is a product of glacial ice accumulation and melting, as well as variable rates of mid-ocean ridge volcanism. The Aptian–Albian supersequence chronicles the drowning and recovery of the platform surrounding oceanic anoxic events 1a and 1b. The Cenomanian–Campanian supersequence similarly documents shelf drowning following oceanic anoxic event 1d, after which the platform evolved to a deep-subtidal system consisting of anoxic/dysoxic shale and chalk in the time surrounding oceanic anoxic event 2. Each period of oceanic anoxia is associated with composite sequence maximum flooding, termination of carbonate shelf sedimentation and deposition of condensed shale units in distally steepened ramp profiles. Composite sequences unaffected by oceanic anoxic events consist of aggradational to progradational shelves with an abundance of grain-dominated facies and shallow-subtidal to intertidal environments. Because they are products of eustacy and global oceanographic processes, the three supersequences and most composite sequences defined in the south Texas passive margin are recognizable in other carbonate platforms and published eustatic sea-level curves.

Oceanographic and eustatic control of carbonate platform evolution and sequence stratigraphy on the Cretaceous (Valanginian–Campanian) passive margin, northern Gulf of Mexico

Response and recovery of the Comanche carbonate platform surrounding multiple Cretaceous oceanic anoxic events, northern Gulf of Mexico

Minimum energy-loss guidance for the aero-assisted plane change of an orbiting vehicle is developed and applied to the plane change of a circular orbit. First, trajectories which minimize the fuel required to change the orbital plane are computed for a realistic vehicle. From these trajectories, it is observed that the fuel weight is minimized if the velocity at exit from the atmosphere is maximized. Next, for the atmospheric turn, approximate optimal controls (angle of attack and bank angle) which maximize the exit velocity are derived. Finally, the minimum-fuel problem is resolved using optimal guidance for the atmospheric part of the trajectory, and the optimization problem reduces to a one-dimensional parameter minimization. Successful plane changes up to 40 deg are demonstrated. Optimal guidance requires up to 14 percent more fuel than the 'true' optimum but only 50 percent of the fuel required by the single-impulse maneuver. Finally, the guidance law developed here is implementable because only algebraic manipulations are required.

David G. Hull

Papers

Conversion of optimal control problems into parameter optimization problems

A parameter optimization approach for the optimal control of large-scale musculoskeletal systems.

Oceanographic and eustatic control of carbonate platform evolution and sequence stratigraphy on the Cretaceous (Valanginian–Campanian) passive margin, northern Gulf of Mexico

Response and recovery of the Comanche carbonate platform surrounding multiple Cretaceous oceanic anoxic events, northern Gulf of Mexico

Minimum energy-loss guidance for aeroassisted orbital plane change