Stochastic approximation (SA) has long been applied for problems of minimizing loss functions or root finding with noisy input information. As with all stochastic search algorithms, there are adjustable algorithm coefficients that must be specified, and that can have a profound effect on algorithm performance. It is known that choosing these coefficients according to an SA analog of the deterministic Newton-Raphson algorithm provides an optimal or near-optimal form of the algorithm. However, directly determining the required Hessian matrix (or Jacobian matrix for root finding) to achieve this algorithm form has often been difficult or impossible in practice. The paper presents a general adaptive SA algorithm that is based on a simple method for estimating the Hessian matrix, while concurrently estimating the primary parameters of interest. The approach applies in both the gradient-free optimization (Kiefer-Wolfowitz) and root-finding/stochastic gradient-based (Robbins-Monro) settings, and is based on the "simultaneous perturbation (SP)" idea introduced previously. The algorithm requires only a small number of loss function or gradient measurements per iteration-independent of the problem dimension-to adaptively estimate the Hessian and parameters of primary interest. Aside from introducing the adaptive SP approach, the paper presents practical implementation guidance, asymptotic theory, and a nontrivial numerical evaluation. Also included is a discussion and numerical analysis comparing the adaptive SP approach with the iterate-averaging approach to accelerated SA.

/pdf/adaptive-stochastic-approximation-by-the-simultaneous-32pbrvg1e5.pdf

Adaptive stochastic approximation by the simultaneous perturbation method

Real-time traffic signal control is an integral part of the urban traffic control system, and providing effective real-time traffic signal control for a large complex traffic network is an extremely challenging distributed control problem. This paper adopts the multiagent system approach to develop distributed unsupervised traffic responsive signal control models, where each agent in the system is a local traffic signal controller for one intersection in the traffic network. The first multiagent system is developed using hybrid computational intelligent techniques. Each agent employs a multistage online learning process to update and adapt its knowledge base and decision-making mechanism. The second multiagent system is developed by integrating the simultaneous perturbation stochastic approximation theorem in fuzzy neural networks (NN). The problem of real-time traffic signal control is especially challenging if the agents are used for an infinite horizon problem, where online learning has to take place continuously once the agent-based traffic signal controllers are implemented into the traffic network. A comprehensive simulation model of a section of the Central Business District of Singapore has been developed using PARAMICS microscopic simulation program. Simulation results show that the hybrid multiagent system provides significant improvement in traffic conditions when evaluated against an existing traffic signal control algorithm as well as the SPSA-NN-based multiagent system as the complexity of the simulation scenario increases. Using the hybrid NN-based multiagent system, the mean delay of each vehicle was reduced by 78% and the mean stoppage time, by 85% compared to the existing traffic signal control algorithm. The promising results demonstrate the efficacy of the hybrid NN-based multiagent system in solving large-scale traffic signal control problems in a distributed manner

/pdf/neural-networks-for-real-time-traffic-signal-control-3wiqqdm62w.pdf

Neural Networks for Real-Time Traffic Signal Control

Simulation optimization (SO) refers to the optimization of an objective function subject to constraints, both of which can be evaluated through a stochastic simulation. To address specific features of a particular simulation—discrete or continuous decisions, expensive or cheap simulations, single or multiple outputs, homogeneous or heterogeneous noise—various algorithms have been proposed in the literature. As one can imagine, there exist several competing algorithms for each of these classes of problems. This document emphasizes the difficulties in SO as compared to algebraic model-based mathematical programming, makes reference to state-of-the-art algorithms in the field, examines and contrasts the different approaches used, reviews some of the diverse applications that have been tackled by these methods, and speculates on future directions in the field.

/pdf/simulation-optimization-a-review-of-algorithms-and-3437395l44.pdf

Simulation optimization: a review of algorithms and applications

Modeling of distributed parameter systems for applications—A synthesized review from time–space separation

The global optimization of sensor locations for structural health monitoring systems is studied in this paper. First, the performance function based on damage detection is presented. Then, genetic algorithms (GAs) are adopted to search for the optimal locations of sensors. However, the simple GAs can result in infeasible solutions to the problem. Some improved strategies are presented in this paper, such as crossover based on identification code, mutation based on two gene bits, and improved convergence. The analytical results from the improved genetic algorithm are compared with the penalty function method and the forced mutation method. It is concluded that the convergence speed with the proposed improved genetic algorithm is faster than that with the penalty function method and the forced mutation method, and the result of placement optimization is better.

http://iopscience.iop.org/article/10.1088/0964-1726/13/3/011/pdf

Optimal placement of sensors for structural health monitoring using improved genetic algorithms

An approach to the selection of optimal sensor locations in distributed parameter systems

In this study, a nonlinear dynamic model of a cement grinding process, including a ball mill and an air separator in closed loop, is developed. This gray-box model consists of a set of algebraic and partial differential equations containing a set of unknown parameters. The selection of a model parametrization, the design of experiments, the estimation of unknown parameters from experimental data, and the model validation are discussed. Based on the resulting model, a dynamic simulator can be developed, which appears as a useful tool to analyze the process behavior and to understand the origin of instabilities observed in real-life operations. As a result, a cascaded control structure for regulating the mill flow rate, and a proportional integral controller for regulating the cement fineness are designed. Experimental data demonstrate the effectiveness of this control scheme. Alternatively, if on line measurements of the recirculated flow rate are available, a feedforward control of the feed flow rate is described, which ensures a better decoupling of mass flow rate and fineness regulation.

M. Remy

Papers

An approach to the selection of optimal sensor locations in distributed parameter systems

Modeling and control of cement grinding processes

Practical issues in distributed parameter estimation: Gradient computation and optimal experiment design

Linear robust control of S. cerevisiae fed-batch cultures at different scales

Modelling, Simulation and Evaluation of Control Loops for a Cement Grinding Process