Showing papers by "Yaakov Bar-Shalom published in 1974"

Dual effect, certainty equivalence, and separation in stochastic control

Abstract: In this paper the various policies in fixed end-time stochastic control are discussed first. The emphasis is on the difference between the feedback and closed-loop policies. It is shown how the closed-loop policy has the important property that it can be actively adaptive, while the feedback policy can only be passively adaptive. The feature of being actively adaptive is possible when the control has a dual effect, i.e., in addition to its effect on the state it affects the state uncertainty. The intimate connection between the neutrality (lack of dual effect) and certainty equivalence properties for a class of problems is proved. This new result is then used to widen the class of problems for which it was previously known that the certainty equivalence property holds.

Application of stochastic control theory to resource allocation under uncertainty

Abstract: The subject of this paper is the application of stochastic control theory to resource allocation under uncertainty. In these problems it is assumed that the results of a given allocation of resources are not known with certainty, but that a limited number of experiments can be performed to reduce the uncertainty. The problem is to develop a policy for performing experiments and allocating resources on the basis of the outcome of the experiments such that a performance index is optimized. The problem is first analyzed using the basic stochastic dynamic programming approach. A computationally practical algorithm for obtaining an approximate solution is then developed. This algorithm preserves the "closed-loop" feature of the dynamic programming solution in that the resulting decision policy depends both on the results of past experiments and on the statistics of the outcomes of future experiments. In other words, the present decision takes into account the value of future information. The concepts are discussed in the context of the general problem of allocating resources to repair machines where it is possible to perform a limited number of diagnostic experiments to learn more about potential failures. Illustrative numerical results are given.

Adaptive dual control for stochastic nonlinear systems with free end-time

Abstract: This short paper discusses the problem of controlling stochastic nonlinear systems where the end-time of the process has to be determined on-line. Since the optimal solution is not obtainable, a dual control method is developed for this class of problems. The algorithm has the property that the control, which is restricted to be causal, utilizes the statistical description of the future observations, i.e., it is of the closed-loop, rather than feedback type. This property allows the controller to enhance the estimation via the dual effect and thus improve the overall performance. The method is illustrated by applying it to an interception problem. Simulation studies on this particular problem provide further insight into how the dual effect of the control can be utilized in nonlinear problems. They also indicate the potential value of the algorithm presented here to practical problems.

Modeling and optimization of an effluent pollution surveillance system

Abstract: A surveillance system for a given number of effluents, each with certain contaminants, is considered. An effluent can be sampled to determine whether any of the contaminants exceeds its standard, in which case a violation is said to have been detected. Damage costs are associated only with undetected violations. These costs depend upon pollutant loadings, the nature of the pollutants, and the assimilative capacity of the receiving water. The objective is the following: find the sampling frequencies for each effluent (during a certain monitoring period) such as to minimize the total cost of the undetected violations. This is to be done subject to a certain budget constraint for the monitoring period. The cost of an undetected violation is defined as the expected value of the damage. These expectations are evaluated using the statistical knowledge available prior to the beginning of the monitoring period. The statistics of the contaminants are updated at the end of each monitoring period. It is shown that a very simple optimization procedure, namely, the maximum marginal return method, can be used to obtain the optimal sampling frequencies.

Hypothesis testing and estimation for air contaminants

Abstract: This paper deals with the problem of enforcing standards of maximum allowable atmospheric contaminants. This enforcement has to be done by deciding whether the actual level of contaminant is in violation or compliance with the standards. Due to inherent uncertainties, a decision is to be made only if the probability of error is below a prescribed level. The measurements of contaminant are modeled as a set of lognormal random variables with unknown mean and variance. A uniformly most powerful unbiased test is developed to test their mean against a given threshold. It is shown that the test can be carried out in a very simple manner with the help of a decision chart. This is done by plotting a point defined by the sufficient statistics of the observations and reading off the decision. The estimate of the actual level of contaminant is also discussed.