This paper first presents necessary and sufficient conditions for the solvability of discrete time, mean-field, stochastic linear-quadratic optimal control problems. Then, by introducing several sequences of bounded linear operators, the problem becomes an operator stochastic LQ problem, in which the optimal control is a linear state feedback. Furthermore, from the form of the optimal control, the problem changes to a matrix dynamic optimization problem. Solving this optimization problem, we obtain the optimal feedback gain and thus the optimal control. Finally, by completing the square, the optimality of the above control is validated.

Discrete Time Mean-Field Stochastic Linear-Quadratic Optimal Control Problems

Optimal control problems in measure spaces governed by elliptic equations are considered for distributed and Neumann boundary control, which are known to promote sparse solutions. Optimality conditions are derived and some of the structural properties of their solutions, in particular sparsity, are discussed. A framework for their approximation is proposed which is efficient for numerical computations and for which we prove convergence and provide error estimates.

/pdf/approximation-of-elliptic-control-problems-in-measure-spaces-3cuqvfjqrw.pdf

Approximation of Elliptic Control Problems in Measure Spaces with Sparse Solutions

Discrete time mean-field stochastic linear-quadratic optimal control problems

This paper considers two classes of large population stochastic differential games connected to optimal and robust decentralized control of large-scale multiagent systems. The first problem ( P1 ) is one where each agent minimizes an exponentiated cost function, capturing risk-sensitive behavior, whereas in the second problem ( P2 ) each agent minimizes a worst-case risk-neutral cost function, where the “worst case” stems from the presence of an adversary entering each agent’s dynamics characterized by a stochastic differential equation. In both problems, the individual agents are coupled through the mean field term included in each agent’s cost function, which captures the average or mass behavior of the agents. We solve both P1 and P2 via mean field game theory. Specifically, we first solve a generic risk-sensitive optimal control problem and a generic stochastic zero-sum differential game, where the corresponding optimal controllers are applied by each agent to construct the mean field systems of P1 and P2 . We then characterize an approximated mass behavior effect on an individual agent via a fixed-point analysis of the mean field system. For each problem, P1 and P2 , we show that the approximated mass behavior is in fact the best estimate of the actual mass behavior in various senses as the population size, $N$ , goes to infinity. Moreover, we show that for finite $N$ , there exist $\epsilon$ - Nash equilibria for both P1 and P2 , where the corresponding individual Nash strategies are decentralized in terms of local state information and the approximated mass behavior. We also show that $\epsilon$ can be taken to be arbitrarily small when $N$ is sufficiently large. We show that the $\epsilon$ - Nash equilibria of P1 and P2 are partially equivalent in the sense that the individual Nash strategies share identical control laws, but the approximated mass behaviors for P1 and P2 are different, since in P2 , the mass behavior is also affected by the associated worst-case disturbance. Finally, we prove that the Nash equilibria for P1 and P2 both feature robustness, and as the parameter characterizing this robustness becomes infinite, the two Nash equilibria become identical and equivalent to that of the risk-neutral case, as in the one-agent risk-sensitive and robust control theory.

Linear Quadratic Risk-Sensitive and Robust Mean Field Games

Linear quadratic mean field Stackelberg differential games

In this paper, distributed games for large-population multiagent systems with random time-varying parameters are investigated, where the agents are coupled via their individual costs and the structure parameters are a family of independent Markov chains with identical generators. The cost function of each agent is a long-run average tracking-type functional with an unknown mean field coupling nonlinear term as “reference signal.” To reduce the computational complexity, the mean field approach is applied to construct distributed strategies. The population statistics effect (PSE) is used to approximate the average effect of all the agents, and the distributed strategies are given through solving a Markov jump tracking problem. Here the PSE is a deterministic quantity and can be obtained by solving the Stackelberg equilibrium of an auxiliary two-player game. It is shown that the closed-loop system is uniformly stable, and the distributed strategies are asymptotically optimal in the sense of Nash equilibrium,...

/pdf/mean-field-games-for-large-population-multiagent-systems-4zz5o1fmek.pdf

Ji-Feng Zhang

Papers

Mean Field Games for Large-Population Multiagent Systems with Markov Jump Parameters