Showing papers by "Eric Chu published in 2011"

Distributed Optimization and Statistical Learning Via the Alternating Direction Method of Multipliers

Abstract: Many problems of recent interest in statistics and machine learning can be posed in the framework of convex optimization. Due to the explosion in size and complexity of modern datasets, it is increasingly important to be able to solve problems with a very large number of features or training examples. As a result, both the decentralized collection or storage of these datasets as well as accompanying distributed solution methods are either necessary or at least highly desirable. In this review, we argue that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas. The method was developed in the 1970s, with roots in the 1950s, and is equivalent or closely related to many other algorithms, such as dual decomposition, the method of multipliers, Douglas–Rachford splitting, Spingarn's method of partial inverses, Dykstra's alternating projections, Bregman iterative algorithms for l1 problems, proximal methods, and others. After briefly surveying the theory and history of the algorithm, we discuss applications to a wide variety of statistical and machine learning problems of recent interest, including the lasso, sparse logistic regression, basis pursuit, covariance selection, support vector machines, and many others. We also discuss general distributed optimization, extensions to the nonconvex setting, and efficient implementation, including some details on distributed MPI and Hadoop MapReduce implementations.

Solution of a nonsymmetric algebraic Riccati equation from a one-dimensional multistate transport model

Abstract: For the steady-state solution of a differential equation from a one-dimensional multistate model in transport theory, we shall derive and study a nonsymmetric algebraic Riccati equation B− − X F− − F+X + X B+X = 0, where F± ≡ (I − F)D± and B± ≡ B D± with positive diagonal matrices D± and possibly low-ranked matrices F and B. We prove the existence of the minimal positive solution X∗ under a set of physically reasonable assumptions and study its numerical computation by fixed-point iteration, Newton’s method and the doubling algorithm. We shall also study several special cases. For example when B and F are low ranked then X∗ = Γ ◦ (∑4 i=1 Ui V T i ) with low-ranked Ui and Vi that can be computed using more efficient iterative processes. Numerical examples will be given to illustrate our theoretical results.

A modified Newton's method for rational Riccati equations arising in stochastic control

Abstract: We consider the solution of the rational matrix equations, or generalized algebraic Riccati equations with rational terms, arising in stochastic optimal control in continuous- and discrete-time. Fixed-point iteration and (modified) Newton's methods will be considered. In particular, the convergence results of a new modified Newton's method, for both continuous- and discrete-time rational Riccati equations, will be presented.

ARE-type iterations for rational Riccati equations arising in stochastic control

Abstract: We consider the solution of the rational matrix equations, or generalized algebraic Riccati equations with rational terms, arising in stochastic optimal control in continuous-and discrete-time. The modified Newton's methods, the DARE-and CARE-type iterations for continuous- and discrete-time rational Riccati equations respectively, will be considered. In particular, the convergence of these new modified Newton's method will be proved.

Robust pole assignment via the Schur-Newton algorithms

Abstract: In [6], the pole assignment problem was considered for the control system ẋ = Ax + Bu with linear state-feedback u = Fx. An algorithm using the Schur form has been proposed, producing suboptimal solutions which can be refined further using optimization. In this paper, the algorithm is improved, with a weighted sum of the feedback gain and the departure from normality being used as the robustness measure. Newton refinement procedure is implemented, producing optimal solutions. Several illustrative numerical examples are presented.