Showing papers in "arXiv: Optimization and Control in 2011"

HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent

Abstract: Stochastic Gradient Descent (SGD) is a popular algorithm that can achieve state-of-the-art performance on a variety of machine learning tasks. Several researchers have recently proposed schemes to parallelize SGD, but all require performance-destroying memory locking and synchronization. This work aims to show using novel theoretical analysis, algorithms, and implementation that SGD can be implemented without any locking. We present an update scheme called HOGWILD! which allows processors access to shared memory with the possibility of overwriting each other's work. We show that when the associated optimization problem is sparse, meaning most gradient updates only modify small parts of the decision variable, then HOGWILD! achieves a nearly optimal rate of convergence. We demonstrate experimentally that HOGWILD! outperforms alternative schemes that use locking by an order of magnitude.

Distributed Delayed Stochastic Optimization

Abstract: We analyze the convergence of gradient-based optimization algorithms that base their updates on delayed stochastic gradient information. The main application of our results is to the development of gradient-based distributed optimization algorithms where a master node performs parameter updates while worker nodes compute stochastic gradients based on local information in parallel, which may give rise to delays due to asynchrony. We take motivation from statistical problems where the size of the data is so large that it cannot fit on one computer; with the advent of huge datasets in biology, astronomy, and the internet, such problems are now common. Our main contribution is to show that for smooth stochastic problems, the delays are asymptotically negligible and we can achieve order-optimal convergence results. In application to distributed optimization, we develop procedures that overcome communication bottlenecks and synchronization requirements. We show $n$-node architectures whose optimization error in stochastic problems---in spite of asynchronous delays---scales asymptotically as $\order(1 / \sqrt{nT})$ after $T$ iterations. This rate is known to be optimal for a distributed system with $n$ nodes even in the absence of delays. We additionally complement our theoretical results with numerical experiments on a statistical machine learning task.

Design of Optimal Sparse Feedback Gains via the Alternating Direction Method of Multipliers

Abstract: We design sparse and block sparse feedback gains that minimize the variance amplification (i.e., the $H_2$ norm) of distributed systems. Our approach consists of two steps. First, we identify sparsity patterns of feedback gains by incorporating sparsity-promoting penalty functions into the optimal control problem, where the added terms penalize the number of communication links in the distributed controller. Second, we optimize feedback gains subject to structural constraints determined by the identified sparsity patterns. In the first step, the sparsity structure of feedback gains is identified using the alternating direction method of multipliers, which is a powerful algorithm well-suited to large optimization problems. This method alternates between promoting the sparsity of the controller and optimizing the closed-loop performance, which allows us to exploit the structure of the corresponding objective functions. In particular, we take advantage of the separability of the sparsity-promoting penalty functions to decompose the minimization problem into sub-problems that can be solved analytically. Several examples are provided to illustrate the effectiveness of the developed approach.

Stochastic gradient descent on Riemannian manifolds

Abstract: Stochastic gradient descent is a simple approach to find the local minima of a cost function whose evaluations are corrupted by noise. In this paper, we develop a procedure extending stochastic gradient descent algorithms to the case where the function is defined on a Riemannian manifold. We prove that, as in the Euclidian case, the gradient descent algorithm converges to a critical point of the cost function. The algorithm has numerous potential applications, and is illustrated here by four examples. In particular a novel gossip algorithm on the set of covariance matrices is derived and tested numerically.

Primal-dual splitting algorithm for solving inclusions with mixtures of composite, Lipschitzian, and parallel-sum monotone operators

Abstract: We propose a primal-dual splitting algorithm for solving monotone inclusions involving a mixture of sums, linear compositions, and parallel sums of set-valued and Lipschitzian operators. An important feature of the algorithm is that the Lipschitzian operators present in the formulation can be processed individually via explicit steps, while the set-valued operators are processed individually via their resolvents. In addition, the algorithm is highly parallel in that most of its steps can be executed simultaneously. This work brings together and notably extends various types of structured monotone inclusion problems and their solution methods. The application to convex minimization problems is given special attention.

A distributed control strategy for reactive power compensation in smart microgrids

Abstract: We consider the problem of optimal reactive power compensation for the minimization of power distribution losses in a smart microgrid. We first propose an approximate model for the power distribution network, which allows us to cast the problem into the class of convex quadratic, linearly constrained, optimization problems. We then consider the specific problem of commanding the microgenerators connected to the microgrid, in order to achieve the optimal injection of reactive power. For this task, we design a randomized, gossip-like optimization algorithm. We show how a distributed approach is possible, where microgenerators need to have only a partial knowledge of the problem parameters and of the state, and can perform only local measurements. For the proposed algorithm, we provide conditions for convergence together with an analytic characterization of the convergence speed. The analysis shows that, in radial networks, the best performance can be achieved when we command cooperation among units that are neighbors in the electric topology. Numerical simulations are included to validate the proposed model and to confirm the analytic results about the performance of the proposed algorithm.

Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation

Abstract: Low-rank representation (LRR) is an effective method for subspace clustering and has found wide applications in computer vision and machine learning. The existing LRR solver is based on the alternating direction method (ADM). It suffers from $O(n^3)$ computation complexity due to the matrix-matrix multiplications and matrix inversions, even if partial SVD is used. Moreover, introducing auxiliary variables also slows down the convergence. Such a heavy computation load prevents LRR from large scale applications. In this paper, we generalize ADM by linearizing the quadratic penalty term and allowing the penalty to change adaptively. We also propose a novel rule to update the penalty such that the convergence is fast. With our linearized ADM with adaptive penalty (LADMAP) method, it is unnecessary to introduce auxiliary variables and invert matrices. The matrix-matrix multiplications are further alleviated by using the skinny SVD representation technique. As a result, we arrive at an algorithm for LRR with complexity $O(rn^2)$, where $r$ is the rank of the representation matrix. Numerical experiments verify that for LRR our LADMAP method is much faster than state-of-the-art algorithms. Although we only present the results on LRR, LADMAP actually can be applied to solving more general convex programs.

Review of Metaheuristics and Generalized Evolutionary Walk Algorithm

Abstract: Metaheuristic algorithms are often nature-inspired, and they are becoming very powerful in solving global optimization problems. More than a dozen of major metaheuristic algorithms have been developed over the last three decades, and there exist even more variants and hybrid of metaheuristics. This paper intends to provide an overview of nature-inspired metaheuristic algorithms, from a brief history to their applications. We try to analyze the main components of these algorithms and how and why they works. Then, we intend to provide a unified view of metaheuristics by proposing a generalized evolutionary walk algorithm (GEWA). Finally, we discuss some of the important open questions.

Projection Onto A Simplex

Abstract: This mini-paper presents a fast and simple algorithm to compute the projection onto the canonical simplex $\triangle^n$. Utilizing the Moreau's identity, we show that the problem is essentially a univariate minimization and the objective function is strictly convex and continuously differentiable. Moreover, it is shown that there are at most n candidates which can be computed explicitly, and the minimizer is the only one that falls into the correct interval.

On the reachability and observability of path and cycle graphs

Abstract: In this paper we investigate the reachability and observability properties of a network system, running a Laplacian based average consensus algorithm, when the communication graph is a path or a cycle. More in detail, we provide necessary and sufficient conditions, based on simple algebraic rules from number theory, to characterize all and only the nodes from which the network system is reachable (respectively observable). Interesting immediate corollaries of our results are: (i) a path graph is reachable (observable) from any single node if and only if the number of nodes of the graph is a power of two, $n=2^i, i\in atural$, and (ii) a cycle is reachable (observable) from any pair of nodes if and only if $n$ is a prime number. For any set of control (observation) nodes, we provide a closed form expression for the (unreachable) unobservable eigenvalues and for the eigenvectors of the (unreachable) unobservable subsystem.

Optimal Control of Vehicular Formations with Nearest Neighbor Interactions

Abstract: We consider the design of optimal localized feedback gains for one-dimensional formations in which vehicles only use information from their immediate neighbors. The control objective is to enhance coherence of the formation by making it behave like a rigid lattice. For the single-integrator model with symmetric gains, we establish convexity, implying that the globally optimal controller can be computed efficiently. We also identify a class of convex problems for double-integrators by restricting the controller to symmetric position and uniform diagonal velocity gains. To obtain the optimal non-symmetric gains for both the single- and the double-integrator models, we solve a parameterized family of optimal control problems ranging from an easily solvable problem to the problem of interest as the underlying parameter increases. When this parameter is kept small, we employ perturbation analysis to decouple the matrix equations that result from the optimality conditions, thereby rendering the unique optimal feedback gain. This solution is used to initialize a homotopy-based Newton's method to find the optimal localized gain. To investigate the performance of localized controllers, we examine how the coherence of large-scale stochastically forced formations scales with the number of vehicles. We establish several explicit scaling relationships and show that the best performance is achieved by a localized controller that is both non-symmetric and spatially-varying.

Software for Generation of Classes of Test Functions with Known Local and Global Minima for Global Optimization

Abstract: A procedure for generating non-differentiable, continuously differentiable, and twice continuously differentiable classes of test functions for multiextremal multidimensional box-constrained global optimization and a corresponding package of C subroutines are presented. Each test class consists of 100 functions. Test functions are generated by defining a convex quadratic function systematically distorted by polynomials in order to introduce local minima. To determine a class, the user defines the following parameters: (i) problem dimension, (ii) number of local minima, (iii) value of the global minimum, (iv) radius of the attraction region of the global minimizer, (v) distance from the global minimizer to the vertex of the quadratic function. Then, all other necessary parameters are generated randomly for all 100 functions of the class. Full information about each test function including locations and values of all local minima is supplied to the user. Partial derivatives are also generated where possible.

H_2-Optimal Decentralized Control over Posets: A State-Space Solution for State-Feedback

Abstract: We develop a complete state-space solution to H_2-optimal decentralized control of poset-causal systems with state-feedback. Our solution is based on the exploitation of a key separability property of the problem, that enables an efficient computation of the optimal controller by solving a small number of uncoupled standard Riccati equations. Our approach gives important insight into the structure of optimal controllers, such as controller degree bounds that depend on the structure of the poset. A novel element in our state-space characterization of the controller is a remarkable pair of transfer functions, that belong to the incidence algebra of the poset, are inverses of each other, and are intimately related to prediction of the state along the different paths on the poset. The results are illustrated by a numerical example.

Information-Geometric Optimization Algorithms: A Unifying Picture via Invariance Principles

Abstract: We present a canonical way to turn any smooth parametric family of probability distributions on an arbitrary search space $X$ into a continuous-time black-box optimization method on $X$, the \emph{information-geometric optimization} (IGO) method. Invariance as a design principle minimizes the number of arbitrary choices. The resulting \emph{IGO flow} conducts the natural gradient ascent of an adaptive, time-dependent, quantile-based transformation of the objective function. It makes no assumptions on the objective function to be optimized. The IGO method produces explicit IGO algorithms through time discretization. It naturally recovers versions of known algorithms and offers a systematic way to derive new ones. The cross-entropy method is recovered in a particular case, and can be extended into a smoothed, parametrization-independent maximum likelihood update (IGO-ML). For Gaussian distributions on $\mathbb{R}^d$, IGO is related to natural evolution strategies (NES) and recovers a version of the CMA-ES algorithm. For Bernoulli distributions on $\{0,1\}^d$, we recover the PBIL algorithm. From restricted Boltzmann machines, we obtain a novel algorithm for optimization on $\{0,1\}^d$. All these algorithms are unified under a single information-geometric optimization framework. Thanks to its intrinsic formulation, the IGO method achieves invariance under reparametrization of the search space $X$, under a change of parameters of the probability distributions, and under increasing transformations of the objective function. Theory strongly suggests that IGO algorithms have minimal loss in diversity during optimization, provided the initial diversity is high. First experiments using restricted Boltzmann machines confirm this insight. Thus IGO seems to provide, from information theory, an elegant way to spontaneously explore several valleys of a fitness landscape in a single run.

Generalized Forward-Backward Splitting

Abstract: This paper introduces the generalized forward-backward splitting algorithm for minimizing convex functions of the form $F + \sum_{i=1}^n G_i$, where $F$ has a Lipschitz-continuous gradient and the $G_i$'s are simple in the sense that their Moreau proximity operators are easy to compute. While the forward-backward algorithm cannot deal with more than $n = 1$ non-smooth function, our method generalizes it to the case of arbitrary $n$. Our method makes an explicit use of the regularity of $F$ in the forward step, and the proximity operators of the $G_i$'s are applied in parallel in the backward step. This allows the generalized forward backward to efficiently address an important class of convex problems. We prove its convergence in infinite dimension, and its robustness to errors on the computation of the proximity operators and of the gradient of $F$. Examples on inverse problems in imaging demonstrate the advantage of the proposed methods in comparison to other splitting algorithms.

Online Learning of Rested and Restless Bandits

Abstract: In this paper we study the online learning problem involving rested and restless multiarmed bandits with multiple plays. The system consists of a single player/user and a set of K finite-state discrete-time Markov chains (arms) with unknown state spaces and statistics. At each time step the player can play M arms. The objective of the user is to decide for each step which M of the K arms to play over a sequence of trials so as to maximize its long term reward. The restless multiarmed bandit is particularly relevant to the application of opportunistic spectrum access (OSA), where a (secondary) user has access to a set of K channels, each of time-varying condition as a result of random fading and/or certain primary users' activities.

Parallel ProXimal Algorithm for Image Restoration Using Hybrid Regularization { Extended version

Abstract: Regularization approaches have demonstrated their effectiveness for solving ill-posed problems. However, in the context of variational restoration methods, a challenging question remains, namely how to find a good regularizer. While total variation introduces staircase effects, wavelet domain regularization brings other artefacts, e.g. ringing. However, a trade-off can be made by introducing a hybrid regularization including several terms non necessarily acting in the same domain (e.g. spatial and wavelet transform domains). While this approach was shown to provide good results for solving deconvolution problems in the presence of additive Gaussian noise, an important issue is to efficiently deal with this hybrid regularization for more general noise models. To solve this problem, we adopt a convex optimization framework where the criterion to be minimized is split in the sum of more than two terms. For spatial domain regularization, isotropic or anisotropic total variation definitions using various gradient filters are considered. An accelerated version of the Parallel Proximal Algorithm is proposed to perform the minimization. Some difficulties in the computation of the proximity operators involved in this algorithm are also addressed in this paper. Numerical experiments performed in the context of Poisson data recovery, show the good behaviour of the algorithm as well as promising results concerning the use of hybrid regularization techniques.

Opportunistic Cooperation in Cognitive Femtocell Networks

Abstract: We investigate opportunistic cooperation between unlicensed secondary users and legacy primary users in a cognitive radio network. Specifically, we consider a model of a cognitive network where a secondary user can cooperatively transmit with the primary user in order to improve the latter's effective transmission rate. In return, the secondary user gets more opportunities for transmitting its own data when the primary user is idle. This kind of interaction between the primary and secondary users is different from the traditional dynamic spectrum access model in which the secondary users try to avoid interfering with the primary users while seeking transmission opportunities on vacant primary channels. In our model, the secondary users need to balance the desire to cooperate more (to create more transmission opportunities) with the need for maintaining sufficient energy levels for their own transmissions. Such a model is applicable in the emerging area of cognitive femtocell networks. We formulate the problem of maximizing the secondary user throughput subject to a time average power constraint under these settings. This is a constrained Markov Decision Problem and conventional solution techniques based on dynamic programming require either extensive knowledge of the system dynamics or learning based approaches that suffer from large convergence times. However, using the technique of Lyapunov optimization, we design a novel greedy and online control algorithm that overcomes these challenges and is provably optimal.

Combining Convex-Concave Decompositions and Linearization Approaches for solving BMIs, with application to Static Output Feedback

Abstract: A novel optimization method is proposed to minimize a convex function subject to bilinear matrix inequality (BMI) constraints. The key idea is to decompose the bilinear mapping as a difference between two positive semidefinite convex mappings. At each iteration of the algorithm the concave part is linearized, leading to a convex subproblem.Applications to various output feedback controller synthesis problems are presented. In these applications the subproblem in each iteration step can be turned into a convex optimization problem with linear matrix inequality (LMI) constraints. The performance of the algorithm has been benchmarked on the data from COMPleib library.

Iteration Complexity of Randomized Block-Coordinate Descent Methods for Minimizing a Composite Function ∗

Abstract: In this paper we develop a randomized block-coordinate descent method for minimizing the sum of a smooth and a simple nonsmooth block-separable convex function and prove that it obtains an $\epsilon$-accurate solution with probability at least $1-\rho$ in at most $O(\tfrac{n}{\epsilon} \log \tfrac{1}{\rho})$ iterations, where $n$ is the number of blocks. For strongly convex functions the method converges linearly. This extends recent results of Nesterov [Efficiency of coordinate descent methods on huge-scale optimization problems, CORE Discussion Paper #2010/2], which cover the smooth case, to composite minimization, while at the same time improving the complexity by the factor of 4 and removing $\epsilon$ from the logarithmic term. More importantly, in contrast with the aforementioned work in which the author achieves the results by applying the method to a regularized version of the objective function with an unknown scaling factor, we show that this is not necessary, thus achieving true iteration complexity bounds. In the smooth case we also allow for arbitrary probability vectors and non-Euclidean norms. Finally, we demonstrate numerically that the algorithm is able to solve huge-scale $\ell_1$-regularized least squares and support vector machine problems with a billion variables.

A Majorize-Minimize subspace approach for l2-l0 image regularization

Abstract: In this work, we consider a class of differentiable criteria for sparse image computing problems, where a nonconvex regularization is applied to an arbitrary linear transform of the target image. As special cases, it includes edge-preserving measures or frame-analysis potentials commonly used in image processing. As shown by our asymptotic results, the l2-l0 penalties we consider may be employed to provide approximate solutions to l0-penalized optimization problems. One of the advantages of the proposed approach is that it allows us to derive an efficient Majorize-Minimize subspace algorithm. The convergence of the algorithm is investigated by using recent results in nonconvex optimization. The fast convergence properties of the proposed optimization method are illustrated through image processing examples. In particular, its effectiveness is demonstrated on several data recovery problems.

Mean Field Asymptotics of Markov Decision Evolutionary Games and Teams

Abstract: We introduce Mean Field Markov games with $N$ players, in which each individual in a large population interacts with other randomly selected players. The states and actions of each player in an interaction together determine the instantaneous payoff for all involved players. They also determine the transition probabilities to move to the next state. Each individual wishes to maximize the total expected discounted payoff over an infinite horizon. We provide a rigorous derivation of the asymptotic behavior of this system as the size of the population grows to infinity. Under indistinguishability per type assumption, we show that under any Markov strategy, the random process consisting of one specific player and the remaining population converges weakly to a jump process driven by the solution of a system of differential equations. We characterize the solutions to the team and to the game problems at the limit of infinite population and use these to construct near optimal strategies for the case of a finite, but large, number of players. We show that the large population asymptotic of the microscopic model is equivalent to a (macroscopic) mean field stochastic game in which a local interaction is described by a single player against a population profile (the mean field limit). We illustrate our model to derive the equations for a dynamic evolutionary Hawk and Dove game with energy level.

Geometry of Injection Regions of Power Networks

Abstract: We investigate the constraints on power flow in networks and its implications to the optimal power flow problem. The constraints are described by the injection region of a network; this is the set of all vectors of power injections, one at each bus, that can be achieved while satisfying the network and operation constraints. If there are no operation constraints, we show the injection region of a network is the set of all injections satisfying the conservation of energy. If the network has a tree topology, e.g., a distribution network, we show that under voltage magnitude, line loss constraints, line flow constraints and certain bus real and reactive power constraints, the injection region and its convex hull have the same Pareto-front. The Pareto-front is of interest since these are the the optimal solutions to the minimization of increasing functions over the injection region. For non-tree networks, we obtain a weaker result by characterize the convex hull of the voltage constraint injection region for lossless cycles and certain combinations of cycles and trees.

Multi-objective integer programming: An improved recursive algorithm

Abstract: This paper introduces an improved recursive algorithm to generate the set of all nondominated objective vectors for the Multi-Objective Integer Programming (MOIP) problem. We significantly improve the earlier recursive algorithm of \"Ozlen and Azizo\u{g}lu by using the set of already solved subproblems and their solutions to avoid solving a large number of IPs. A numerical example is presented to explain the workings of the algorithm, and we conduct a series of computational experiments to show the savings that can be obtained. As our experiments show, the improvement becomes more significant as the problems grow larger in terms of the number of objectives.

Event-triggered and self-triggered stabilization of distributed networked control systems

Abstract: Event-triggered and self-triggered control have recently been proposed as implementation strategies that considerably reduce the resources required for control. Although most of the work so far has focused on closing a single control loop, some researchers have started to investigate how these new implementation strategies can be applied when closing multiple-feedback loops in the presence of physically distributed sensors and actuators. In this paper, we consider a scenario where the distributed sensors, actuators, and controllers communicate via a shared wired channel. We use our recent prescriptive framework for the event-triggered control of nonlinear systems to develop novel policies suitable for the considered distributed scenario. Afterwards, we explain how self-triggering rules can be deduced from the developed event-triggered strategies.

A Simplified Approach to Recovery Conditions for Low Rank Matrices

Abstract: Recovering sparse vectors and low-rank matrices from noisy linear measurements has been the focus of much recent research. Various reconstruction algorithms have been studied, including $\ell_1$ and nuclear norm minimization as well as $\ell_p$ minimization with $p<1$. These algorithms are known to succeed if certain conditions on the measurement map are satisfied. Proofs of robust recovery for matrices have so far been much more involved than in the vector case. In this paper, we show how several robust classes of recovery conditions can be extended from vectors to matrices in a simple and transparent way, leading to the best known restricted isometry and nullspace conditions for matrix recovery. Our results rely on the ability to "vectorize" matrices through the use of a key singular value inequality.

On the Mathematical Structure of Balanced Chemical Reaction Networks Governed by Mass Action Kinetics

Abstract: Motivated by recent progress on the interplay between graph theory, dynamics, and systems theory, we revisit the analysis of chemical reaction networks described by mass action kinetics. For reaction networks possessing a thermodynamic equilibrium we derive a compact formulation exhibiting at the same time the structure of the complex graph and the stoichiometry of the network, and which admits a direct thermodynamical interpretation. This formulation allows us to easily characterize the set of equilibria and their stability properties. Furthermore, we develop a framework for interconnection of chemical reaction networks. Finally we discuss how the established framework leads to a new approach for model reduction.

Ergodic Mirror Descent

Abstract: We generalize stochastic subgradient descent methods to situations in which we do not receive independent samples from the distribution over which we optimize, but instead receive samples that are coupled over time. We show that as long as the source of randomness is suitably ergodic---it converges quickly enough to a stationary distribution---the method enjoys strong convergence guarantees, both in expectation and with high probability. This result has implications for stochastic optimization in high-dimensional spaces, peer-to-peer distributed optimization schemes, decision problems with dependent data, and stochastic optimization problems over combinatorial spaces.

Provably Safe and Robust Learning-Based Model Predictive Control

Abstract: Controller design faces a trade-off between robustness and performance, and the reliability of linear controllers has caused many practitioners to focus on the former. However, there is renewed interest in improving system performance to deal with growing energy constraints. This paper describes a learning-based model predictive control (LBMPC) scheme that provides deterministic guarantees on robustness, while statistical identification tools are used to identify richer models of the system in order to improve performance; the benefits of this framework are that it handles state and input constraints, optimizes system performance with respect to a cost function, and can be designed to use a wide variety of parametric or nonparametric statistical tools. The main insight of LBMPC is that safety and performance can be decoupled under reasonable conditions in an optimization framework by maintaining two models of the system. The first is an approximate model with bounds on its uncertainty, and the second model is updated by statistical methods. LBMPC improves performance by choosing inputs that minimize a cost subject to the learned dynamics, and it ensures safety and robustness by checking whether these same inputs keep the approximate model stable when it is subject to uncertainty. Furthermore, we show that if the system is sufficiently excited, then the LBMPC control action probabilistically converges to that of an MPC computed using the true dynamics.

A unifying Lyapunov-based framework for the event-triggered control of nonlinear systems

Abstract: We present a prescriptive framework for the event-triggered control of nonlinear systems. Rather than closing the loop periodically, as traditionally done in digital control, in event-triggered implementations the loop is closed according to a state-dependent criterion. Event-triggered control is especially well suited for embedded systems and networked control systems since it reduces the amount of resources needed for control such as communication bandwidth. By modeling the event-triggered implementations as hybrid systems, we provide Lyapunov-based conditions to guarantee the stability of the resulting closed-loop system and explain how they can be utilized to synthesize event-triggering rules. We illustrate the generality of the approach by showing how it encompasses several existing event-triggering policies and by developing new strategies which further reduce the resources needed for control.