scispace - formally typeset
Search or ask a question

Showing papers on "Convex optimization published in 2007"


Journal ArticleDOI
TL;DR: In this paper, the preconditioned conjugate gradients (PCG) algorithm is used to compute the search direction for sparse least-squares programs (LSPs), which can be reformulated as convex quadratic programs, and then solved by several standard methods such as interior-point methods.
Abstract: Recently, a lot of attention has been paid to regularization based methods for sparse signal reconstruction (e.g., basis pursuit denoising and compressed sensing) and feature selection (e.g., the Lasso algorithm) in signal processing, statistics, and related fields. These problems can be cast as -regularized least-squares programs (LSPs), which can be reformulated as convex quadratic programs, and then solved by several standard methods such as interior-point methods, at least for small and medium size problems. In this paper, we describe a specialized interior-point method for solving large-scale -regularized LSPs that uses the preconditioned conjugate gradients algorithm to compute the search direction. The interior-point method can solve large sparse problems, with a million variables and observations, in a few tens of minutes on a PC. It can efficiently solve large dense problems, that arise in sparse signal recovery with orthogonal transforms, by exploiting fast algorithms for these transforms. The method is illustrated on a magnetic resonance imaging data set.

2,047 citations


Journal ArticleDOI
TL;DR: This paper introduces two-step 1ST (TwIST) algorithms, exhibiting much faster convergence rate than 1ST for ill-conditioned problems, and introduces a monotonic version of TwIST (MTwIST); although the convergence proof does not apply, the effectiveness of the new methods are experimentally confirmed on problems of image deconvolution and of restoration with missing samples.
Abstract: Iterative shrinkage/thresholding (1ST) algorithms have been recently proposed to handle a class of convex unconstrained optimization problems arising in image restoration and other linear inverse problems. This class of problems results from combining a linear observation model with a nonquadratic regularizer (e.g., total variation or wavelet-based regularization). It happens that the convergence rate of these 1ST algorithms depends heavily on the linear observation operator, becoming very slow when this operator is ill-conditioned or ill-posed. In this paper, we introduce two-step 1ST (TwIST) algorithms, exhibiting much faster convergence rate than 1ST for ill-conditioned problems. For a vast class of nonquadratic convex regularizers (lscrP norms, some Besov norms, and total variation), we show that TwIST converges to a minimizer of the objective function, for a given range of values of its parameters. For noninvertible observation operators, we introduce a monotonic version of TwIST (MTwIST); although the convergence proof does not apply to this scenario, we give experimental evidence that MTwIST exhibits similar speed gains over IST. The effectiveness of the new methods are experimentally confirmed on problems of image deconvolution and of restoration with missing samples.

1,870 citations


Journal ArticleDOI
TL;DR: The implementation of the penalized likelihood methods for estimating the concentration matrix in the Gaussian graphical model is nontrivial, but it is shown that the computation can be done effectively by taking advantage of the efficient maxdet algorithm developed in convex optimization.
Abstract: SUMMARY We propose penalized likelihood methods for estimating the concentration matrix in the Gaussian graphical model. The methods lead to a sparse and shrinkage estimator of the concentration matrix that is positive definite, and thus conduct model selection and estimation simultaneously. The implementation of the methods is nontrivial because of the positive definite constraint on the concentration matrix, but we show that the computation can be done effectively by taking advantage of the efficient maxdet algorithm developed in convex optimization. We propose a BIC-type criterion for the selection of the tuning parameter in the penalized likelihood methods. The connection between our methods and existing methods is illustrated. Simulations and real examples demonstrate the competitive performance of the new methods.

1,824 citations


Journal ArticleDOI
TL;DR: It is shown that coordinate descent is very competitive with the well-known LARS procedure in large lasso problems, can deliver a path of solutions efficiently, and can be applied to many other convex statistical problems such as the garotte and elastic net.
Abstract: We consider ``one-at-a-time'' coordinate-wise descent algorithms for a class of convex optimization problems. An algorithm of this kind has been proposed for the $L_1$-penalized regression (lasso) in the literature, but it seems to have been largely ignored. Indeed, it seems that coordinate-wise algorithms are not often used in convex optimization. We show that this algorithm is very competitive with the well-known LARS (or homotopy) procedure in large lasso problems, and that it can be applied to related methods such as the garotte and elastic net. It turns out that coordinate-wise descent does not work in the ``fused lasso,'' however, so we derive a generalized algorithm that yields the solution in much less time that a standard convex optimizer. Finally, we generalize the procedure to the two-dimensional fused lasso, and demonstrate its performance on some image smoothing problems.

1,785 citations


Journal ArticleDOI
TL;DR: In this paper, coordinate-wise descent is used to solve the L1-penalized regression problem in the fused lasso problem, which is a non-separable problem in which coordinate descent does not work.
Abstract: We consider “one-at-a-time” coordinate-wise descent algorithms for a class of convex optimization problems. An algorithm of this kind has been proposed for the L1-penalized regression (lasso) in the literature, but it seems to have been largely ignored. Indeed, it seems that coordinate-wise algorithms are not often used in convex optimization. We show that this algorithm is very competitive with the well-known LARS (or homotopy) procedure in large lasso problems, and that it can be applied to related methods such as the garotte and elastic net. It turns out that coordinate-wise descent does not work in the “fused lasso,” however, so we derive a generalized algorithm that yields the solution in much less time that a standard convex optimizer. Finally, we generalize the procedure to the two-dimensional fused lasso, and demonstrate its performance on some image smoothing problems. 1. Introduction. In this paper we consider statistical models that lead to convex optimization problems with inequality constraints. Typically, the optimization for these problems is carried out using a standard quadratic programming algorithm. The purpose of this paper is to explore “one-at-a-time” coordinate-wise descent algorithms for these problems. The equivalent of a coordinate descent algorithm has been proposed for the L1-penalized regression (lasso) in the literature, but it is not commonly used. Moreover, coordinate-wise algorithms seem too simple, and they are not often used in convex optimization, perhaps because they only work in specialized problems. We ourselves never appreciated the value of coordinate descent methods for convex statistical problems before working on this paper. In this paper we show that coordinate descent is very competitive with the wellknown LARS (or homotopy) procedure in large lasso problems, can deliver a path of solutions efficiently, and can be applied to many other convex statistical problems such as the garotte and elastic net. We then go on to explore a nonseparable problem in which coordinate-wise descent does not work—the “fused lasso.” We derive a generalized algorithm that yields the solution in much less time that a standard convex optimizer. Finally, we generalize the procedure to

1,619 citations


Posted Content
TL;DR: This paper analyzes several new methods for solving optimization problems with the objective function formed as a sum of two convex terms: one is smooth and given by a black-box oracle, and another is general but simple and its structure is known.
Abstract: In this paper we analyze several new methods for solving optimization problems with the objective function formed as a sum of two convex terms: one is smooth and given by a black-box oracle, and another is general but simple and its structure is known. Despite to the bad properties of the sum, such problems, both in convex and nonconvex cases, can be solved with eciency typical for the good part of the objective. For convex problems of the above structure, we consider primal and dual variants of the gradient method (converge as O ‡ 1 k · ), and an accelerated multistep version with convergence rate O ‡ 1 k2 · , where k is the iteration counter. For all methods, we suggest some ecient “line search” procedures and show that the additional computational work necessary for estimating the unknown problem class parameters can only multiply the complexity of each iteration by a small constant factor. We present also the results of preliminary computational experiments, which confirm the superiority of the accelerated scheme.

1,338 citations


Journal ArticleDOI
TL;DR: The problem of finding the (symmetric) edge weights that result in the least mean-square deviation in steady state is considered and it is shown that this problem can be cast as a convex optimization problem, so the global solution can be found efficiently.

1,166 citations


Journal ArticleDOI
TL;DR: Several algorithms achieving logarithmic regret are proposed, which besides being more general are also much more efficient to implement, and give rise to an efficient algorithm based on the Newton method for optimization, a new tool in the field.
Abstract: In an online convex optimization problem a decision-maker makes a sequence of decisions, i.e., chooses a sequence of points in Euclidean space, from a fixed feasible set. After each point is chosen, it encounters a sequence of (possibly unrelated) convex cost functions. Zinkevich (ICML 2003) introduced this framework, which models many natural repeated decision-making problems and generalizes many existing problems such as Prediction from Expert Advice and Cover's Universal Portfolios. Zinkevich showed that a simple online gradient descent algorithm achieves additive regret $O(\sqrt{T})$ , for an arbitrary sequence of T convex cost functions (of bounded gradients), with respect to the best single decision in hindsight. In this paper, we give algorithms that achieve regret O(log?(T)) for an arbitrary sequence of strictly convex functions (with bounded first and second derivatives). This mirrors what has been done for the special cases of prediction from expert advice by Kivinen and Warmuth (EuroCOLT 1999), and Universal Portfolios by Cover (Math. Finance 1:1---19, 1991). We propose several algorithms achieving logarithmic regret, which besides being more general are also much more efficient to implement. The main new ideas give rise to an efficient algorithm based on the Newton method for optimization, a new tool in the field. Our analysis shows a surprising connection between the natural follow-the-leader approach and the Newton method. We also analyze other algorithms, which tie together several different previous approaches including follow-the-leader, exponential weighting, Cover's algorithm and gradient descent.

1,124 citations


Journal ArticleDOI
TL;DR: This paper shows that while retaining the same simplicity, the convergence rate of I-ELM can be further improved by recalculating the output weights of the existing nodes based on a convex optimization method when a new hidden node is randomly added.

1,068 citations


Journal ArticleDOI
TL;DR: This work presents a systematic method of distributed algorithms for power control that is geometric-programming-based and shows that in the high Signal-to- interference Ratios (SIR) regime, these nonlinear and apparently difficult, nonconvex optimization problems can be transformed into convex optimized problems in the form of geometric programming.
Abstract: In wireless cellular or ad hoc networks where Quality of Service (QoS) is interference-limited, a variety of power control problems can be formulated as nonlinear optimization with a system-wide objective, e.g., maximizing the total system throughput or the worst user throughput, subject to QoS constraints from individual users, e.g., on data rate, delay, and outage probability. We show that in the high Signal-to- interference Ratios (SIR) regime, these nonlinear and apparently difficult, nonconvex optimization problems can be transformed into convex optimization problems in the form of geometric programming; hence they can be very efficiently solved for global optimality even with a large number of users. In the medium to low SIR regime, some of these constrained nonlinear optimization of power control cannot be turned into tractable convex formulations, but a heuristic can be used to compute in most cases the optimal solution by solving a series of geometric programs through the approach of successive convex approximation. While efficient and robust algorithms have been extensively studied for centralized solutions of geometric programs, distributed algorithms have not been explored before. We present a systematic method of distributed algorithms for power control that is geometric-programming-based. These techniques for power control, together with their implications to admission control and pricing in wireless networks, are illustrated through several numerical examples.

906 citations


Journal ArticleDOI
TL;DR: A path following algorithm for L1‐regularized generalized linear models that efficiently computes solutions along the entire regularization path by using the predictor–corrector method of convex optimization.
Abstract: Summary. We introduce a path following algorithm for L1-regularized generalized linear models. The L1-regularization procedure is useful especially because it, in effect, selects variables according to the amount of penalization on the L1-norm of the coefficients, in a manner that is less greedy than forward selection–backward deletion. The generalized linear model path algorithm efficiently computes solutions along the entire regularization path by using the predictor–corrector method of convex optimization. Selecting the step length of the regularization parameter is critical in controlling the overall accuracy of the paths; we suggest intuitive and flexible strategies for choosing appropriate values. We demonstrate the implementation with several simulated and real data sets.

Posted Content
TL;DR: The algorithm, Regularized Orthogonal Matching Pursuit (ROMP), seeks to provide the benefits of the two major approaches to sparse recovery, and combines the speed and ease of implementation of the greedy methods with the strong guarantees of the convex programming methods.
Abstract: We demonstrate a simple greedy algorithm that can reliably recover a d-dimensional vector v from incomplete and inaccurate measurements x. Here our measurement matrix is an N by d matrix with N much smaller than d. Our algorithm, Regularized Orthogonal Matching Pursuit (ROMP), seeks to close the gap between two major approaches to sparse recovery. It combines the speed and ease of implementation of the greedy methods with the strong guarantees of the convex programming methods. For any measurement matrix that satisfies a Uniform Uncertainty Principle, ROMP recovers a signal with O(n) nonzeros from its inaccurate measurements x in at most n iterations, where each iteration amounts to solving a Least Squares Problem. The noise level of the recovery is proportional to the norm of the error, up to a log factor. In particular, if the error vanishes the reconstruction is exact. This stability result extends naturally to the very accurate recovery of approximately sparse signals.

Journal ArticleDOI
TL;DR: In this paper, the adaptive Lasso estimator is proposed for Cox's proportional hazards model, which is based on a penalized log partial likelihood with the adaptively weighted L 1 penalty on regression coefficients.
Abstract: SUMMARY We investigate the variable selection problem for Cox's proportional hazards model, and propose a unified model selection and estimation procedure with desired theoretical properties and computational convenience. The new method is based on a penalized log partial likelihood with the adaptively weighted L1 penalty on regression coefficients, providing what we call the adaptive Lasso estimator. The method incorporates different penalties for different coefficients: unimportant variables receive larger penalties than important ones, so that important variables tend to be retained in the selection process, whereas unimportant variables are more likely to be dropped. Theoretical properties, such as consistency and rate of convergence of the estimator, are studied. We also show that, with proper choice of regularization parameters, the proposed estimator has the oracle properties. The convex optimization nature of the method leads to an efficient algorithm. Both simulated and real examples show that the method performs competitively.

Journal ArticleDOI
TL;DR: This paper presents a methodology for safety verification of continuous and hybrid systems in the worst-case and stochastic settings, and computes an upper bound on the probability that a trajectory of the system reaches the unsafe set, a bound whose validity is proven by the existence of a barrier certificate.
Abstract: This paper presents a methodology for safety verification of continuous and hybrid systems in the worst-case and stochastic settings. In the worst-case setting, a function of state termed barrier certificate is used to certify that all trajectories of the system starting from a given initial set do not enter an unsafe region. No explicit computation of reachable sets is required in the construction of barrier certificates, which makes it possible to handle nonlinearity, uncertainty, and constraints directly within this framework. In the stochastic setting, our method computes an upper bound on the probability that a trajectory of the system reaches the unsafe set, a bound whose validity is proven by the existence of a barrier certificate. For polynomial systems, barrier certificates can be constructed using convex optimization, and hence the method is computationally tractable. Some examples are provided to illustrate the use of the method.

Journal ArticleDOI
TL;DR: A decomposition method based on the Douglas-Rachford algorithm for monotone operator-splitting for signal recovery problems and applications to non-Gaussian image denoising in a tight frame are demonstrated.
Abstract: Under consideration is the large body of signal recovery problems that can be formulated as the problem of minimizing the sum of two (not necessarily smooth) lower semicontinuous convex functions in a real Hilbert space. This generic problem is analyzed and a decomposition method is proposed to solve it. The convergence of the method, which is based on the Douglas-Rachford algorithm for monotone operator-splitting, is obtained under general conditions. Applications to non-Gaussian image denoising in a tight frame are also demonstrated.

Journal ArticleDOI
TL;DR: This work presents a convex programming algorithm for the numerical solution of the minimum fuel powered descent guidance problem associated with Mars pinpoint landing as a finite-dimensional convex optimization problem as a second-order cone programming problem.
Abstract: We present a convex programming algorithm for the numerical solution of the minimum fuel powered descent guidance problem associated with Mars pinpoint landing. Our main contribution is the formulation of the trajectory optimization problem, which has nonconvex control constraints, as a finite-dimensional convex optimization problem, specifically as a second-order cone programming problem. Second-order cone programming is a subclass of convex programming, and there are efficient second-order cone programming solvers with deterministic convergence properties. Consequently, the resulting guidance algorithm can potentially be implemented onboard a spacecraft for real-time applications.

Proceedings Article
11 Mar 2007
TL;DR: Stochastic variants of the wellknown BFGS quasi-Newton optimization method, in both full and memory-limited (LBFGS) forms, are developed for online optimization of convex functions, which asymptotically outperforms previous stochastic gradient methods for parameter estimation in conditional random fields.
Abstract: We develop stochastic variants of the wellknown BFGS quasi-Newton optimization method, in both full and memory-limited (LBFGS) forms, for online optimization of convex functions. The resulting algorithm performs comparably to a well-tuned natural gradient descent but is scalable to very high-dimensional problems. On standard benchmarks in natural language processing, it asymptotically outperforms previous stochastic gradient methods for parameter estimation in conditional random fields. We are working on analyzing the convergence of online (L)BFGS, and extending it to nonconvex optimization problems.

Journal ArticleDOI
TL;DR: This dynamic program is based on a rigorous extension of secondary structure models to the multistranded case, addressing representation and distinguishability issues that do not arise for single-stranded structures.
Abstract: Motivated by the analysis of natural and engineered DNA and RNA systems, we present the first algorithm for calculating the partition function of an unpseudoknotted complex of multiple interacting nucleic acid strands This dynamic program is based on a rigorous extension of secondary structure models to the multistranded case, addressing representation and distinguishability issues that do not arise for single-stranded structures We then derive the form of the partition function for a fixed volume containing a dilute solution of nucleic acid complexes This expression can be evaluated explicitly for small numbers of strands, allowing the calculation of the equilibrium population distribution for each species of complex Alternatively, for large systems (eg, a test tube), we show that the unique complex concentrations corresponding to thermodynamic equilibrium can be obtained by solving a convex programming problem Partition function and concentration information can then be used to calculate equilibrium base-pairing observables The underlying physics and mathematical formulation of these problems lead to an interesting blend of approaches, including ideas from graph theory, group theory, dynamic programming, combinatorics, convex optimization, and Lagrange duality

Proceedings ArticleDOI
20 Jun 2007
TL;DR: This paper suggests a method for multiclass learning with many classes by simultaneously learning shared characteristics common to the classes, and predictors for the classes in terms of these characteristics.
Abstract: This paper suggests a method for multiclass learning with many classes by simultaneously learning shared characteristics common to the classes, and predictors for the classes in terms of these characteristics. We cast this as a convex optimization problem, using trace-norm regularization and study gradient-based optimization both for the linear case and the kernelized setting.

Journal ArticleDOI
TL;DR: A relaxation method is described which yields an easily computable upper bound on the optimal solution of portfolio selection, and a heuristic method for finding a suboptimal portfolio which is based on solving a small number of convex optimization problems.
Abstract: We consider the problem of portfolio selection, with transaction costs and constraints on exposure to risk. Linear transaction costs, bounds on the variance of the return, and bounds on different shortfall probabilities are efficiently handled by convex optimization methods. For such problems, the globally optimal portfolio can be computed very rapidly. Portfolio optimization problems with transaction costs that include a fixed fee, or discount breakpoints, cannot be directly solved by convex optimization. We describe a relaxation method which yields an easily computable upper bound via convex optimization. We also describe a heuristic method for finding a suboptimal portfolio, which is based on solving a small number of convex optimization problems (and hence can be done efficiently). Thus, we produce a suboptimal solution, and also an upper bound on the optimal solution. Numerical experiments suggest that for practical problems the gap between the two is small, even for large problems involving hundreds of assets. The same approach can be used for related problems, such as that of tracking an index with a portfolio consisting of a small number of assets.

Journal ArticleDOI
TL;DR: This paper overviews several selected topics in this popular area, specifically, recent extensions of the basic concept of robust counterpart of an optimization problem with uncertain data, tractability of robust counterparts, links between RO and traditional chance constrained settings of problems with stochastic data, and a novel generic application of the RO methodology in Robust Linear Control.
Abstract: Robust Optimization is a rapidly developing methodology for handling optimization problems affected by non-stochastic “uncertain-but- bounded” data perturbations. In this paper, we overview several selected topics in this popular area, specifically, (1) recent extensions of the basic concept of robust counterpart of an optimization problem with uncertain data, (2) tractability of robust counterparts, (3) links between RO and traditional chance constrained settings of problems with stochastic data, and (4) a novel generic application of the RO methodology in Robust Linear Control.

Proceedings Article
03 Dec 2007
TL;DR: This paper furnishes an alternative means of expressing the ARD cost function using auxiliary functions that naturally addresses both of these issues and suggest alternative cost functions and update procedures for selecting features and promoting sparse solutions in a variety of general situations.
Abstract: Automatic relevance determination (ARD) and the closely-related sparse Bayesian learning (SBL) framework are effective tools for pruning large numbers of irrelevant features leading to a sparse explanatory subset. However, popular update rules used for ARD are either difficult to extend to more general problems of interest or are characterized by non-ideal convergence properties. Moreover, it remains unclear exactly how ARD relates to more traditional MAP estimation-based methods for learning sparse representations (e.g., the Lasso). This paper furnishes an alternative means of expressing the ARD cost function using auxiliary functions that naturally addresses both of these issues. First, the proposed reformulation of ARD can naturally be optimized by solving a series of re-weighted l1 problems. The result is an efficient, extensible algorithm that can be implemented using standard convex programming toolboxes and is guaranteed to converge to a local minimum (or saddle point). Secondly, the analysis reveals that ARD is exactly equivalent to performing standard MAP estimation in weight space using a particular feature- and noise-dependent, non-factorial weight prior. We then demonstrate that this implicit prior maintains several desirable advantages over conventional priors with respect to feature selection. Overall these results suggest alternative cost functions and update procedures for selecting features and promoting sparse solutions in a variety of general situations. In particular, the methodology readily extends to handle problems such as non-negative sparse coding and covariance component estimation.

Journal ArticleDOI
TL;DR: Investigation of the relevancy of SVR to superresolution proceeds with the possibility of using a single and general support vector regression for all image content, and the results are impressive for small training sets.
Abstract: A thorough investigation of the application of support vector regression (SVR) to the superresolution problem is conducted through various frameworks. Prior to the study, the SVR problem is enhanced by finding the optimal kernel. This is done by formulating the kernel learning problem in SVR form as a convex optimization problem, specifically a semi-definite programming (SDP) problem. An additional constraint is added to reduce the SDP to a quadratically constrained quadratic programming (QCQP) problem. After this optimization, investigation of the relevancy of SVR to superresolution proceeds with the possibility of using a single and general support vector regression for all image content, and the results are impressive for small training sets. This idea is improved upon by observing structural properties in the discrete cosine transform (DCT) domain to aid in learning the regression. Further improvement involves a combination of classification and SVR-based techniques, extending works in resolution synthesis. This method, termed kernel resolution synthesis, uses specific regressors for isolated image content to describe the domain through a partitioned look of the vector space, thereby yielding good results

Journal ArticleDOI
TL;DR: Two procedures for designing state-feedback control laws are given: one casts the controller design into a convex optimization by introducing some over design and the other utilizes the cone complementarity linearization idea to cast the controllerDesign into a sequential minimization problem subject to linear matrix inequality constraints, which can be readily solved using standard numerical software.
Abstract: This paper investigates the problem of stabilization for a Takagi-Sugeno (T-S) fuzzy system with nonuniform uncertain sampling. The sampling is not required to be periodic, and the only assumption is that the distance between any two consecutive sampling instants is less than a given bound. By using the input delay approach, the T-S fuzzy system with variable uncertain sampling is transformed into a continuous-time T-S fuzzy system with a delay in the state. Though the resulting closed-loop state-delayed T-S fuzzy system takes a standard form, the existing results on delay T-S fuzzy systems cannot be used for our purpose due to their restrictive assumptions on the derivative of state delay. A new condition guaranteeing asymptotic stability of the closed-loop sampled-data system is derived by a Lyapunov approach plus the free weighting matrix technique. Based on this stability condition, two procedures for designing state-feedback control laws are given: one casts the controller design into a convex optimization by introducing some over design and the other utilizes the cone complementarity linearization idea to cast the controller design into a sequential minimization problem subject to linear matrix inequality constraints, which can be readily solved using standard numerical software. An illustrative example is provided to show the applicability and effectiveness of the proposed controller design methodology.

Journal ArticleDOI
TL;DR: An accelerated version of the cubic regularization of Newton’s method that converges for the same problem class with order, keeping the complexity of each iteration unchanged and arguing that for the second-order schemes, the class of non-degenerate problems is different from the standard class.
Abstract: In this paper we propose an accelerated version of the cubic regularization of Newton’s method (Nesterov and Polyak, in Math Program 108(1): 177–205, 2006). The original version, used for minimizing a convex function with Lipschitz-continuous Hessian, guarantees a global rate of convergence of order $$O\big({1 \over k^2}\big)$$, where k is the iteration counter. Our modified version converges for the same problem class with order $$O\big({1 \over k^3}\big)$$, keeping the complexity of each iteration unchanged. We study the complexity of both schemes on different classes of convex problems. In particular, we argue that for the second-order schemes, the class of non-degenerate problems is different from the standard class.

Journal ArticleDOI
TL;DR: This work addresses the problem of minimum mean square error (MMSE) transceiver design for point-to-multipoint transmission in multiuser multiple-input-multiple-output (MIMO) systems and proposes two globally optimum algorithms based on convex optimization.
Abstract: We address the problem of minimum mean square error (MMSE) transceiver design for point-to-multipoint transmission in multiuser multiple-input-multiple-output (MIMO) systems. We focus on the problem of minimizing the downlink sum-MSE under a sum power constraint. It is shown that this problem can be solved efficiently by exploiting a duality between the downlink and uplink MSE feasible regions. We propose two different optimization frameworks for downlink MMSE transceiver design. The first one solves an equivalent uplink problem, then the result is transferred to the original downlink problem. Duality ensures that any uplink MMSE scheme, e.g., linear MMSE reception or MMSE-decision feedback equalization (DFE), has a downlink counterpart. We propose two globally optimum algorithms based on convex optimization. The basic idea of the second framework is to perform optimization in an alternating manner by switching between the virtual uplink and downlink channels. This strategy exploits that the same MSE can be achieved in both links for a given choice of transmit and receive filters. This iteration is proven to be convergent.

Proceedings Article
03 Dec 2007
TL;DR: An algorithm is provided, Adaptive Online Gradient Descent, which interpolates between the results of Zinkevich for linear functions and of Hazan et al for strongly convex functions, achieving intermediate rates between √T and log T and shows strong optimality of the algorithm.
Abstract: We study the rates of growth of the regret in online convex optimization. First, we show that a simple extension of the algorithm of Hazan et al eliminates the need for a priori knowledge of the lower bound on the second derivatives of the observed functions. We then provide an algorithm, Adaptive Online Gradient Descent, which interpolates between the results of Zinkevich for linear functions and of Hazan et al for strongly convex functions, achieving intermediate rates between √T and log T. Furthermore, we show strong optimality of the algorithm. Finally, we provide an extension of our results to general norms.

Journal ArticleDOI
TL;DR: This work proposes a versatile convex variational formulation for optimization over orthonormal bases that covers a wide range of problems, and establishes the strong convergence of a proximal thresholding algorithm to solve it.
Abstract: The notion of soft thresholding plays a central role in problems from various areas of applied mathematics, in which the ideal solution is known to possess a sparse decomposition in some orthonormal basis. Using convex-analytical tools, we extend this notion to that of proximal thresholding and investigate its properties, providing, in particular, several characterizations of such thresholders. We then propose a versatile convex variational formulation for optimization over orthonormal bases that covers a wide range of problems, and we establish the strong convergence of a proximal thresholding algorithm to solve it. Numerical applications to signal recovery are demonstrated.

01 Jan 2007
TL;DR: In this paper, the developing theory of geometric random walks is outlined and general methods for estimating convergence (the mixinging rate), isoperimetric inequalities in R and their intimate connection to random walks are discussed.
Abstract: The developing theory of geometric random walks is outlined here. Three aspects—general methods for estimating convergence (the “mixing” rate), isoperimetric inequalities in R and their intimate connection to random walks, and algorithms for fundamental problems (volume computation and convex optimization) that are based on sampling by random walks—are discussed.

Journal ArticleDOI
TL;DR: The simulation results indicate that in the presence of uncertain CSI the proposed approaches can satisfy the users' QoS requirements for a significantly larger set of uncertainties than existing methods, and require less transmission power to do so.
Abstract: We consider the design of linear precoders (beamformers) for broadcast channels with Quality of Service (QoS) constraints for each user, in scenarios with uncertain channel state information (CSI) at the transmitter. We consider a deterministically-bounded model for the channel uncertainty of each user, and our goal is to design a robust precoder that minimizes the total transmission power required to satisfy the users' QoS constraints for all channels within a specified uncertainty region around the transmitter's estimate of each user's channel. Since this problem is not known to be computationally tractable, we will derive three conservative design approaches that yield convex and computationally-efficient restrictions of the original design problem. The three approaches yield semidefinite program (SDP) formulations that offer different trade-offs between the degree of conservatism and the size of the SDP. We will also show how these conservative approaches can be used to derive efficiently-solvable quasi-convex restrictions of some related design problems, including the robust counterpart to the problem of maximizing the minimum signal-to-interference-plus-noise-ratio (SINR) subject to a given power constraint. Our simulation results indicate that in the presence of uncertain CSI the proposed approaches can satisfy the users' QoS requirements for a significantly larger set of uncertainties than existing methods, and require less transmission power to do so.