Showing papers on "Concave function published in 2019"

A refined convergence analysis of $$\hbox {pDCA}_{e}$$ pDCA e with applications to simultaneous sparse recovery and outlier detection

Abstract: We consider the problem of minimizing a difference-of-convex (DC) function, which can be written as the sum of a smooth convex function with Lipschitz gradient, a proper closed convex function and a continuous possibly nonsmooth concave function. We refine the convergence analysis in Wen et al. (Comput Optim Appl 69, 297–324, 2018) for the proximal DC algorithm with extrapolation ( $$\hbox {pDCA}_e$$ ) and show that the whole sequence generated by the algorithm is convergent without imposing differentiability assumptions in the concave part. Our analysis is based on a new potential function and we assume such a function is a Kurdyka–Łojasiewicz (KL) function. We also establish a relationship between our KL assumption and the one used in Wen et al. (2018). Finally, we demonstrate how the $$\hbox {pDCA}_e$$ can be applied to a class of simultaneous sparse recovery and outlier detection problems arising from robust compressed sensing in signal processing and least trimmed squares regression in statistics. Specifically, we show that the objectives of these problems can be written as level-bounded DC functions whose concave parts are typically nonsmooth. Moreover, for a large class of loss functions and regularizers, the KL exponent of the corresponding potential function are shown to be 1/2, which implies that the $$\hbox {pDCA}_e$$ is locally linearly convergent when applied to these problems. Our numerical experiments show that the $$\hbox {pDCA}_e$$ usually outperforms the proximal DC algorithm with nonmonotone linesearch (Liu et al. in Math Program, 2018. https://doi.org/10.1007/s10107-018-1327-8 , Appendix A) in both CPU time and solution quality for this particular application.

Capacity Upper Bounds for Deletion-type Channels

Abstract: We develop a systematic approach, based on convex programming and real analysis for obtaining upper bounds on the capacity of the binary deletion channel and, more generally, channels with i.i.d. insertions and deletions. Other than the classical deletion channel, we give special attention to the Poisson-repeat channel introduced by Mitzenmacher and Drinea (IEEE Transactions on Information Theory, 2006). Our framework can be applied to obtain capacity upper bounds for any repetition distribution (the deletion and Poisson-repeat channels corresponding to the special cases of Bernoulli and Poisson distributions). Our techniques essentially reduce the task of proving capacity upper bounds to maximizing a univariate, real-valued, and often concave function over a bounded interval. The corresponding univariate function is carefully designed according to the underlying distribution of repetitions, and the choices vary depending on the desired strength of the upper bounds as well as the desired simplicity of the function (e.g., being only efficiently computable versus having an explicit closed-form expression in terms of elementary, or common special, functions). Among our results, we show the following: (1) The capacity of the binary deletion channel with deletion probability d is at most (1 − d) φ for d ≥ 1/2 and, assuming that the capacity function is convex, is at most 1 − d log(4/φ) for d d outside the limiting case d → 0 that is fully explicit and proved without computer assistance. (2) We derive the first set of capacity upper bounds for the Poisson-repeat channel. Our results uncover further striking connections between this channel and the deletion channel and suggest, somewhat counter-intuitively, that the Poisson-repeat channel is actually analytically simpler than the deletion channel and may be of key importance to a complete understanding of the deletion channel. (3) We derive several novel upper bounds on the capacity of the deletion channel. All upper bounds are maximums of efficiently computable, and concave, univariate real functions over a bounded domain. In turn, we upper bound these functions in terms of explicit elementary and standard special functions, whose maximums can be found even more efficiently (and sometimes analytically, for example, for d = 1/2).

Stochastic Primal-Dual Algorithms with Faster Convergence than O(1/√T) for Problems without Bilinear Structure.

Abstract: Previous studies on stochastic primal-dual algorithms for solving min-max problems with faster convergence heavily rely on the bilinear structure of the problem, which restricts their applicability to a narrowed range of problems. The main contribution of this paper is the design and analysis of new stochastic primal-dual algorithms that use a mixture of stochastic gradient updates and a logarithmic number of deterministic dual updates for solving a family of convex-concave problems with no bilinear structure assumed. Faster convergence rates than $O(1/\sqrt{T})$ with $T$ being the number of stochastic gradient updates are established under some mild conditions of involved functions on the primal and the dual variable. For example, for a family of problems that enjoy a weak strong convexity in terms of the primal variable and has a strongly concave function of the dual variable, the convergence rate of the proposed algorithm is $O(1/T)$. We also investigate the effectiveness of the proposed algorithms for learning robust models and empirical AUC maximization.

Analysis of a Two-Layer Neural Network via Displacement Convexity

Abstract: Fitting a function by using linear combinations of a large number $N$ of `simple' components is one of the most fruitful ideas in statistical learning. This idea lies at the core of a variety of methods, from two-layer neural networks to kernel regression, to boosting. In general, the resulting risk minimization problem is non-convex and is solved by gradient descent or its variants. Unfortunately, little is known about global convergence properties of these approaches. Here we consider the problem of learning a concave function $f$ on a compact convex domain $\Omega\subseteq {\mathbb R}^d$, using linear combinations of `bump-like' components (neurons). The parameters to be fitted are the centers of $N$ bumps, and the resulting empirical risk minimization problem is highly non-convex. We prove that, in the limit in which the number of neurons diverges, the evolution of gradient descent converges to a Wasserstein gradient flow in the space of probability distributions over $\Omega$. Further, when the bump width $\delta$ tends to $0$, this gradient flow has a limit which is a viscous porous medium equation. Remarkably, the cost function optimized by this gradient flow exhibits a special property known as displacement convexity, which implies exponential convergence rates for $N\to\infty$, $\delta\to 0$. Surprisingly, this asymptotic theory appears to capture well the behavior for moderate values of $\delta, N$. Explaining this phenomenon, and understanding the dependence on $\delta,N$ in a quantitative manner remains an outstanding challenge.

A general double-proximal gradient algorithm for d.c. programming

Abstract: The possibilities of exploiting the special structure of d.c. programs, which consist of optimising the difference of convex functions, are currently more or less limited to variants of the DCA proposed by Pham Dinh Tao and Le Thi Hoai An in 1997. These assume that either the convex or the concave part, or both, are evaluated by one of their subgradients. In this paper we propose an algorithm which allows the evaluation of both the concave and the convex part by their proximal points. Additionally, we allow a smooth part, which is evaluated via its gradient. In the spirit of primal-dual splitting algorithms, the concave part might be the composition of a concave function with a linear operator, which are, however, evaluated separately. For this algorithm we show that every cluster point is a solution of the optimisation problem. Furthermore, we show the connection to the Toland dual problem and prove a descent property for the objective function values of a primal-dual formulation of the problem. Convergence of the iterates is shown if this objective function satisfies the Kurdyka-Łojasiewicz property. In the last part, we apply the algorithm to an image processing model.

One-Bit Compressive Sensing via Schur-Concave Function Minimization

Abstract: Much effort has been devoted to recovering sparse signals from one-bit measurements in recent years. However, it is still quite challenging to recover signals with high fidelity, which is desired in practical one-bit compressive sensing (1-bit CS) applications. We introduce the notion of Schur-concavity in this paper and propose to construct signals by taking advantage of Schur-Concave functions , which are capable of enhancing sparsity. Specifically, the Schur-concave functions can be employed to measure the degree of concentration, and the sparse solutions are obtained at the minima. As a representative of the Schur-concave family, the normalized $\ell _1$ Shannon entropy function ( $\ell _1$ -SEF) is exploited. The resulting optimization problem is nonconvex. Hence, we convert it into a series of weighted ${\ell _1}$ -norm subproblems, which are solved iteratively by a generalized fixed-point continuation algorithm. Numerical results are provided to illustrate the effectiveness and superiority of the proposed 1-bit CS algorithm.

A reverse Rogers-Shephard inequality for log-concave functions

Abstract: We will prove a reverse Rogers–Shephard inequality for log-concave functions. In some particular cases, the method used for general log-concave functions can be slightly improved, allowing us to prove volume estimates for polars of $$\ell _p$$ -diferences of convex bodies under the condition that their polar bodies have opposite barycenters.

A Generalized Direction in Interior Point Method for Monotone Linear Complementarity Problems

Abstract: In this paper, we present a new interior point method with full Newton step for monotone linear complementarity problems. The specificity of our method is to compute the Newton step using a modified system similar to that introduced by Darvay in 2003. We prove that this new method possesses the best known upper bound complexity for these methods. Moreover, we extend results known in the literature since we consider a general family of smooth concave functions in the Newton system instead of the square root. Some computational results are included to illustrate the validity of the proposed algorithm.

Quantitative combinatorial geometry for concave functions

Abstract: We prove several exact quantitative versions of Helly's and Tverberg's theorems, which guarantee that a finite family of convex sets in $R^d$ has a large intersection. Our results characterize conditions that are sufficient for the intersection of a family of convex sets to contain a "witness set" which is large under some concave or log-concave measure. The possible witness sets include ellipsoids, zonotopes, and $H$-convex sets. Our results also bound the complexity of finding the best approximation of a family of convex sets by a single zonotope or by a single $H$-convex set. We obtain colorful and fractional variants of all our Helly-type theorems.

A unified DC programming framework and efficient DCA based approaches for large scale batch reinforcement learning

Abstract: We investigate a powerful nonconvex optimization approach based on Difference of Convex functions (DC) programming and DC Algorithm (DCA) for reinforcement learning, a general class of machine learning techniques which aims to estimate the optimal learning policy in a dynamic environment typically formulated as a Markov decision process (with an incomplete model). The problem is tackled as finding the zero of the so-called optimal Bellman residual via the linear value-function approximation for which two optimization models are proposed: minimizing the $$\ell _{p}$$ -norm of a vector-valued convex function, and minimizing a concave function under linear constraints. They are all formulated as DC programs for which attractive DCA schemes are developed. Numerical experiments on various examples of the two benchmarks of Markov decision process problems—Garnet and Gridworld problems, show the efficiency of our approaches in comparison with two existing DCA based algorithms and two state-of-the-art reinforcement learning algorithms.

Subdeterminants and Concave Integer Quadratic Programming

Abstract: We consider the NP-hard problem of minimizing a separable concave quadratic function over the integral points in a polyhedron, and we denote by $\Delta$ the largest absolute value of the subdetermi...

A Concave Value Function Extension for the Dynamic Programming Approach to Revenue Management in Attended Home Delivery

Abstract: We study the approximate dynamic programming approach to revenue management in the context of attended home delivery. We draw on results from dynamic programming theory for Markov decision problems, convex optimisation and discrete convex analysis to show that the underlying dynamic programming operator has a unique fixed point. Moreover, we also show that -- under certain assumptions -- for all time steps in the dynamic program, the value function admits a continuous extension, which is a finite-valued, concave function of its state variables. This result opens the road for achieving scalable implementations of the proposed formulation, as it allows making informed choices of basis functions in an approximate dynamic programming context. We illustrate our findings using a simple numerical example and conclude with suggestions on how our results can be exploited in future work to obtain closer approximations of the value function.

An extension of Petrović's inequality for \(h-\)convex (\(h-\)concave) functions in plane

Abstract: In this paper, Petrović’s inequality is generalized for h−convex functions on coordinates with the condition that h is supermultiplicative. In the case, when h is submultiplicative, Petrović’s inequality is generalized for h−concave functions. Also particular cases for P−function, Godunova-Levin functions, s−Godunova-Levin functions and s−convex functions has been discussed.

A Concave Value Function Extension for the Dynamic Programming Approach to Revenue Management in Attended Home Delivery

Abstract: We study the approximate dynamic programming approach to revenue management in the context of attended home delivery. We draw on results from dynamic programming theory for Markov decision problems, convex optimisation and discrete convex analysis to show that the underlying dynamic programming operator has a unique fixed point. Moreover, we also show that - under certain assumptions - for all time steps in the dynamic program, the value function admits a continuous extension, which is a finite-valued, concave function of its state variables. This result opens the road for achieving scalable implementations of the proposed formulation, as it allows making informed choices of basis functions in an approximate dynamic programming context. We illustrate our findings using a simple numerical example and conclude with suggestions on how our results can be exploited in future work to obtain closer approximations of the value function.

On the convex Poincaré inequality and weak transportation inequalities

Abstract: We prove that for a probability measure on $\mathbb{R}^{n}$, the Poincare inequality for convex functions is equivalent to the weak transportation inequality with a quadratic-linear cost. This generalizes recent results by Gozlan, Roberto, Samson, Shu, Tetali and Feldheim, Marsiglietti, Nayar, Wang, concerning probability measures on the real line. The proof relies on modified logarithmic Sobolev inequalities of Bobkov–Ledoux type for convex and concave functions, which are of independent interest. We also present refined concentration inequalities for general (not necessarily Lipschitz) convex functions, complementing recent results by Bobkov, Nayar, and Tetali.

Concave elliptic equations and generalized Khovanskii-Teissier inequalities

Abstract: We explain a general construction through which concave elliptic operators on complex manifolds give rise to concave functions on cohomology. In particular, this leads to generalized versions of the Khovanskii-Teissier inequalities.

(m1,m2)-Convexity and Some New Hermite-Hadamard Type Inequalities

Abstract: In this manuscript, a new class of extended (m1,m2)-convex and concave functions is introduced. After some properties of (m1,m2)-convex functions have been given, the inequalities obtained with Holder and Holder-Iscan and power-mean and improwed power-mean integral inequalities have been compared and it has been shown that the inequality with Holder-Iscan inequality gives a better approach than with Holder integral inequality and improwed power-mean inequality gives a better approach than with power-mean inequality.

The Dyck bound in the concave 1-dimensional random assignment model

Abstract: We consider models of assignment for random $N$ blue points and $N$ red points on an interval of length $2N$, in which the cost for connecting a blue point in $x$ to a red point in $y$ is the concave function $|x-y|^p$, for $0 1$, where the optimal matching is trivially determined, here the optimization is non-trivial. The purpose of this paper is to introduce a special configuration, that we call the \emph{Dyck matching}, and to study its statistical properties. We compute exactly the average cost, in the asymptotic limit of large $N$, together with the first subleading correction. The scaling is remarkable: it is of order $N$ for $p \frac{1}{2}$, and it is universal for a wide class of models. We conjecture that the average cost of the Dyck matching has the same scaling in $N$ as the cost of the optimal matching, and we produce numerical data in support of this conjecture. We hope to produce a proof of this claim in future work.

Statistical Approach to Identify Threshold and Point of Departure in Dose–Response Data

Abstract: The study presents an integrated, rigorous statistical approach to define the likelihood of a threshold and point of departure (POD) based on dose-response data using nested family of bent-hyperbola models. The family includes four models: the full bent-hyperbola model, which allows for transition between two linear regiments with various levels of smoothness; a bent-hyperbola model reduced to a spline model, where the transition is fixed to a knot; a bent-hyperbola model with a restricted negative asymptote slope of zero, named hockey-stick with arc (HS-Arc); and spline model reduced further to a hockey-stick type model (HS), where the first linear segment has a slope of zero. A likelihood-ratio test is used to discriminate between the models and determine if the more flexible versions of the model provide better or significantly better fit than a hockey-stick type model. The full bent-hyperbola model can accommodate both threshold and nonthreshold behavior, can take on concave up and concave down shapes with various levels of curvature, can approximate the biochemically relevant Michaelis-Menten model, and even be reduced to a straight line. Therefore, with the use of this model, the presence or absence of a threshold may even become irrelevant and the best fit of the full bent-hyperbola model be used to characterize the dose-response behavior and risk levels, with no need for mode of action (MOA) information. Point of departure (POD), characterized by exposure level at which some predetermined response is reached, can be defined using the full model or one of the better fitting reduced models.

Dynamic Programming for Optimal Delivery Time Slot Pricing

Abstract: We study the dynamic programming approach to revenue management in the context of attended home delivery. We draw on results from dynamic programming theory for Markov decision problems to show that the underlying Bellman operator has a unique fixed point. We then provide a closed-form expression for the resulting fixed point and show that it admits a natural interpretation. Moreover, we also show that -- under certain technical assumptions -- the value function, which has a discrete domain and a continuous codomain, admits a continuous extension, which is a finite-valued, concave function of its state variables, at every time step. This result opens the road for achieving scalable implementations of the proposed formulation in future work, as it allows making informed choices of basis functions in an approximate dynamic programming context. We illustrate our findings on a simple numerical example and provide suggestions on how our results can be exploited to obtain closer approximations of the exact value function.

Convex analysis on polyhedral spaces

Abstract: We introduce notions of concavity for functions on balanced polyhedral spaces, and we show that concave functions on such spaces satisfy several strong continuity properties.

End-to-End Distributed Flow Control for Networks with Nonconcave Utilities

Abstract: This paper proposes near-optimal decentralized allocation for traffic generated by real-time applications in communication networks. The quality of experience perceived by users in practical applications cannot be accurately modeled using concave functions. Therefore, we tackle the problem of optimizing general nonconcave network utilities. The approach for solving the resulting nonconvex network utility maximization problem relies on designing a sequence of convex relaxations whose solutions converge to a point that characterizes an optimal solution of the original problem. Three different algorithms are designed for solving the proposed convex relaxation, and their theoretical convergence guarantees are studied. All proposed algorithms are distributed in nature, where each user independently controls its traffic in a way that drives the overall network traffic allocation to an optimal operating point subject to resource constraints. All computations required by the algorithms are performed independently and locally at each user using local information available to that user. We highlight the tradeoff between the convergence speed and the network overhead required by each algorithm. Furthermore, we demonstrate the robustness and scalability of these algorithms by showing that traffic is automatically rerouted in case of a link failure or having new users joining the network. Numerical results are presented to validate our findings.

Network Utility Maximization under Maximum Delay Constraints and Throughput Requirements

Abstract: We consider a multiple-unicast network flow problem of maximizing aggregate user utilities under link capacity constraints, maximum delay constraints, and user throughput requirements. A user's utility is a concave function of the achieved throughput or the experienced maximum delay. We first prove that it is NP-complete either (i) to construct a feasible solution meeting all constraints, or (ii) to obtain an optimal solution after we relax maximum delay constraints or throughput requirements. We then leverage a novel understanding between nonconvex maximum-delay-aware problems and their convex average-delay-aware counterparts, and design a polynomial-time approximation algorithm named PASS. PASS achieves constant or problem-dependent approximation ratios, at the cost of violating maximum delay constraints or throughput requirements by up to constant or problem-dependent ratios, under realistic conditions. We empirically evaluate our solutions using simulations of supporting video-conferencing traffic across Amazon EC2 datacenters. Compared to conceivable baselines, PASS obtains up to 100% improvement of utilities, meeting throughput requirements but relaxing maximum delay constraints that are acceptable for video conferencing applications.

Some remarks on Petty projection of log-concave functions

Abstract: In this note we study the Petty projection of a log-concave function, which has been recently introduced in [9]. Moreover, we present some new inequalities involving this new notion, partly complementing and correcting some results from [9].

The Loewner function of a log-concave function

Abstract: We introduce the notion of Loewner (ellipsoid) function for a log concave function and show that it is an extension of the Loewner ellipsoid for convex bodies. We investigate its duality relation to the recently defined John (ellipsoid) function by Alonso-Gutierrez, Merino, Jimenez and Villa. For convex bodies, John and Loewner ellipsoids are dual to each other. Interestingly, this need not be the case for the John function and the Loewner function.

Random concave functions

Abstract: Spaces of convex and concave functions appear naturally in theory and applications. For example, convex regression and log-concave density estimation are important topics in nonparametric statistics. In stochastic portfolio theory, concave functions on the unit simplex measure the concentration of capital, and their gradient maps define novel investment strategies. The gradient maps may also be regarded as optimal transport maps on the simplex. In this paper we construct and study probability measures supported on spaces of concave functions. These measures may serve as prior distributions in Bayesian statistics and Cover's universal portfolio, and induce distribution-valued random variables via optimal transport. The random concave functions are constructed on the unit simplex by taking a suitably scaled (mollified, or soft) minimum of random hyperplanes. Depending on the regime of the parameters, we show that as the number of hyperplanes tends to infinity there are several possible limiting behaviors. In particular, there is a transition from a deterministic almost sure limit to a non-trivial limiting distribution that can be characterized using convex duality and Poisson point processes.