Mathematical programming

Variable importance (VI) tools describe how much covariates contribute to a prediction model's accuracy. However, important variables for one well-performing model (for example, a linear model $f(\mathbf{x})=\mathbf{x}^{T}\beta$ with a fixed coefficient vector $\beta$) may be unimportant for another model. In this paper, we propose model class reliance (MCR) as the range of VI values across all well-performing model in a prespecified class. Thus, MCR gives a more comprehensive description of importance by accounting for the fact that many prediction models, possibly of different parametric forms, may fit the data well. In the process of deriving MCR, we show several informative results for permutation-based VI estimates, based on the VI measures used in Random Forests. Specifically, we derive connections between permutation importance estimates for a single prediction model, U-statistics, conditional variable importance, conditional causal effects, and linear model coefficients. We then give probabilistic bounds for MCR, using a novel, generalizable technique. We apply MCR to a public data set of Broward County criminal records to study the reliance of recidivism prediction models on sex and race. In this application, MCR can be used to help inform VI for unknown, proprietary models.

All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously

Variable importance (VI) tools describe how much covariates contribute to a prediction model's accuracy. However, important variables for one well-performing model (for example, a linear model f (x) = x T β with a fixed coefficient vector β) may be unimportant for another model. In this paper, we propose model class reliance (MCR) as the range of VI values across all well-performing model in a prespecified class. Thus, MCR gives a more comprehensive description of importance by accounting for the fact that many prediction models, possibly of different parametric forms, may fit the data well. In the process of deriving MCR, we show several informative results for permutation-based VI estimates, based on the VI measures used in Random Forests. Specifically, we derive connections between permutation importance estimates for a single prediction model, U-statistics, conditional variable importance, conditional causal effects, and linear model coefficients. We then give probabilistic bounds for MCR, using a novel, generalizable technique. We apply MCR to a public data set of Broward County criminal records to study the reliance of recidivism prediction models on sex and race. In this application, MCR can be used to help inform VI for unknown, proprietary models.

/pdf/all-models-are-wrong-but-many-are-useful-learning-a-variable-25wmw1yyng.pdf

We introduce the Suggest-and-Improve framework for general nonconvex quadratically constrained quadratic programs (QCQPs). Using this framework, we generalize a number of known methods and provide heuristics to get approximate solutions to QCQPs for which no specialized methods are available. We also introduce an open-source Python package QCQP, which implements the heuristics discussed in the paper.

General Heuristics for Nonconvex Quadratically Constrained Quadratic Programming

The deployment of cloud and edge computing forms a three-tier mobile computing network, where each task can be processed locally, by the edge nodes, or by the remote cloud server. In this paper, we consider a cooperative three-tier computing network by leveraging the vertical cooperation among devices, edge nodes and cloud servers, as well as the horizontal cooperation between edge nodes. In this network, we jointly optimize the offloading decision and the computation resource allocation to minimize the average task duration subject to the limited battery capacity of devices. However, the formulated problem is a large-scale mixed integer non-linear optimization problem with the growing number of base stations and devices, which is NP-hard in general. To develop an efficient offloading scheme with low complexity, we conduct a series of reformulation based on reformulation linearization technology and further propose a parallel optimization framework by utilizing alternating direction method of multipliers (ADMM) method and difference of convex functions (D.C.) programming. The proposed scheme decomposes the large-scale problem into some smaller subproblems, which are done across the multiple computation units in a parallel fashion to speed up the computation process. Simulation results demonstrate that the proposed scheme can obtain a near-optimal performance with low complexity, and can reduce up to 24% of the task duration compared with other schemes. Simulation also shows how much the vertical and horizontal computation cooperations affect the system performance under different network parameters.

Cooperative Task Offloading in Three-Tier Mobile Computing Networks: An ADMM Framework

We consider the NP-hard problem of minimizing a convex quadratic function over the integer lattice $${\mathbf{Z}}^n$$
 . We present a simple semidefinite programming (SDP) relaxation for obtaining a nontrivial lower bound on the optimal value of the problem. By interpreting the solution to the SDP relaxation probabilistically, we obtain a randomized algorithm for finding good suboptimal solutions, and thus an upper bound on the optimal value. The effectiveness of the method is shown for numerical problem instances of various sizes.

/pdf/a-semidefinite-programming-method-for-integer-convex-3w1vrti2v2.pdf

A semidefinite programming method for integer convex quadratic minimization

We consider the NP-hard problem of minimizing a convex quadratic function over the integer lattice ${\bf Z}^n$. We present a simple semidefinite programming (SDP) relaxation for obtaining a nontrivial lower bound on the optimal value of the problem. By interpreting the solution to the SDP relaxation probabilistically, we obtain a randomized algorithm for finding good suboptimal solutions, and thus an upper bound on the optimal value. The effectiveness of the method is shown for numerical problem instances of various sizes.

A Semidefinite Programming Method for Integer Convex Quadratic Minimization

The technique of semidefinite programming (SDP) relaxation can be used to obtain a nontrivial bound on the optimal value of a nonconvex quadratically constrained quadratic program (QCQP). We explore concave quadratic inequalities that hold for any vector in the integer lattice Z n , and show that adding these inequalities to a mixed-integer nonconvex QCQP can improve the SDP-based bound on the optimal value. This scheme is tested using several numerical problem instances of the max-cut problem and the integer least squares problem.

Jaehyun Park

Papers

General Heuristics for Nonconvex Quadratically Constrained Quadratic Programming

A semidefinite programming method for integer convex quadratic minimization

A Semidefinite Programming Method for Integer Convex Quadratic Minimization

Concave Quadratic Cuts for Mixed-Integer Quadratic Problems