Showing papers on "Nonlinear programming published in 2018"

Optimization Methods for Large-Scale Machine Learning

Abstract: This paper provides a review and commentary on the past, present, and future of numerical optimization algorithms in the context of machine learning applications. Through case studies on text classification and the training of deep neural networks, we discuss how optimization problems arise in machine learning and what makes them challenging. A major theme of our study is that large-scale machine learning represents a distinctive setting in which the stochastic gradient (SG) method has traditionally played a central role while conventional gradient-based nonlinear optimization techniques typically falter. Based on this viewpoint, we present a comprehensive theory of a straightforward, yet versatile SG algorithm, discuss its practical behavior, and highlight opportunities for designing algorithms with improved performance. This leads to a discussion about the next generation of optimization methods for large-scale machine learning, including an investigation of two main streams of research on techniques th...

Gait and Trajectory Optimization for Legged Systems Through Phase-Based End-Effector Parameterization

Abstract: We present a single trajectory optimization formulation for legged locomotion that automatically determines the gait sequence, step timings, footholds, swing-leg motions, and six-dimensional body motion over nonflat terrain, without any additional modules. Our phase-based parameterization of feet motion and forces allows to optimize over the discrete gait sequence using only continuous decision variables. The system is represented using a simplified centroidal dynamics model that is influenced by the feet's location and forces. We explicitly enforce friction cone constraints, depending on the shape of the terrain. The nonlinear programming problem solver generates highly dynamic motion plans with full flight phases for a variety of legged systems with arbitrary morphologies in an efficient manner. We validate the feasibility of the generated plans in simulation and on the real quadruped robot ANYmal. Additionally, the entire solver software TOWR , which used to generate these motions is made freely available.

Optim: A mathematical optimization package for Julia

Abstract: Optim provides a range of optimization capabilities written in the Julia programming language (Bezanson et al. 2017). Our aim is to enable researchers, users, and other Julia packages to solve optimization problems without writing such algorithms themselves. The package supports optimization on manifolds, functions of complex numbers, and input types such as arbitrary precision vectors and matrices. We have implemented routines for derivative free, first-order, and second-order optimization methods. The user can provide derivatives themselves, or request that they are calculated using automatic differentiation or finite difference methods. The main focus of the package has currently been on unconstrained optimization, however, box-constrained optimization is supported, and a more comprehensive support for constraints is underway. Similar to Optim, the C library NLopt (Johnson 2008) contains a collection of nonlinear optimization routines. In Python, scipy.optimize supports many of the same algorithms as Optim does, and Pymanopt (Townsend, Niklas, and Weichwald 2016) is a toolbox for manifold optimization. Within the Julia community, the packages BlackBoxOptim.jl and Optimize.jl provide optimization capabilities focusing on derivative-free and large-scale smooth problems respectively. The packages Convex.jl and JuMP.jl (Dunning, Huchette, and Lubin 2017) define modelling languages for which users can formulate optimization problems. In contrast to the previously mentioned optimization codes, Convex and JuMP work as abstraction layers between the user and solvers from a other packages.

Accelerated Methods for Non-Convex Optimization

Abstract: We present an accelerated gradient method for nonconvex optimization problems with Lipschitz continuous first and second derivatives In a time $O(\epsilon^{-7/4} \log(1/ \epsilon) )$, the method f

Exploiting integrality in the global optimization of mixed-integer nonlinear programming problems with BARON

Abstract: In this paper, we present recent developments in the global optimization software BARON to address problems with integer variables. A primary development was the addition of mixed-integer linear programming relaxations to BARON's portfolio of linear and nonlinear programming relaxations, aiming to improve dual bounds and offer good starting points for primal heuristics. Since such relaxations necessitate the solution of NP-hard problems, their introduction to a branch-and-bound algorithm raises many practical issues regarding their effective implementation. In addition to describing BARON's dynamic strategy for deciding under what conditions to activate integer programming relaxations in the course of branch-and-bound, the paper also describes cutting plane and probing techniques that originate from the literature of integer linear programming and have been adapted in BARON to solve nonlinear problems. Finally, we describe BARON's primal heuristics for finding good solutions of mixed-integer nonlinear pro...

Optimal Coordination of Directional Overcurrent Relays in Microgrids by Using Cuckoo-Linear Optimization Algorithm and Fault Current Limiter

Abstract: Nowadays, distributed generations (DGs) and microgrids play a significant role in development of power systems, but DGs produce several protection problems especially in optimal coordination of directional overcurrent relays (DORs). Optimal coordination of DORs is a highly constrained nonlinear programming problem, which has attracted attention of many research studies in recent years. Cuckoo optimization algorithm (COA) is introduced as an optimization tool based on behavior patterns of Cuckoos. In this paper, COA and linear programming (LP) are combined as a new hybrid COA-LP optimization algorithm in order to optimize the coordination protection of DORs in microgrids and find the optimum value of fault current limiter at the point of common coupling. The optimum settings of DORs are obtained for both modes of operation: 1) grid-connected and 2) island mode. The proposed algorithm is applied to Canadian distribution benchmark and a modified IEEE 14 bus networks and compared with particle swarm optimization (PSO) and genetic algorithm (GA). The obtained results show that the computational efficiency of the proposed COA-LP is better than PSO and GA in terms of speed and total operating time of DORs.

Differentiable programming for image processing and deep learning in halide

Abstract: Gradient-based optimization has enabled dramatic advances in computational imaging through techniques like deep learning and nonlinear optimization. These methods require gradients not just of simple mathematical functions, but of general programs which encode complex transformations of images and graphical data. Unfortunately, practitioners have traditionally been limited to either hand-deriving gradients of complex computations, or composing programs from a limited set of coarse-grained operators in deep learning frameworks. At the same time, writing programs with the level of performance needed for imaging and deep learning is prohibitively difficult for most programmers. We extend the image processing language Halide with general reverse-mode automatic differentiation (AD), and the ability to automatically optimize the implementation of gradient computations. This enables automatic computation of the gradients of arbitrary Halide programs, at high performance, with little programmer effort. A key challenge is to structure the gradient code to retain parallelism. We define a simple algorithm to automatically schedule these pipelines, and show how Halide's existing scheduling primitives can express and extend the key AD optimization of "checkpointing." Using this new tool, we show how to easily define new neural network layers which automatically compile to high-performance GPU implementations, and how to solve nonlinear inverse problems from computational imaging. Finally, we show how differentiable programming enables dramatically improving the quality of even traditional, feed-forward image processing algorithms, blurring the distinction between classical and deep methods.

Passenger demand oriented train scheduling and rolling stock circulation planning for an urban rail transit line

Abstract: We study the integration of train scheduling and rolling stock circulation planning under time-varying passenger demand for an urban rail transit line, where the practical train operation constraints, e.g., the capacity of trains, the number of available rolling stocks, and the entering/exiting depot operations, are considered. Three solution approaches are proposed to solve the resulting multi-objective mixed-integer nonlinear programming (MINLP) problem to deliver both an irregular train schedule (i.e., departure and arrival times of all train services) and a rolling stock circulation plan (including entering/exiting depot operations of rolling stocks and connections between train services) simultaneously. We first present an iterative nonlinear programming (INP) approach, where the solutions of the original MINLP problem are obtained by solving a nonlinear programming problem and a mixed integer linear programming (MILP) problem iteratively. Moreover, an equivalent MILP formulation of the original MINLP model is developed and an approximated MILP approach is proposed to reduce the number of constraints introduced by passenger demand. A case study is conducted based on the practical data of the Beijing Yizhuang line, where the three proposed approaches are compared with a state-of-the-art approach and a practical method used by the traffic planners. This comparison shows the effectiveness and efficiency of the three proposed approaches.

Distributed Model Predictive Control for On-Connected Microgrid Power Management

Abstract: This paper considers the energy dispatching optimization of a grid-connected microgrid in a park in the city of Shanghai in a distributed framework, in order to improve its economic and environment-friendly performance. The microgrid is composed of distributed generations (DGs), energy storage systems, and shiftable loads. Some properties of the microgrid make the dispatching problem difficult. For example, the disturbance in renewable energy is unpredictable; the operation of shiftable loads is discrete and the relationship between the generated power and the total operation cost is nonlinear. A novel distributed model predictive control (MPC) is proposed for the power dispatching optimization of the microgrid in this paper, where there is a local MPC for each of the following entities: DGs, storage batteries, and shiftable loads. In this method, the centralized mix-integer programming problem of microgrid energy dispatching is converted into several interacted nonlinear programming problems and integer programming problems, and subsystem-based MPCs coordinate with each other via iteratively minimizing the cost over the entire system. In this way, the realization of plug-and-play property becomes easier and the computational load is reduced. The numerical results show the effectiveness of the proposed method.

Iterative Parameter Estimation for Signal Models Based on Measured Data

Abstract: This paper studies the modeling of multi-frequency signals based on measured data. With the use of the hierarchical identification principle and the iterative search, several iterative parameter estimation algorithms are derived for the signal models with the known frequencies and the unknown frequencies. For the multi-frequency signals, the hierarchical estimation algorithms are derived by means of parameter decomposition. Through the decomposition, the original optimization problem is transformed into the combination of the nonlinear optimization and the linear optimization problems. The simulation results show that the proposed hierarchical algorithms have better performance than the overall estimation algorithms without parameter decomposition.

A nonlinear-programming methodology for multi-attribute decision-making problem with interval-valued intuitionistic fuzzy soft sets information

Abstract: Interval-valued intuitionistic fuzzy (IVIF) soft set is one of the useful extensions of the fuzzy soft set which efficiently deals with the uncertain data for the decision-making processes. In this paper, an attempt has been made to present a nonlinear-programming (NP) model based on the technique for order preference by similarity to ideal solution (TOPSIS), to solve multi-attribute decision-making problems. In this approach, both ratings of alternatives on attributes and weights of attributes are represented by IVIF sets. Based on the available information, NP models are constructed on the basis of the concepts of the relative-closeness coefficient and the weighted distance. Some NP models are further deduced to calculate relative-closeness of sets of alternatives which can be used to generate the ranking order of the alternatives. A real example is taken to demonstrate the applicability and validity of the proposed methodology.

Energy-Efficient Device-to-Device Communications for Green Smart Cities

Abstract: To afford effective service of real-time monitoring and responses for smart cities, it is desired to provide ubiquitous network connections and high data rate services. However, the huge demands for ubiquitous high data rate wireless communications have caused a sharp increase in energy consumption and green house gas emission. In order to realize a sustainable smart city, it is critical to incorporate green communication technique into smart city developments. Device-to-device (D2D) communication has been recognized as one of the key technologies to improve data rate and reduce power consumption, which allows two physically nearby located user equipments to communicate directly with each other. In this paper, with the target of achieving green communications through D2D, we investigate the joint optimization of uplink subcarrier assignment (SA) and power allocation (PA) in D2D underlying cellular networks. Specifically, the problem formulation is to minimize the energy cost of all users in the system while guaranteeing the required data rate of both the D2D user equipments (DUEs) and cellular user equipments. Such an optimization problem is in general a mixed-integer nonlinear programming problem that is NP-hard. To make this problem tractable, we decompose it into the SA and PA subproblems. In particular, we design a heuristic algorithm to assign subcarrier by assuming that the transmit power is evenly allocated over all subcarriers. After that, we solve the PA subproblem by exploiting the difference between the concave function (D.C.) structure of the constraints and transform it into a convex optimization problem. Simulation results demonstrate the remarkable improvement in terms of power consumption by using our algorithms.

A Neurodynamic Approach to Distributed Optimization With Globally Coupled Constraints

Abstract: In this paper, a distributed neurodynamic approach is proposed for constrained convex optimization. The objective function is a sum of local convex subproblems, whereas the constraints of these subproblems are coupled. Each local objective function is minimized individually with the proposed neurodynamic optimization approach. Through information exchange between connected neighbors only, all nodes can reach consensus on the Lagrange multipliers of all global equality and inequality constraints, and the decision variables converge to the global optimum in a distributed manner. Simulation results of two power system cases are discussed to substantiate the effectiveness and characteristics of the proposed approach.

Collision-Free Navigation of Autonomous Vehicles Using Convex Quadratic Programming-Based Model Predictive Control

Abstract: Collision-free navigation of autonomous vehicles by means of convex quadratic programming (CQP) based model predictive control (MPC) is considered in this paper. A new collision-free navigation function is designed for real-time collision avoidance of an autonomous vehicle in both static and dynamic environments. Furthermore, vehicle shape is taken into consideration during trajectory generation as a convex polygonal region defined by linear constraints rather than a single point. Then, the MPC optimization problem with the vehicle shape is solved as a CQP-based MPC scheme in the sense of path planning. Compared to the previous MPC, which can only be reduced to a nonlinear programming problem, the control sequences of CQP-based MPC can be obtained quickly with improved real-time system performance. Simulations in diverse scenarios, including a real vehicle dataset, show the validity of the proposed approach.

Non-linear programming method for multi-criteria decision making problems under interval neutrosophic set environment

Abstract: Interval neutrosophic sets (INSs) capturing the uncertainties by characterizing into the intervals of the truth, the indeterminacy, and the falsity membership degrees, is a more flexible way to explore the decision-making applications. In this paper, we develop a nonlinear programming (NP) model based on the technique for order preference by similarity to ideal solution (TOPSIS), to solve decision-making problems in which criterion values and their importance are given in the form of interval neutrosophic numbers (INNs). Based on the concept of closeness coefficient, we firstly construct a pair of the nonlinear fractional programming model and then transform it into the linear programming model. Furthermore, to determine the ranking of considered alternatives, likelihood-based comparison relations are constructed. Finally, an illustrative example demonstrates the applicability of the proposed method for dealing with decision making problems with incomplete knowledge.

AC OPF for Smart Distribution Networks: An Efficient and Robust Quadratic Approach

Abstract: Smart grid schemes in which multiple network elements and participants are managed for the benefit of the distribution network (e.g., energy loss reduction, restoration, etc.) require sophisticated algorithms to control them in the most suitable manner while catering for network constraints. Such a complex decision making process can be solved by tailoring the ac optimal power flow (OPF) problem to the corresponding smart grid scheme. Non-linear programming ac OPF formulations, however, can suffer from scalability and robustness issues which in turn might limit their adoption. Here, a novel quadratic programming formulation is proposed and compared against the non-linear, quadratically constrained, and linearized approaches. Two cases are carried out to assess their performance: 1) management of distributed generation units to maximize renewable energy harvesting (continuous control variables) and 2) control of capacitors to minimize energy losses (discrete control variables). The results demonstrate that the proposed quadratic approach significantly outperforms the more conventional formulations in both computational efficiency and robustness. This makes it a suitable alternative to be at the heart of the decision making of complex, real-time schemes to be adopted by future smart distribution networks.

Autonomous Entry Guidance for Hypersonic Vehicles by Convex Optimization

Abstract: Advanced entry guidance systems could potentially enable future vehicles to generate feasible/optimal flight trajectories onboard for the latest mission requirements and track the new trajectories ...

Linear RGB-D SLAM for Planar Environments

Abstract: We propose a new formulation for including orthogonal planar features as a global model into a linear SLAM approach based on sequential Bayesian filtering. Previous planar SLAM algorithms estimate the camera poses and multiple landmark planes in a pose graph optimization. However, since it is formulated as a high dimensional nonlinear optimization problem, there is no guarantee the algorithm will converge to the global optimum. To overcome these limitations, we present a new SLAM method that jointly estimates camera position and planar landmarks in the map within a linear Kalman filter framework. It is rotations that make the SLAM problem highly nonlinear. Therefore, we solve for the rotational motion of the camera using structural regularities in the Manhattan world (MW), resulting in a linear SLAM formulation. We test our algorithm on standard RGB-D benchmarks as well as additional large indoors environments, demonstrating comparable performance to other state-of-the-art SLAM methods without the use of expensive nonlinear optimization.

Integration of real-time traffic management and train control for rail networks: Part 1: Optimization problems and solution approaches

Abstract: We study the integration of real-time traffic management and train control by using mixed-integer nonlinear programming (MINLP) and mixed-integer linear programming (MILP) approaches. Three innovative integrated optimization approaches for real-time traffic management that inherently include train control are developed to deliver both a train dispatching solution (including train routes, orders, departure and arrival times at passing stations) and a train control solution (i.e., train speed trajectories). Train speed is considered variable, and the blocking time of a train on a block section dynamically depends on its real speed. To formulate the integrated problem, we first propose an MINLP problem (PNLP), which is solved by a two-level approach. This MINLP problem is then reformulated by approximating the nonlinear terms with piecewise affine functions, resulting in an MILP problem (PPWA). Moreover, we consider a preprocessing method to generate the possible speed profile options for each train on each block section, one of which is further selected by a proposed MILP problem (PTSPO) with respect to safety, capacity, and speed consistency constraints. This problem is solved by means of a custom-designed two-step approach, in order to speed up the solving procedure. Numerical experiments are conducted using data from the Dutch railway network to comparatively evaluate the effectiveness and efficiency of the three proposed approaches with heterogeneous traffic. According to the experimental results, the MILP approach (PTSPO) yields the best overall performance within the required computation time. The experimental results demonstrate the benefits of the integration, i.e., train delays can be reduced by managing train speed.

Successive Convexification: A Superlinearly Convergent Algorithm for Non-convex Optimal Control Problems

Abstract: This paper presents the SCvx algorithm, a successive convexification algorithm designed to solve non-convex constrained optimal control problems with global convergence and superlinear convergence-rate guarantees. The proposed algorithm can handle nonlinear dynamics and non-convex state and control constraints. It solves the original problem to optimality by successively linearizing non-convex dynamics and constraints about the solution of the previous iteration. The resulting convex subproblems are numerically tractable, and can be computed quickly and reliably using convex optimization solvers, making the SCvx algorithm well suited for real-time applications. Analysis is presented to show that the algorithm converges both globally and superlinearly, guaranteeing i) local optimality recovery: if the converged solution is feasible with respect to the original problem, then it is also a local optimum; ii) strong convergence: if the Kurdyka-Lojasiewicz (KL) inequality holds at the converged solution, then the solution is unique. The superlinear rate of convergence is obtained by exploiting the structure of optimal control problems, showcasing that faster rate of convergence can be achieved by leveraging specific problem properties when compared to generic nonlinear programming methods. Numerical simulations are performed for a non-convex quad-rotor motion planning problem, and corresponding results obtained using Sequential Quadratic Programming (SQP) and general purpose Interior Point Method (IPM) solvers are provided for comparison. The results show that the convergence rate of the SCvx algorithm is indeed superlinear, and that SCvx outperforms the other two methods by converging in less number of iterations.

An affine scaling interior trust-region method combining with nonmonotone line search filter technique for linear inequality constrained minimization

Abstract: This paper proposes an affine scaling interior trust-region method in association with nonmonotone line search filter technique for solving nonlinear optimization problems subject to linear inequality constraints. Based on a Newton step which is derived from the complementarity conditions of linear inequality constrained optimization, a trust-region subproblem subject only to an ellipsoidal constraint is defined by minimizing a quadratic model with an appropriate quadratic function and scaling matrix. The nonmonotone schemes combining with trust-region strategy and line search filter technique can bring about speeding up the convergence progress in the case of high nonlinear. A new backtracking relevance condition is given which assures global convergence without using the switching condition used in the traditional line search filter technique. The fast local convergence rate of the proposed algorithm is achieved which is not depend on any external restoration procedure. The preliminary numerical experim...

Strict Constraint Qualifications and Sequential Optimality Conditions for Constrained Optimization

Abstract: Sequential optimality conditions for constrained optimization are necessarily satisfied by local minimizers, independently of the fulfillment of constraint qualifications. These conditions support the employment of different stopping criteria for practical optimization algorithms. On the other hand, when an appropriate property on the constraints holds at a point that satisfies a sequential optimality condition, such a point also satisfies the Karush-Kuhn-Tucker conditions. Those properties will be called strict constraint qualifications in this paper. As a consequence, for each sequential optimality condition, it is natural to ask for its weakest strict associated constraint qualification. This problem has been solved in a recent paper for the Approximate Karush-Kuhn-Tucker sequential optimality condition. In the present paper, we characterize the weakest strict constraint qualifications associated with other sequential optimality conditions that are useful for defining stopping criteria of algorithms. I...

A Framework of Customizing Electricity Retail Prices

Abstract: The problem of designing customized pricing strategies for different residential users is investigated based on the identification results of residential electric appliances and classifications of end-users according to their consumption behaviors. This study is based on the following assumptions: 1) Each retailer purchases electricity from the forward contract market, day-ahead spot market, and real-time market; 2) the competition among retailers is modeled by a market share function; 3) each retailer adopts fixed time-of-use prices for end-users; 4) the price fluctuations in day-ahead and real-time spot markets as well as uncertainty of electricity consumption behaviors are considered as main sources of risk. Under these assumptions, a pricing framework for retailers is established based on the bilevel programming framework and the optimal clustering in a time sequence. Meanwhile, profit risk is considered by taking conditional value at risk as the risk measure. The proposed bilevel optimization model is finally reformulated into a mixed-integer nonlinear programming problem by solving Karush–Kuhn–Tucker conditions. The online optimization solvers provided by the network-enabled optimization system server and the commercial solver AMPL/GUROBI are used to solve the developed models, respectively. Finally, a case study is employed to demonstrate the feasibility and efficiency of the developed models and algorithms.