Showing papers on "Discrete optimization published in 2020"

Coordinated Charging Scheduling of Electric Vehicles: A Mixed-Variable Differential Evolution Approach

Abstract: The increasing popularity of battery-limited electric vehicles puts forward an important issue of how to charge the vehicles effectively. This problem, commonly referred to as Electric Vehicle Charging Scheduling (EVCS), has been proven to be NP-hard. Most of the existing works formulate the EVCS problem simply as a constrained shortest path finding problem and treat it by discrete optimization. However, other variables such as the charging amount of energy and the charging option at a station need to be considered in practical use. This paper hence formulates the EVCS problem as a hierarchical mixed-variable optimization problem, considering the dependency among the station selection, the charging option at each station and the charging amount settings. To adapt to the new problem model, we specifically design a Mixed-Variable Differentiate Evolution (MVDE) as the scheduling algorithm for our proposed EVCS system. The MVDE contains several specific operators, including a charging station route construction, a hierarchical mixed-variable mutation operator and a constraint-aware evaluation operator. Experimental results validate the effectiveness of our proposed MVDE-based system on both synthetic and real-world transportation networks.

Improving Inference for Neural Image Compression

Abstract: We consider the problem of lossy image compression with deep latent variable models. State-of-the-art methods build on hierarchical variational autoencoders (VAEs) and learn inference networks to predict a compressible latent representation of each data point. Drawing on the variational inference perspective on compression, we identify three approximation gaps which limit performance in the conventional approach: an amortization gap, a discretization gap, and a marginalization gap. We propose remedies for each of these three limitations based on ideas related to iterative inference, stochastic annealing for discrete optimization, and bits-back coding, resulting in the first application of bits-back coding to lossy compression. In our experiments, which include extensive baseline comparisons and ablation studies, we achieve new state-of-the-art performance on lossy image compression using an established VAE architecture, by changing only the inference method.

Cooperative decision-making for mixed traffic: A ramp merging example

Abstract: The rapid conceptual development and commercialization of connected automated vehicle (CAV) has led to the problem of mixed traffic, i.e., traffic mixed with CAVs and conventional human-operated vehicles (HVs). The paper studies cooperative decision-making for mixed traffic (CDMMT). Using discrete optimization, a CDMMT mechanism is developed to facilitate ramp merging, and to properly capture the cooperative and non-cooperative behaviors in mixed traffic. The CDMMT mechanism can be described as a bi-level optimization program in which state-constrained optimal control-based trajectory design problems are imbedded in a sequencing problem. A bi-level dynamic programming-based solution approach is developed to efficiently solve the problem. The proposed modeling mechanism and solution approach are generic to deterministic decisions and can guarantee system-efficient solutions. A micro-simulation environment is built for model validation and analysis of mixed traffic. The results show that compared to the scenario with 100% HVs, ramp-merging can be smoother in mixed traffic environment. At high CAV penetration, the section throughput increases about 18%. With the proposed CDMMT mechanism, traffic throughput can be further increased by 10–15%. The proposed methods form the basis of traffic analysis and cooperative control at ramp-merging sections under mixed traffic environment.

Learning to Hash with Graph Neural Networks for Recommender Systems

Abstract: Recommender systems in industry generally include two stages: recall and ranking. Recall refers to efficiently identify hundreds of candidate items that user may interest in from a large volume of item corpus, while the latter aims to output a precise ranking list using complex ranking models. Recently, graph representation learning has attracted much attention in supporting high quality candidate search at scale. Despite its effectiveness in learning embedding vectors for objects in the user-item interaction network, the computational costs to infer users’ preferences in continuous embedding space are tremendous. In this work, we investigate the problem of hashing with graph neural networks (GNNs) for high quality retrieval, and propose a simple yet effective discrete representation learning framework to jointly learn continuous and discrete codes. Specifically, a deep hashing with GNNs (HashGNN) is presented, which consists of two components, a GNN encoder for learning node representations, and a hash layer for encoding representations to hash codes. The whole architecture is trained end-to-end by jointly optimizing two losses, i.e., reconstruction loss from reconstructing observed links, and ranking loss from preserving the relative ordering of hash codes. A novel discrete optimization strategy based on straight through estimator (STE) with guidance is proposed. The principal idea is to avoid gradient magnification in back-propagation of STE with continuous embedding guidance, in which we begin from learning an easier network that mimic the continuous embedding and let it evolve during the training until it finally goes back to STE. Comprehensive experiments over three publicly available and one real-world Alibaba company datasets demonstrate that our model not only can achieve comparable performance compared with its continuous counterpart but also runs multiple times faster during inference.

Sparse Regression: Scalable Algorithms and Empirical Performance

Abstract: In this paper, we review state-of-the-art methods for feature selection in statistics with an application-oriented eye. Indeed, sparsity is a valuable property and the profusion of research on the topic might have provided little guidance to practitioners. We demonstrate empirically how noise and correlation impact both the accuracy—the number of correct features selected—and the false detection—the number of incorrect features selected—for five methods: the cardinality-constrained formulation, its Boolean relaxation, l1 regularization and two methods with non-convex penalties. A cogent feature selection method is expected to exhibit a two-fold convergence, namely the accuracy and false detection rate should converge to 1 and 0 respectively, as the sample size increases. As a result, proper method should recover all and nothing but true features. Empirically, the integer optimization formulation and its Boolean relaxation are the closest to exhibit this two properties consistently in various regimes of noise and correlation. In addition, apart from the discrete optimization approach which requires a substantial, yet often affordable, computational time, all methods terminate in times comparable with the glmnet package for Lasso. We released code for methods that were not publicly implemented. Jointly considered, accuracy, false detection and computational time provide a comprehensive assessment of each feature selection method and shed light on alternatives to the Lasso-regularization which are not as popular in practice yet.

Discrete Model Compression With Resource Constraint for Deep Neural Networks

Abstract: In this paper, we target to address the problem of compression and acceleration of Convolutional Neural Networks (CNNs). Specifically, we propose a novel structural pruning method to obtain a compact CNN with strong discriminative power. To find such networks, we propose an efficient discrete optimization method to directly optimize channel-wise differentiable discrete gate under resource constraint while freezing all the other model parameters. Although directly optimizing discrete variables is a complex non-smooth, non-convex and NP-hard problem, our optimization method can circumvent these difficulties by using the straight-through estimator. Thus, our method is able to ensure that the sub-network discovered within the training process reflects the true sub-network. We further extend the discrete gate to its stochastic version in order to thoroughly explore the potential sub-networks. Unlike many previous methods requiring per-layer hyper-parameters, we only require one hyper-parameter to control FLOPs budget. Moreover, our method is globally discrimination-aware due to the discrete setting. The experimental results on CIFAR-10 and ImageNet show that our method is competitive with state-of-the-art methods.

Theory of evolutionary computation : recent developments in discrete optimization

Abstract: Probabilistic Tools for the Analysis of Randomized Optimization Heuristics.- Drift Analysis.- Complexity Theory for Discrete Black-Box Optimization Heuristics.- Parameterized Complexity Analysis of Randomized Search Heuristics.- Analysing Stochastic Search Heuristics Operating on a Fixed Budget.- Theory of Parameter Control for Discrete Black-Box Optimization: Provable Performance Gains Through Dynamic Parameter Choices.- Analysis of Evolutionary Algorithms in Dynamic and Stochastic Environments.- The Benefits of Population Diversity in Evolutionary Algorithms: A Survey of Rigorous Runtime Analyses.- Theory of Estimation-of-Distribution Algorithms.- Theoretical Foundations of Immune-Inspired Randomized Search Heuristics for Optimization.- Computational Complexity Analysis of Genetic Programming.

Label Embedding Online Hashing for Cross-Modal Retrieval

Abstract: Supervised cross-modal hashing has gained a lot of attention recently. However, most existing methods learn binary codes or hash functions in a batch-based scheme, which is inefficient in an online scenario, i.e., data points come in a streaming fashion. Online hashing is a promising solution; however, there still exist several challenges, e.g., how to effectively exploit semantic information, how to discretely solve the binary optimization problem, how to efficiently update hash codes and hash functions. To address these issues, in this paper, we propose a novel supervised online cross-modal hashing method, i.e., Label EMbedding ONline hashing, LEMON for short. It builds a label embedding framework including label similarity preserving and label reconstructing, which may generate discriminative binary codes and reduce the computational complexity. Furthermore, it not only preserves the pairwise similarity of incoming data, but also establishes a connection between newly coming data and existing data by the inner product minimization on a block similarity matrix. In the light of this, it can exploit more similarity information and make the optimization less sensitive to incoming data, leading to effective binary codes. In addition, we design a discrete optimization algorithm to solve the binary optimization problem without relaxation. Therefore, the quantization error can be reduced. Moreover, its computational complexity is only relevant to the size of incoming data, making it very efficient and scalable to large-scale datasets. Extensive experimental results on three benchmark datasets demonstrate that LEMON outperforms some state-of-the-art offline and online cross-modal hashing methods in terms of accuracy and efficiency.

Performance Analysis and Optimization of the Coverage Probability in Dual Hop LoRa Networks With Different Fading Channels

Abstract: In this work, the performance evaluation and the optimization of dual-hop LoRa network are investigated. In particular, the coverage probability (Pcov) of edge end-devices (EDs) is computed in closed-form expressions under various fading channels, i.e., Nakagami- $m$ and Rayleigh fading. The Pcov under Nakagami- $m$ fading is computed in the approximated closed-form expressions; the Pcov under Rayleigh fading, on the other hand, is calculated in the exact closed-form expressions. In addition, we also investigate the impact of different kinds of interference on the performance of the Pcov, i.e., intra-SF interference, inter-SF interference (or capture effect) and both intra- and inter-SF interference. Our findings show that the impact of imperfect orthogonality is not non-negligible, along with the intra-SF interference. Moreover, based on the proposed mathematical framework, we formulate an optimization problem, which finds the optimal location of the relay to maximize the coverage probability. Since it is a mixed integer program with a non-convex objective function, we decompose the original problem with discrete optimization variables into sub-problems with a convex feasible set. After that, each sub-problem is effectively solved by utilizing the gradient descent approach. Monte Carlo simulations are supplied to verify the correctness of our mathematical framework. In addition, the results manifest that our proposed optimization algorithm converges rapidly, and the coverage probability is significantly improved when the location of relay is optimized.

Multimodularity in the Stochastic Appointment Scheduling Problem with Discrete Arrival Epochs

Abstract: We address the problem of designing appointment scheduling strategies in a stochastic environment accounting for patient no-shows, nonpunctuality, general stochastic service times, and unscheduled ...

Discrete Optimization for Unsupervised Sentence Summarization with Word-Level Extraction

Abstract: Automatic sentence summarization produces a shorter version of a sentence, while preserving its most important information. A good summary is characterized by language fluency and high information overlap with the source sentence. We model these two aspects in an unsupervised objective function, consisting of language modeling and semantic similarity metrics. We search for a high-scoring summary by discrete optimization. Our proposed method achieves a new state-of-the art for unsupervised sentence summarization according to ROUGE scores. Additionally, we demonstrate that the commonly reported ROUGE F1 metric is sensitive to summary length. Since this is unwillingly exploited in recent work, we emphasize that future evaluation should explicitly group summarization systems by output length brackets.

Closing the Gap for Makespan Scheduling via Sparsification Techniques

Abstract: Makespan scheduling on identical machines is one of the most basic and fundamental packing problems studied in the discrete optimization literature It asks for an assignment of n jobs to a set of

Bidirectional Discrete Matrix Factorization Hashing for Image Search

Abstract: Unsupervised image hashing has recently gained significant momentum due to the scarcity of reliable supervision knowledge, such as class labels and pairwise relationship. Previous unsupervised methods heavily rely on constructing sufficiently large affinity matrix for exploring the geometric structure of data. Nevertheless, due to lack of adequately preserving the intrinsic information of original visual data, satisfactory performance can hardly be achieved. In this article, we propose a novel approach, called bidirectional discrete matrix factorization hashing (BDMFH), which alternates two mutually promoted processes of 1) learning binary codes from data and 2) recovering data from the binary codes. In particular, we design the inverse factorization model, which enforces the learned binary codes inheriting intrinsic structure from the original visual data. Moreover, we develop an efficient discrete optimization algorithm for the proposed BDMFH. Comprehensive experimental results on three large-scale benchmark datasets show that the proposed BDMFH not only significantly outperforms the state-of-the-arts but also provides the satisfactory computational efficiency.

Learning Sparse Classifiers: Continuous and Mixed Integer Optimization Perspectives

Abstract: We consider a discrete optimization formulation for learning sparse classifiers, where the outcome depends upon a linear combination of a small subset of features. Recent work has shown that mixed integer programming (MIP) can be used to solve (to optimality) $\ell_0$-regularized regression problems at scales much larger than what was conventionally considered possible. Despite their usefulness, MIP-based global optimization approaches are significantly slower compared to the relatively mature algorithms for $\ell_1$-regularization and heuristics for nonconvex regularized problems. We aim to bridge this gap in computation times by developing new MIP-based algorithms for $\ell_0$-regularized classification. We propose two classes of scalable algorithms: an exact algorithm that can handle $p\approx 50,000$ features in a few minutes, and approximate algorithms that can address instances with $p\approx 10^6$ in times comparable to the fast $\ell_1$-based algorithms. Our exact algorithm is based on the novel idea of \textsl{integrality generation}, which solves the original problem (with $p$ binary variables) via a sequence of mixed integer programs that involve a small number of binary variables. Our approximate algorithms are based on coordinate descent and local combinatorial search. In addition, we present new estimation error bounds for a class of $\ell_0$-regularized estimators. Experiments on real and synthetic data demonstrate that our approach leads to models with considerably improved statistical performance (especially, variable selection) when compared to competing methods.

Discrete Optimization for Unsupervised Sentence Summarization with Word-Level Extraction

Abstract: Automatic sentence summarization produces a shorter version of a sentence, while preserving its most important information. A good summary is characterized by language fluency and high information overlap with the source sentence. We model these two aspects in an unsupervised objective function, consisting of language modeling and semantic similarity metrics. We search for a high-scoring summary by discrete optimization. Our proposed method achieves a new state-of-the art for unsupervised sentence summarization according to ROUGE scores. Additionally, we demonstrate that the commonly reported ROUGE F1 metric is sensitive to summary length. Since this is unwillingly exploited in recent work, we emphasize that future evaluation should explicitly group summarization systems by output length brackets.

A Survey on Recent Progress in the Theory of Evolutionary Algorithms for Discrete Optimization

Abstract: The theory of evolutionary computation for discrete search spaces has made significant progress in the last ten years This survey summarizes some of the most important recent results in this research area It discusses fine-grained models of runtime analysis of evolutionary algorithms, highlights recent theoretical insights on parameter tuning and parameter control, and summarizes the latest advances for stochastic and dynamic problems We regard how evolutionary algorithms optimize submodular functions and we give an overview over the large body of recent results on estimation of distribution algorithms Finally, we present the state of the art of drift analysis, one of the most powerful analysis technique developed in this field

Multiobjective discrete optimization using the TOPSIS and entropy method for protection of pedestrian lower extremity

Abstract: Design of vehicle frontal structures signifies an important topic of studies on protection of pedestrian lower extremity. Conventional optimization has been largely focused on continuous variable problems without involving complexity of human model. Nevertheless, design of frontal structures is commonly discrete from a manufacturing perspective and the responses of the lower extremity of pedestrian are highly nonlinear in nature. For this reason, this study aimed to develop a multiobjective discrete optimization (MODO) algorithm for design of frontal structures involving a pedestrian model. In the proposed MODO method, the order preference by similarity to ideal solution (TOPSIS) was coupled with the entropy algorithm to develop a multiple attribute decision making (MADM) model for converting multiple conflicting objectives into a unified single cost function. The presented optimization procedure is iterated using the successive orthogonal experiment to deal with a large number of design variables and design levels. In this study, the proposed method was first verified by two benchmark examples; and then was applied to a more sophisticated real-life design case of vehicle frontal structure to better protect pedestrian lower extremity from impacting injury. The finite element model was validated via experimental tests first and the optimized design was then prototyped for the further physical tests. The results showed that the algorithm is able to achieve an optimal design in a fairly efficient manner. The proposed algorithm exhibits considerable potential to solve other complex engineering design problems.

An efficient discrete optimization algorithm for performance-based design optimization of steel frames:

Abstract: Performance-based design optimization of steel frames, with element sections selected from standard sections, is a computationally intensive task. In this article, an efficient discrete optimizatio...

Algorithms for Tensor Network Contraction Ordering

Abstract: Contracting tensor networks is often computationally demanding. Well-designed contraction sequences can dramatically reduce the contraction cost. We explore the performance of simulated annealing and genetic algorithms, two common discrete optimization techniques, to this ordering problem. We benchmark their performance as well as that of the commonly-used greedy search on physically relevant tensor networks. Where computationally feasible, we also compare them with the optimal contraction sequence obtained by an exhaustive search. Furthermore, we present a systematic comparison with state-of-the-art tree decomposition and graph partitioning algorithms in the context of random regular graph tensor networks. We find that the algorithms we consider consistently outperform a greedy search given equal computational resources, with an advantage that scales with tensor network size. We compare the obtained contraction sequences and identify signs of highly non-local optimization, with the more sophisticated algorithms sacrificing run-time early in the contraction for better overall performance.

Properties of Laplace Operators for Tetrahedral Meshes

Abstract: Discrete Laplacians for triangle meshes are a fundamental tool in geometry processing. The so‐called cotan Laplacian is widely used since it preserves several important properties of its smooth counterpart. It can be derived from different principles: either considering the piecewise linear nature of the primal elements or associating values to the dual vertices. Both approaches lead to the same operator in the two‐dimensional setting. In contrast, for tetrahedral meshes, only the primal construction is reminiscent of the cotan weights, involving dihedral angles. We provide explicit formulas for the lesser‐known dual construction. In both cases, the weights can be computed by adding the contributions of individual tetrahedra to an edge. The resulting two different discrete Laplacians for tetrahedral meshes only retain some of the properties of their two‐dimensional counterpart. In particular, while both constructions have linear precision, only the primal construction is positive semi‐definite and only the dual construction generates positive weights and provides a maximum principle for Delaunay meshes. We perform a range of numerical experiments that highlight the benefits and limitations of the two constructions for different problems and meshes.

Self-Supervised Learning of Generative Spin-Glasses with Normalizing Flows.

Abstract: Spin-glasses are universal models that can capture complex behavior of many-body systems at the interface of statistical physics and computer science including discrete optimization, inference in graphical models, and automated reasoning. Computing the underlying structure and dynamics of such complex systems is extremely difficult due to the combinatorial explosion of their state space. Here, we develop deep generative continuous spin-glass distributions with normalizing flows to model correlations in generic discrete problems. We use a self-supervised learning paradigm by automatically generating the data from the spin-glass itself. We demonstrate that key physical and computational properties of the spin-glass phase can be successfully learned, including multi-modal steady-state distributions and topological structures among metastable states. Remarkably, we observe that the learning itself corresponds to a spin-glass phase transition within the layers of the trained normalizing flows. The inverse normalizing flows learns to perform reversible multi-scale coarse-graining operations which are very different from the typical irreversible renormalization group techniques.