The time-frequency and time-scale communities have recently developed a large number of overcomplete waveform dictionaries --- stationary wavelets, wavelet packets, cosine packets, chirplets, and warplets, to name a few. Decomposition into overcomplete systems is not unique, and several methods for decomposition have been proposed, including the method of frames (MOF), Matching pursuit (MP), and, for special dictionaries, the best orthogonal basis (BOB).
Basis Pursuit (BP) is a principle for decomposing a signal into an "optimal" superposition of dictionary elements, where optimal means having the smallest l1 norm of coefficients among all such decompositions. We give examples exhibiting several advantages over MOF, MP, and BOB, including better sparsity and superresolution. BP has interesting relations to ideas in areas as diverse as ill-posed problems, in abstract harmonic analysis, total variation denoising, and multiscale edge denoising.
BP in highly overcomplete dictionaries leads to large-scale optimization problems. With signals of length 8192 and a wavelet packet dictionary, one gets an equivalent linear program of size 8192 by 212,992. Such problems can be attacked successfully only because of recent advances in linear programming by interior-point methods. We obtain reasonable success with a primal-dual logarithmic barrier method and conjugate-gradient solver.

Atomic Decomposition by Basis Pursuit

The time-frequency and time-scale communities have recently developed a large number of overcomplete waveform dictionaries---stationary wavelets, wavelet packets, cosine packets, chirplets, and warplets, to name a few. Decomposition into overcomplete systems is not unique, and several methods for decomposition have been proposed, including the method of frames (MOF), matching pursuit (MP), and, for special dictionaries, the best orthogonal basis (BOB).
Basis pursuit (BP) is a principle for decomposing a signal into an "optimal"' superposition of dictionary elements, where optimal means having the smallest l1 norm of coefficients among all such decompositions. We give examples exhibiting several advantages over MOF, MP, and BOB, including better sparsity and superresolution. BP has interesting relations to ideas in areas as diverse as ill-posed problems, abstract harmonic analysis, total variation denoising, and multiscale edge denoising.
BP in highly overcomplete dictionaries leads to large-scale optimization problems. With signals of length 8192 and a wavelet packet dictionary, one gets an equivalent linear program of size 8192 by 212,992. Such problems can be attacked successfully only because of recent advances in linear and quadratic programming by interior-point methods. We obtain reasonable success with a primal-dual logarithmic barrier method and conjugate-gradient solver.

/pdf/atomic-decomposition-by-basis-pursuit-2z0e8wwyak.pdf

We discuss formulations of integer programs with a huge number of variables and their solution by column generation methods, i.e., implicit pricing of nonbasic variables to generate new columns or to prove LP optimality at a node of the branch-and-bound tree. We present classes of models for which this approach decomposes the problem, provides tighter LP relaxations, and eliminates symmetry. We then discuss computational issues and implementation of column generation, branch-and-bound algorithms, including special branching rules and efficient ways to solve the LP relaxation. We also discuss the relationship with Lagrangian duality.

/pdf/branch-and-price-column-generation-for-solving-huge-integer-4jvlny6blr.pdf

Branch-And-Price: Column Generation for Solving Huge Integer Programs

I What Are the Ages of My Three Sons?.- 1 Why Are Some Problems Difficult to Solve?.- II How Important Is a Model?.- 2 Basic Concepts.- III What Are the Prices in 7-11?.- 3 Traditional Methods - Part 1.- IV What Are the Numbers?.- 4 Traditional Methods - Part 2.- V What's the Color of the Bear?.- 5 Escaping Local Optima.- VI How Good Is Your Intuition?.- 6 An Evolutionary Approach.- VII One of These Things Is Not Like the Others.- 7 Designing Evolutionary Algorithms.- VIII What Is the Shortest Way?.- 8 The Traveling Salesman Problem.- IX Who Owns the Zebra?.- 9 Constraint-Handling Techniques.- X Can You Tune to the Problem?.- 10 Tuning the Algorithm to the Problem.- XI Can You Mate in Two Moves?.- 11 Time-Varying Environments and Noise.- XII Day of the Week of January 1st.- 12 Neural Networks.- XIII What Was the Length of the Rope?.- 13 Fuzzy Systems.- XIV Everything Depends on Something Else.- 14 Coevolutionary Systems.- XV Who's Taller?.- 15 Multicriteria Decision-Making.- XVI Do You Like Simple Solutions?.- 16 Hybrid Systems.- 17 Summary.- Appendix A: Probability and Statistics.- A.1 Basic concepts of probability.- A.2 Random variables.- A.2.1 Discrete random variables.- A.2.2 Continuous random variables.- A.3 Descriptive statistics of random variables.- A.4 Limit theorems and inequalities.- A.5 Adding random variables.- A.6 Generating random numbers on a computer.- A.7 Estimation.- A.8 Statistical hypothesis testing.- A.9 Linear regression.- A.10 Summary.- Appendix B: Problems and Projects.- B.1 Trying some practical problems.- B.2 Reporting computational experiments with heuristic methods.- References.

/pdf/how-to-solve-it-modern-heuristics-1pu40hrf57.pdf

How to Solve It: Modern Heuristics

This paper presents a new mixed-integer linear formulation for the unit commitment problem of thermal units. The formulation proposed requires fewer binary variables and constraints than previously reported models, yielding a significant computational saving. Furthermore, the modeling framework provided by the new formulation allows including a precise description of time-dependent startup costs and intertemporal constraints such as ramping limits and minimum up and down times. A commercially available mixed-integer linear programming algorithm has been applied to efficiently solve the unit commitment problem for practical large-scale cases. Simulation results back these conclusions

A computationally efficient mixed-integer linear formulation for the thermal unit commitment problem

This book presents the latest findings on one of the most intensely investigated subjects in computational mathematics--the traveling salesman problem. It sounds simple enough: given a set of cities and the cost of travel between each pair of them, the problem challenges you to find the cheapest route by which to visit all the cities and return home to where you began. Though seemingly modest, this exercise has inspired studies by mathematicians, chemists, and physicists. Teachers use it in the classroom. It has practical applications in genetics, telecommunications, and neuroscience.The authors of this book are the same pioneers who for nearly two decades have led the investigation into the traveling salesman problem. They have derived solutions to almost eighty-six thousand cities, yet a general solution to the problem has yet to be discovered. Here they describe the method and computer code they used to solve a broad range of large-scale problems, and along the way they demonstrate the interplay of applied mathematics with increasingly powerful computing platforms. They also give the fascinating history of the problem--how it developed, and why it continues to intrigue us.

The Traveling Salesman Problem: A Computational Study (Princeton Series in Applied Mathematics)

This paper is an invited contribution to the 50th anniversary issue of the journalOperations Research, published by the Institute of Operations Research and Management Science (INFORMS). It describes one person's perspective on the development of computational tools for linear programming. The paper begins with a short personal history, followed by historical remarks covering the some 40 years of linear-programming developments that predate my own involvement in this subject. It concludes with a more detailed look at the evolution of computational linear programming since 1987.

/pdf/solving-real-world-linear-programs-a-decade-and-more-of-jzxvnfwv41.pdf

Solving Real-World Linear Programs: A Decade and More of Progress

This paper reports on the fifth version of theMixed Integer Programming Library. The miplib 2010 is the first miplib release that has been assembled by a large group from academia and from industry, all of whom work in integer programming. There was mutual consent that the concept of the library had to be expanded in order to fulfill the needs of the community. The new version comprises 361 instances sorted into several groups. This includes the main benchmark test set of 87 instances, which are all solvable by today’s codes, and also the challenge test set with 164 instances, many of which are currently unsolved. For the first time, we include scripts to run automated tests in a predefined way. Further, there is a solution checker to test the accuracy of provided solutions using exact arithmetic.

MIPLIB 2010 - Mixed Integer Programming Library version 5

For many years the principal solution technique used in the practice of mixed-integer programming has remained largely unchanged: Linear programming based branch-and-bound, introduced by Land and Doig (1960). This, in spite of the fact that there has been significant progress in the theory of integer programming and in the closely related field of combinatorial optimization. Many of the ideas developed there have received extensive computational testing, but, until recently, relatively little of that work has made it into the commercial codes used by practitioners. That situation has now changed. Several such codes, among them LINGO1, OSL2, and XPRESS-MP3, as well as the CPLEX4 code studied in this paper, now include cutting-plane capabilities as well as other ideas from the backlog of accumulated theory. As suggested by the title of this paper, the gap between theory and practice is indeed closing.

/pdf/mip-theory-and-practice-closing-the-gap-20jbxux2ap.pdf

MIP: Theory and Practice - Closing the Gap

TSPLIB is Gerhard Reinelt''s library of some hundred instances of the traveling salesman problem. Some of these instances arise from drilling holes in printed circuit boards; others arise from X-ray crystallography; yet others have been constructed artificially. None of them (with a single exception) is contrived to be hard and none of them is contrived to be easy; their sizes range from 17 to 85,900 cities; some of them have been solved and others have not. We have solved twenty previously unsolved problems from the TSPLIB. One of them is the problem with 225 cities that was contrived to be hard; the sizes of the remaining nineteen range from 1,000 to 7,397 cities. Like all the successful computer programs for solving the TSP, our computer program follows the scheme designed by George Dantzig, Ray Fulkerson, and Selmer Johnson in the early nineteen-fifties. The purpose of this preliminary report is to describe *some* of our innovations in implementing the Dantzig-Fulkerson-Johnson scheme; we are planning to write up a more comprehensive account of our work soon.

Robert E. Bixby

Papers

The Traveling Salesman Problem: A Computational Study (Princeton Series in Applied Mathematics)

Solving Real-World Linear Programs: A Decade and More of Progress

MIPLIB 2010 - Mixed Integer Programming Library version 5

MIP: Theory and Practice - Closing the Gap

Finding Cuts in the TSP (A preliminary report)