The formalism of probabilistic graphical models provides a unifying framework for capturing complex dependencies among random variables, and building large-scale multivariate statistical models. Graphical models have become a focus of research in many statistical, computational and mathematical fields, including bioinformatics, communication theory, statistical physics, combinatorial optimization, signal and image processing, information retrieval and statistical machine learning. Many problems that arise in specific instances — including the key problems of computing marginals and modes of probability distributions — are best studied in the general setting. Working with exponential family representations, and exploiting the conjugate duality between the cumulant function and the entropy for exponential families, we develop general variational representations of the problems of computing likelihoods, marginal probabilities and most probable configurations. We describe how a wide variety of algorithms — among them sum-product, cluster variational methods, expectation-propagation, mean field methods, max-product and linear programming relaxation, as well as conic programming relaxations — can all be understood in terms of exact or approximate forms of these variational representations. The variational approach provides a complementary alternative to Markov chain Monte Carlo as a general source of approximation methods for inference in large-scale statistical models.

/pdf/graphical-models-exponential-families-and-variational-3i6dwlfta1.pdf

Graphical Models, Exponential Families, and Variational Inference

We present randomized approximation algorithms for the maximum cut (MAX CUT) and maximum 2-satisfiability (MAX 2SAT) problems that always deliver solutions of expected value at least.87856 times the optimal value. These algorithms use a simple and elegant technique that randomly rounds the solution to a nonlinear programming relaxation. This relaxation can be interpreted both as a semidefinite program and as an eigenvalue minimization problem. The best previously known approximation algorithms for these problems had performance guarantees of 1/2 for MAX CUT and 3/4 or MAX 2SAT. Slight extensions of our analysis lead to a.79607-approximation algorithm for the maximum directed cut problem (MAX DICUT) and a.758-approximation algorithm for MAX SAT, where the best previously known approximation algorithms had performance guarantees of 1/4 and 3/4, respectively. Our algorithm gives the first substantial progress in approximating MAX CUT in nearly twenty years, and represents the first use of semidefinite programming in the design of approximation algorithms.

/pdf/improved-approximation-algorithms-for-maximum-cut-and-3utsb8qos4.pdf

Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming

This review summarizes recent developments in the theory of spin glasses, as well as pertinent experimental data. The most characteristic properties of spin glass systems are described, and related phenomena in other glassy systems (dielectric and orientational glasses) are mentioned. The Edwards-Anderson model of spin glasses and its treatment within the replica method and mean-field theory are outlined, and concepts such as "frustration," "broken replica symmetry," "broken ergodicity," etc., are discussed. The dynamic approach to describing the spin glass transition is emphasized. Monte Carlo simulations of spin glasses and the insight gained by them are described. Other topics discussed include site-disorder models, phenomenological theories for the frozen phase and its excitations, phase diagrams in which spin glass order and ferromagnetism or antiferromagnetism compete, the Ne\'el model of superparamagnetism and related approaches, and possible connections between spin glasses and other topics in the theory of disordered condensed-matter systems.

Spin glasses: Experimental facts, theoretical concepts, and open questions

Facility location and supply chain management - A review

We provide Ising formulations for many NP-complete and NP-hard problems, including all of Karp's 21 NP-complete problems This collects and extends mappings to the Ising model from partitioning, covering and satisfiability In each case, the required number of spins is at most cubic in the size of the problem This work may be useful in designing adiabatic quantum optimization algorithms

Ising formulations of many NP problems

In a spin glass with Ising spins, the problems of computing the magnetic partition function and finding a ground state are studied. In a finite two-dimensional lattice these problems can be solved by algorithms that require a number of steps bounded by a polynomial function of the size of the lattice. In contrast to this fact, the same problems are shown to belong to the class of NP-hard problems, both in the two-dimensional case within a magnetic field, and in the three-dimensional case. NP-hardness of a problem suggests that it is very unlikely that a polynomial algorithm could exist to solve it.

/pdf/on-the-computational-complexity-of-ising-spin-glass-models-9y5mjo8v6b.pdf

On the computational complexity of Ising spin glass models

We study the problem of finding ground states of spin glasses with exterior magnetic field, and the problem of minimizing the number of vias holes on a printed circuit board, or contacts on a chip subject to pin preassignments and layer preferences. The former problem comes up in solid-state physics, and the latter in very-large-scale-integrated VLSI circuit design and in printed circuit board design. Both problems can be reduced to the max-cut problem in graphs. Based on a partial characterization of the cut polytope, we design a cutting plane algorithm and report on computational experience with it. Our method has been used to solve max-cut problems on graphs with up to 1,600 nodes.

An application of combinatorial optimization to statistical physics and circuit layout design

The cut polytopeP C (G) of a graphG=(V, E) is the convex hull of the incidence vectors of all edge sets of cuts ofG. We show some classes of facet-defining inequalities ofP C (G). We describe three methods with which new facet-defining inequalities ofP C (G) can be constructed from known ones. In particular, we show that inequalities associated with chordless cycles define facets of this polytope; moreover, for these inequalities a polynomial algorithm to solve the separation problem is presented. We characterize the facet defining inequalities ofP C (G) ifG is not contractible toK 5. We give a simple characterization of adjacency inP C (G) and prove that for complete graphs this polytope has diameter one and thatP C (G) has the Hirsch property. A relationship betweenP C (G) and the convex hull of incidence vectors of balancing edge sets of a signed graph is studied.

On the cut polytope

We present an extension to the subgradient algorithm to produce primal as well as dual solutions. It can be seen as a fast way to carry out an approximation of Dantzig-Wolfe decomposition. This gives a fast method for producing approximations for large scale linear programs. It is based on a new theorem in linear programming duality. We present successful experience with linear programs coming from set partitioning, set covering, max-cut and plant location.

The volume algorithm: producing primal solutions with a subgradient method

We present computational experience with a cutting plane algorithm for 0–1 quadratic programming without constraints. Our approach is based on a reduction of this problem to a max-cut problem in a graph and on a partial linear description of the cut polytope.

Francisco Barahona

Papers

On the computational complexity of Ising spin glass models

An application of combinatorial optimization to statistical physics and circuit layout design

On the cut polytope

The volume algorithm: producing primal solutions with a subgradient method

Experiments in quadratic 0-1 programming