Numerical Optimization presents a comprehensive and up-to-date description of the most effective methods in continuous optimization. It responds to the growing interest in optimization in engineering, science, and business by focusing on the methods that are best suited to practical problems. For this new edition the book has been thoroughly updated throughout. There are new chapters on nonlinear interior methods and derivative-free methods for optimization, both of which are used widely in practice and the focus of much current research. Because of the emphasis on practical methods, as well as the extensive illustrations and exercises, the book is accessible to a wide audience. It can be used as a graduate text in engineering, operations research, mathematics, computer science, and business. It also serves as a handbook for researchers and practitioners in the field. The authors have strived to produce a text that is pleasant to read, informative, and rigorous - one that reveals both the beautiful nature of the discipline and its practical side.

Numerical Optimization

(1977). Graph Theory with Applications. Journal of the Operational Research Society: Vol. 28, Volume 28, issue 1, pp. 237-238.

https://link.springer.com/content/pdf/10.1057%2Fjors.1977.45.pdf

Graph theory with applications

We present a new family of subgradient methods that dynamically incorporate knowledge of the geometry of the data observed in earlier iterations to perform more informative gradient-based learning. Metaphorically, the adaptation allows us to find needles in haystacks in the form of very predictive but rarely seen features. Our paradigm stems from recent advances in stochastic optimization and online learning which employ proximal functions to control the gradient steps of the algorithm. We describe and analyze an apparatus for adaptively modifying the proximal function, which significantly simplifies setting a learning rate and results in regret guarantees that are provably as good as the best proximal function that can be chosen in hindsight. We give several efficient algorithms for empirical risk minimization problems with common and important regularization functions and domain constraints. We experimentally study our theoretical analysis and show that adaptive subgradient methods outperform state-of-the-art, yet non-adaptive, subgradient algorithms.

Adaptive Subgradient Methods for Online Learning and Stochastic Optimization.

Models for the processes by which ideas and influence propagate through a social network have been studied in a number of domains, including the diffusion of medical and technological innovations, the sudden and widespread adoption of various strategies in game-theoretic settings, and the effects of "word of mouth" in the promotion of new products. Recently, motivated by the design of viral marketing strategies, Domingos and Richardson posed a fundamental algorithmic problem for such social network processes: if we can try to convince a subset of individuals to adopt a new product or innovation, and the goal is to trigger a large cascade of further adoptions, which set of individuals should we target?We consider this problem in several of the most widely studied models in social network analysis. The optimization problem of selecting the most influential nodes is NP-hard here, and we provide the first provable approximation guarantees for efficient algorithms. Using an analysis framework based on submodular functions, we show that a natural greedy strategy obtains a solution that is provably within 63% of optimal for several classes of models; our framework suggests a general approach for reasoning about the performance guarantees of algorithms for these types of influence problems in social networks.We also provide computational experiments on large collaboration networks, showing that in addition to their provable guarantees, our approximation algorithms significantly out-perform node-selection heuristics based on the well-studied notions of degree centrality and distance centrality from the field of social networks.

/pdf/maximizing-the-spread-of-influence-through-a-social-network-3nnj0jic76.pdf

Maximizing the spread of influence through a social network

This is the second half of a two part series devoted to the tabu search metastrategy for optimization problems. Part I introduced the fundamental ideas of tabu search as an approach for guiding other heuristics to overcome the limitations of local optimality, both in a deterministic and a probabilistic framework. Part I also reported successful applications from a wide range of settings, in which tabu search frequently made it possible to obtain higher quality solutions than previously obtained with competing strategies, generally with less computational effort. Part II, in this issue, examines refinements and more advanced aspects of tabu search. Following a brief review of notation, Part II introduces new dynamic strategies for managing tabu lists, allowing fuller exploitation of underlying evaluation functions. In turn, the elements of staged search and structured move sets are characterized, which bear on the issue of finiteness. Three ways of applying tabu search to the solution of integer programmin...

https://leeds-faculty.colorado.edu/glover/ts - part ii-orsa-aw.pdf

Tabu Search—Part II

FOUNDATIONS. The Scope of Integer and Combinatorial Optimization. Linear Programming. Graphs and Networks. Polyhedral Theory. Computational Complexity. Polynomial-Time Algorithms for Linear Programming. Integer Lattices. GENERAL INTEGER PROGRAMMING. The Theory of Valid Inequalities. Strong Valid Inequalities and Facets for Structured Integer Programs. Duality and Relaxation. General Algorithms. Special-Purpose Algorithms. Applications of Special- Purpose Algorithms. COMBINATORIAL OPTIMIZATION. Integral Polyhedra. Matching. Matroid and Submodular Function Optimization. References. Indexes.

/pdf/integer-and-combinatorial-optimization-3vyexr8d0i.pdf

Integer and Combinatorial Optimization

The principles of integer programming are directed toward finding solutions to problems from the fields of economic planning, engineering design, and combinatorial optimization. This highly respected and much-cited text, a standard of graduate-level courses since 1972, presents a comprehensive treatment of the first two decades of research on integer programming.

Integer programming

LetN be a finite set andz be a real-valued function defined on the set of subsets ofN that satisfies z(S)+z(T)źz(SźT)+z(SźT) for allS, T inN. Such a function is called submodular. We consider the problem maxSźN{a(S):|S|≤K,z(S) submodular}.

Several hard combinatorial optimization problems can be posed in this framework. For example, the problem of finding a maximum weight independent set in a matroid, when the elements of the matroid are colored and the elements of the independent set can have no more thanK colors, is in this class. The uncapacitated location problem is a special case of this matroid optimization problem.

We analyze greedy and local improvement heuristics and a linear programming relaxation for this problem. Our results are worst case bounds on the quality of the approximations. For example, whenz(S) is nondecreasing andz(0) = 0, we show that a "greedy" heuristic always produces a solution whose value is at least 1 ź[(K ź 1)/K]K times the optimal value. This bound can be achieved for eachK and has a limiting value of (e ź 1)/e, where e is the base of the natural logarithm.

An analysis of approximations for maximizing submodular set functions--I

LetN be a finite set andz be a real-valued function defined on the set of subsets ofN that satisfies z(S)+z(T)gez(SxcupT)+z(SxcapT) for allS, T inN. Such a function is called submodular. We consider the problem maxSsubN{a(S):|S|leK,z(S) submodular}. Several hard combinatorial optimization problems can be posed in this framework. For example, the problem of finding a maximum weight independent set in a matroid, when the elements of the matroid are colored and the elements of the independent set can have no more thanK colors, is in this class. The uncapacitated location problem is a special case of this matroid optimization problem. We analyze greedy and local improvement heuristics and a linear programming relaxation for this problem. Our results are worst case bounds on the quality of the approximations. For example, whenz(S) is nondecreasing andz(0) = 0, we show that a ldquogreedyrdquo heuristic always produces a solution whose value is at least 1 –[(K – 1)/K] K times the optimal value. This bound can be achieved for eachK and has a limiting value of (e – 1)/e, where e is the base of the natural logarithm.

An analysis of approximations for maximizing submodular set functions II

We discuss formulations of integer programs with a huge number of variables and their solution by column generation methods, i.e., implicit pricing of nonbasic variables to generate new columns or to prove LP optimality at a node of the branch-and-bound tree. We present classes of models for which this approach decomposes the problem, provides tighter LP relaxations, and eliminates symmetry. We then discuss computational issues and implementation of column generation, branch-and-bound algorithms, including special branching rules and efficient ways to solve the LP relaxation. We also discuss the relationship with Lagrangian duality.

/pdf/branch-and-price-column-generation-for-solving-huge-integer-4jvlny6blr.pdf

George L. Nemhauser

Papers

Integer and Combinatorial Optimization

Integer programming

An analysis of approximations for maximizing submodular set functions--I

An analysis of approximations for maximizing submodular set functions II

Branch-And-Price: Column Generation for Solving Huge Integer Programs