Handbook of Approximation: Algorithms and Metaheuristics

Online algorithms are a hallmark of worst case optimization under uncertainty. On the other hand, in practice, the input is often far from worst case, and has some predictable characteristics. A recent line of work has shown how to use machine learned predictions to circumvent strong lower bounds on competitive ratios in classic online problems such as ski rental and caching. We study how predictive techniques can be used to break through worst case barriers in online scheduling. The makespan minimization problem with restricted assignments is a classic problem in online scheduling theory. Worst case analysis of this problem gives Ω(log m) lower bounds on the competitive ratio in the online setting. We identify a robust quantity that can be predicted and then used to guide online algorithms to achieve better performance. Our predictions are compact in size, having dimension linear in the number of machines, and can be learned using standard off the shelf methods. The performance guarantees of our algorithms depend on the accuracy of the predictions, given predictions with error η, we show how to construct O(log η) competitive fractional assignments. We then give an online algorithm that rounds any fractional assignment into an integral schedule. Our algorithm is O((log log m)3)-competitive and we give a nearly matching Ω(log log m) lower bound for online rounding algorithms.1 Altogether, we give algorithms that, equipped with predictions with error η, achieve O(log η (log log m)3) competitive ratios, breaking the Ω(log m) lower bound even for moderately accurate predictions.

/pdf/online-scheduling-via-learned-weights-racw5dsegn.pdf

Online scheduling via learned weights

We study the general integer programming problem where the number of variables $n$ is a variable part of the input. We consider two natural parameters of the constraint matrix $A$: its numeric measure $a$ and its sparsity measure $d$. We show that integer programming can be solved in time $g(a,d)\textrm{poly}(n,L)$, where $g$ is some computable function of the parameters $a$ and $d$, and $L$ is the binary encoding length of the input. In particular, integer programming is fixed-parameter tractable parameterized by $a$ and $d$, and is solvable in polynomial time for every fixed $a$ and $d$. Our results also extend to nonlinear separable convex objective functions. Moreover, for linear objectives, we derive a strongly-polynomial algorithm, that is, with running time $g(a,d)\textrm{poly}(n)$, independent of the rest of the input data. 
We obtain these results by developing an algorithmic framework based on the idea of iterative augmentation: starting from an initial feasible solution, we show how to quickly find augmenting steps which rapidly converge to an optimum. A central notion in this framework is the Graver basis of the matrix $A$, which constitutes a set of fundamental augmenting steps. The iterative augmentation idea is then enhanced via the use of other techniques such as new and improved bounds on the Graver basis, rapid solution of integer programs with bounded variables, proximity theorems and a new proximity-scaling algorithm, the notion of a reduced objective function, and others. 
As a consequence of our work, we advance the state of the art of solving block-structured integer programs. In particular, we develop near-linear time algorithms for $n$-fold, tree-fold, and $2$-stage stochastic integer programs. We also discuss some of the many applications of these classes.

An Algorithmic Theory of Integer Programming

We consider integer programming problems in standard form $\max \{c^Tx : Ax = b, \, x\geq 0, \, x \in Z^n\}$ where $A \in Z^{m \times n}$, $b \in Z^m$ and $c \in Z^n$. We show that such an integer program can be solved in time $(m \Delta)^{O(m)} \cdot \|b\|_\infty^2$, where $\Delta$ is an upper bound on each absolute value of an entry in $A$. This improves upon the longstanding best bound of Papadimitriou (1981) of $(m\cdot \Delta)^{O(m^2)}$, where in addition, the absolute values of the entries of $b$ also need to be bounded by $\Delta$. Our result relies on a lemma of Steinitz that states that a set of vectors in $R^m$ that is contained in the unit ball of a norm and that sum up to zero can be ordered such that all partial sums are of norm bounded by $m$. We also use the Steinitz lemma to show that the $\ell_1$-distance of an optimal integer and fractional solution, also under the presence of upper bounds on the variables, is bounded by $m \cdot (2\,m \cdot \Delta+1)^m$. Here $\Delta$ is again an upper bound on the absolute values of the entries of $A$. The novel strength of our bound is that it is independent of $n$. We provide evidence for the significance of our bound by applying it to general knapsack problems where we obtain structural and algorithmic results that improve upon the recent literature.

Proximity results and faster algorithms for Integer Programming using the Steinitz Lemma

We consider integer programming problems in standard form max l cTx : Ax = b, xg 0, x ∈ Znr where A ∈ Zm × n, b ∈ Zm, and c ∈ Zn. We show that such an integer program can be solved in time (m ⋅ Δ)O(m) ⋅ \Vert b\Vert∞2, where Δ is an upper bound on each absolute value of an entry in A. This improves upon the longstanding best bound of Papadimitriou [27] of (m⋅ Δ)O(m2), where in addition, the absolute values of the entries of b also need to be bounded by Δ. Our result relies on a lemma of Steinitz that states that a set of vectors in Rm that is contained in the unit ball of a norm and that sum up to zero can be ordered such that all partial sums are of norm bounded by m.We also use the Steinitz lemma to show that the e1-distance of an optimal integer and fractional solution, also under the presence of upper bounds on the variables, is bounded by m ⋅ (2,m⋅ Δ +1)m. Here Δ is again an upper bound on the absolute values of the entries of A. The novel strength of our bound is that it is independent of n.We provide evidence for the significance of our bound by applying it to general knapsack problems where we obtain structural and algorithmic results that improve upon the recent literature.

Proximity Results and Faster Algorithms for Integer Programming Using the Steinitz Lemma

As power management has become a primary concern in modern data centers, computing resources are being scaled dynamically to minimize energy consumption. We initiate the study of a variant of the classic online speed scaling problem, in which machine learning predictions about the future can be integrated naturally. Inspired by recent work on learning-augmented online algorithms, we propose an algorithm which incorporates predictions in a black-box manner and outperforms any online algorithm if the accuracy is high, yet maintains provable guarantees if the prediction is very inaccurate. We provide both theoretical and experimental evidence to support our claims.

/pdf/learning-augmented-energy-minimization-via-speed-scaling-23br6mv692.pdf

Learning Augmented Energy Minimization via Speed Scaling

Integer programs with a fixed number of constraints can be solved in pseudo-polynomial time. We present a surprisingly simple algorithm and matching conditional lower bounds. Consider an IP in standard form $\max\{c^T x : A x = b, x\in \mathbb Z^n_{\ge 0}\}$, where $A\in \mathbb Z^{m\times n}, b\in \mathbb Z^m$ and $c\in \mathbb Z^n$. Let $\Delta$ be an upper bound on the absolute values in $A$. We show that this IP can be solved in time $O(m\Delta)^{2m} \cdot \log(\lVert b \rVert_\infty)$. The previous best algorithm has a running time of $n\cdot O(m\Delta)^{2m}\cdot \lVert b \rVert_1^2$. 
The hardness of (min, +)-convolution has been used to prove conditional lower bounds on a number of polynomially solvable problems. We show that improving our algorithm for IPs of any fixed number of constraints is equivalent to improving (min, +)-convolution. More precisely, for any fixed $m$ there exists an algorithm for solving IPs with $m$ constraints in time $O(m(\Delta + \lVert b \rVert_\infty))^{2m-\delta}$ for some $\delta > 0$, if and only if there is a truly sub-quadratic algorithm for (min, +)-convolution. 
For the feasibility problem, where the IP has no objective function, we improve the running time to $O(m\Delta)^{m} \log(\Delta) \log (\Delta + \lVert b \rVert_\infty)$. We also give a matching lower bound here: For every fixed $m$ and $\delta > 0$ there is no algorithm for testing feasibility of IPs with $m$ constraints in time $n^{O(1)} \cdot O(m(\Delta + \lVert b \rVert_\infty))^{m - \delta}$ unless the SETH is false.

On Integer Programming and Convolution

We consider the classical problem of Scheduling on Unrelated Machines. In this problem a set of jobs is to be distributed among a set of machines and the maximum load (makespan) is to be minimized. The processing time pij of a job j depends on the machine i it is assigned to. Lenstra, Shmoys and Tardos gave a polynomial time 2-approximation for this problem [8]. In this paper we focus on a prominent special case, the Restricted Assignment problem, in which pij ∈ {pj, ∞}. The configuration-LP is a linear programming relaxation for the Restricted Assignment problem. It was shown by Svensson that the multiplicative gap between integral and fractional solution, the integrality gap, is at most 2 − 1/17 ≈ 1.9412 [11]. In this paper we significantly simplify his proof and achieve a bound of 2 − 1/6 ≈ 1.8333. As a direct consequence this provides a polynomial (2 − 1/6 + ϵ)-estimation algorithm for the Restricted Assignment problem by approximating the configuration-LP. The best lower bound known for the integrality gap is 1.5 and no estimation algorithm with a guarantee better than 1.5 exists unless P = NP.

On the configuration-LP of the restricted assignment problem

Integer programs with a constant number of constraints are solvable in pseudo-polynomial time. We give a new algorithm with a better pseudo-polynomial running time than previous results. Moreover, we establish a strong connection to the problem (min, +)-convolution. (min, +)-convolution has a trivial quadratic time algorithm and it has been conjectured that this cannot be improved significantly. We show that further improvements to our pseudo-polynomial algorithm for any fixed number of constraints are equivalent to improvements for (min, +)-convolution. This is a strong evidence that our algorithm's running time is the best possible. We also present a faster specialized algorithm for testing feasibility of an integer program with few constraints and for this we also give a tight lower bound, which is based on the SETH.

On Integer Programming, Discrepancy, and Convolution

We study an important case of ILPs $\max\{c^Tx \ \vert\ \mathcal Ax = b, l \leq x \leq u,\, x \in \mathbb{Z}^{n t} \} $ with $n\cdot t$ variables and lower and upper bounds $\ell, u\in\mathbb Z^{nt}$. In $n$-fold ILPs non-zero entries only appear in the first $r$ rows of the matrix $\mathcal A$ and in small blocks of size $s\times t$ along the diagonal underneath. Despite this restriction many optimization problems can be expressed in this form. It is known that $n$-fold ILPs can be solved in FPT time regarding the parameters $s, r,$ and $\Delta$, where $\Delta$ is the greatest absolute value of an entry in $\mathcal A$. The state-of-the-art technique is a local search algorithm that subsequently moves in an improving direction. Both, the number of iterations and the search for such an improving direction take time $\Omega(n)$, leading to a quadratic running time in $n$. We introduce a technique based on Color Coding, which allows us to compute these improving directions in logarithmic time after a single initialization step. This leads to the first algorithm for $n$-fold ILPs with a running time that is near-linear in the number $nt$ of variables, namely $(rs\Delta)^{O(r^2s + s^2)} L^2 \cdot nt \log^{O(1)}(nt)$, where $L$ is the encoding length of the largest integer in the input. In contrast to the algorithms in recent literature, we do not need to solve the LP relaxation in order to handle unbounded variables. Instead, we give a structural lemma to introduce appropriate bounds. If, on the other hand, we are given such an LP solution, the running time can be decreased by a factor of $L$.

Lars Rohwedder

Papers

Learning Augmented Energy Minimization via Speed Scaling

On Integer Programming and Convolution

On the configuration-LP of the restricted assignment problem

On Integer Programming, Discrepancy, and Convolution

Near-Linear Time Algorithm for n-fold ILPs via Color Coding