to be done in this area. Face recognition is a problem that scarcely clamoured for attention before the computer age but, having surfaced, has involved a wide range of techniques and has attracted the attention of some fine minds (David Mumford was a Fields Medallist in 1974). This singular application of mathematical modelling to a messy applied problem of obvious utility and importance but with no unique solution is a pretty one to share with students: perhaps, returning to the source of our opening quotation, we may invert Duncan's earlier observation, 'There is an art to find the mind's construction in the face!'.

Linear and nonlinear waves, by G. B. Whitham. Pp.636. £50. 1999. ISBN 0 471 35942 4 (Wiley).

This monograph presents the main complexity theorems in convex optimization and their corresponding algorithms. Starting from the fundamental theory of black-box optimization, the material progresses towards recent advances in structural optimization and stochastic optimization. Our presentation of black-box optimization, strongly influenced by the seminal book of Nesterov, includes the analysis of cutting plane methods, as well as accelerated gradient descent schemes. We also pay special attention to non-Euclidean settings relevant algorithms include Frank-Wolfe, mirror descent, and dual averaging and discuss their relevance in machine learning. We provide a gentle introduction to structural optimization with FISTA to optimize a sum of a smooth and a simple non-smooth term, saddle-point mirror prox Nemirovski's alternative to Nesterov's smoothing, and a concise description of interior point methods. In stochastic optimization we discuss stochastic gradient descent, mini-batches, random coordinate descent, and sublinear algorithms. We also briefly touch upon convex relaxation of combinatorial problems and the use of randomness to round solutions, as well as random walks based methods.

/pdf/convex-optimization-algorithms-and-complexity-4uuru7i83p.pdf

Convex Optimization: Algorithms and Complexity

Introduction. 1. Mathematical Preliminaries. Submodular Systems and Base Polyhedra. 2. From Matroids to Submodular Systems. Matroids. Polymatroids. Submodular Systems. 3. Submodular Systems and Base Polyhedra. Fundamental Operations on Submodular Systems. Greedy Algorithm. Structures of Base Polyhedra. Intersecting- and Crossing-Submodular Functions. Related Polyhedra. Submodular Systems of Network Type. Neoflows. 4. The Intersection Problem. The Intersection Theorem. The Discrete Separation Theorem. The Common Base Problem. 5. Neoflows. The Equivalence of the Neoflow Problems. Feasibility for Submodular Flows. Optimality for Submodular Flows. Algorithms for Neoflows. Matroid Optimization. Submodular Analysis. 6. Submodular Functions and Convexity. Conjugate Functions and a Fenchel-Type Min-Max Theorem for Submodular and Supermodular Functions. Subgradients of Submodular Functions. The Lovasz Extensions of Submodular Functions. 7. Submodular Programs. Submodular Programs - Unconstrained Optimization. Submodular Programs - Constrained Optimization. Nonlinear Optimization with Submodular Constraints. 8. Separable Convex Optimization. Optimality Conditions. A Decomposition Algorithm. Discrete Optimization. 9. The Lexicographically Optimal Base Problem. Nonlinear Weight Functions. Linear Weight Functions. 10. The Weighted Max-Min and Min-Max Problems. Continuous Variables. Discrete Variables. 11. The Fair Resource Allocation Problem. Continuous Variables. Discrete Variables. 12. The Neoflow Problem with a Separable Convex Cost Function. References. Index.

Submodular functions and optimization

Recently, increased computational power and data availability, as well as algorithmic advances, have led machine learning (ML) techniques to impressive results in regression, classification, data generation and reinforcement learning tasks. Despite these successes, the proximity to the physical limits of chip fabrication alongside the increasing size of datasets is motivating a growing number of researchers to explore the possibility of harnessing the power of quantum computation to speed up classical ML algorithms. Here we review the literature in quantum ML and discuss perspectives for a mixed readership of classical ML and quantum computation experts. Particular emphasis will be placed on clarifying the limitations of quantum algorithms, how they compare with their best classical counterparts and why quantum resources are expected to provide advantages for learning problems. Learning in the presence of noise and certain computationally hard problems in ML are identified as promising directions for the field. Practical questions, such as how to upload classical data into quantum form, will also be addressed.

https://royalsocietypublishing.org/doi/pdf/10.1098/rspa.2017.0551

Quantum machine learning: a classical perspective.

A number of improvements are provided relating to computer aided surgery utilizing an on tool tracking system. The various improvements relate generally to both the methods used during computer aided surgery and the devices used during such procedures. Other improvements relate to the structure of the tools used during a procedure and how the tools can be controlled using the OTT device. Still other improvements relate to methods of providing feedback during a procedure to improve either the efficiency or quality, or both, for a procedure including the rate of and type of data processed depending upon a CAS mode.

On-board tool tracking system and methods of computer assisted surgery

In this paper we improve upon the running time for finding a point in a convex set given a separation oracle. In particular, given a separation oracle for a convex set K a#x2282; Rn that is contained in a box of radius R we show how to either compute a point in K or prove that K does not contain a ball of radius a#x03B5; using an expected O(n log(nR=a#x03B5;)) evaluations of the oracle and additional time O(n3 logO(1)(nR=a#x03B5;)). This matches the oracle complexity and improves upon the O(na#x03C9;+1 log(nR=a#x03B5;)) additional time of the previous fastest algorithm achieved over 25 years ago by Vaidya [91] for the current value of the matrix multiplication constant a#x03C9; < 2:373 [98], [36] when R=a#x03B5; = O(poly(n)). Using a mix of standard reductions and new techniques we show how our algorithm can be used to improve the running time for solving classic problems in continuous and combinatorial optimization. In particular we provide the following running time improvements: a#x03B5; Sub modular Function Minimization: n is the size of the ground set, M is the maximum absolute value of function values and EO is the time for function evaluation. Our weakly and strongly polynomial time algorithms have a running time of O(n2 lognM EO+n3 logO(1) nM) and O(n3 log2 n EO+n4 logO(1) n), improving upon the previous best of O((n4 · EO+n5) logM) and O(n5 · EO + n6) respectively. a#x03B5; Sub modular Flow: n = |V|, m = |E|, C is the maximum edge cost in absolute value and U is maximum edge capacity in absolute value. We obtain a faster weakly polynomial running time of O(n2 log nCU · EO + n3 logO(1) nCU), improving upon the previous best of O(mn5 log nU · EO) and O(n4h min {log C, log U}) from 15 years ago by a factor of O(n4). We also achieve faster strongly polynomial time algorithms as a consequence of our result on sub modular minimization. a#x03B5; Matroid Intersection: n is the size of the ground set, r is the maximum size of independent sets, M is the maximum absolute value of element weight, Trank and Tind are the time for each rank and independence oracle query. We obtain a running time of O((nr log2 nTrank+n3 logO(1) n) lognM) and O((n2 log nTind+n3 logO(1) n) lognM), achieving the first quadratic bound on the query complexity for the independence and rank oracles. In the unweighted case, this is the first improvement since 1986 for independence oracle. a#x03B5; Semi definite Programming: n is the number of constraints, m is the number of dimensions and S is the total number of non-zeros in the constraint matrices. We obtain a running time of O(n(n2 +ma#x03C9; +S)), improving upon the previous best of O(n(na#x03C9; +ma#x03C9; +S)) for the regime S is small.

A Faster Cutting Plane Method and its Implications for Combinatorial and Convex Optimization

Methods and apparatus for magnetic resonance imaging (MRI) assisted cryosurgery. Optimal probe placements and cooling parameters are calculated prior to cryosurgery using MRI data. A MRI compatible cryoprobe and a stereotactic probe positioning device are provided. The resolution of MR images is enhanced by mounting a radio frequency MR coil on the intracorporeal end of a cryoprobe. During cryosurgery the temperature distribution in the frozen region is solved by determining the boundary of the frozen region and solving the heat equation for the known boundary conditions. During cryosurgery the temperature distribution in the unfrozen region is determined by T1 measurements. The process of freezing is controlled using information from the solution of the energy equation in the frozen region and temperature measurements in the unfrozen region. After cryosurgery the extent of the tissue damage may be ascertained using phosphorus-31 and/or sodium-23 spectroscopy with a special coil set on the cryosurgical probe.

Magnetic resonance imaging assisted cryosurgery

Given a separation oracle for a convex set K ⊂ ℝ n that is contained in a box of radius R, the goal is to either compute a point in K or prove that K does not contain a ball of radius є. We propose a new cutting plane algorithm that uses an optimal O(n log(κ)) evaluations of the oracle and an additional O(n 2) time per evaluation, where κ = nR/є. This improves upon Vaidya’s O( SO · n log(κ) + n ω+1 log(κ)) time algorithm [Vaidya, FOCS 1989a] in terms of polynomial dependence on n, where ω O( SO · n log(κ) + n 3 log O(1) (κ)) time algorithm [Lee, Sidford and Wong, FOCS 2015] in terms of dependence on κ. For many important applications in economics, κ = Ω(exp(n)) and this leads to a significant difference between log(κ) and (log(κ)). We also provide evidence that the n 2 time per evaluation cannot be improved and thus our running time is optimal. A bottleneck of previous cutting plane methods is to compute leverage scores, a measure of the relative importance of past constraints. Our result is achieved by a novel multi-layered data structure for leverage score maintenance, which is a sophisticated combination of diverse techniques such as random projection, batched low-rank update, inverse maintenance, polynomial interpolation, and fast rectangular matrix multiplication. Interestingly, our method requires a combination of different fast rectangular matrix multiplication algorithms. Our algorithm not only works for the classical convex optimization setting, but also generalizes to convex-concave games. We apply our algorithm to improve the runtimes of many interesting problems, e.g., Linear Arrow-Debreu Markets, Fisher Markets, and Walrasian equilibrium.

An improved cutting plane method for convex optimization, convex-concave games, and its applications

We consider the generalizations of two classical problems, online bipartite matching and ski rental, in the field of online algorithms, and establish a novel connection between them.

Two-sided Online Bipartite Matching and Vertex Cover: Beating the Greedy Algorithm

We improve upon the running time for finding a point in a convex set given a separation oracle. In particular, given a separation oracle for a convex set $K\subset \mathbb{R}^n$ contained in a box of radius $R$, we show how to either find a point in $K$ or prove that $K$ does not contain a ball of radius $\epsilon$ using an expected $O(n\log(nR/\epsilon))$ oracle evaluations and additional time $O(n^3\log^{O(1)}(nR/\epsilon))$. This matches the oracle complexity and improves upon the $O(n^{\omega+1}\log(nR/\epsilon))$ additional time of the previous fastest algorithm achieved over 25 years ago by Vaidya for the current matrix multiplication constant $\omega<2.373$ when $R/\epsilon=n^{O(1)}$. 
Using a mix of standard reductions and new techniques, our algorithm yields improved runtimes for solving classic problems in continuous and combinatorial optimization: 
Submodular Minimization: Our weakly and strongly polynomial time algorithms have runtimes of $O(n^2\log nM\cdot\text{EO}+n^3\log^{O(1)}nM)$ and $O(n^3\log^2 n\cdot\text{EO}+n^4\log^{O(1)}n)$, improving upon the previous best of $O((n^4\text{EO}+n^5)\log M)$ and $O(n^5\text{EO}+n^6)$. 
Matroid Intersection: Our runtimes are $O(nrT_{\text{rank}}\log n\log (nM) +n^3\log^{O(1)}(nM))$ and $O(n^2\log (nM) T_{\text{ind}}+n^3 \log^{O(1)} (nM))$, achieving the first quadratic bound on the query complexity for the independence and rank oracles. In the unweighted case, this is the first improvement since 1986 for independence oracle. 
Submodular Flow: Our runtime is $O(n^2\log nCU\cdot\text{EO}+n^3\log^{O(1)}nCU)$, improving upon the previous bests from 15 years ago roughly by a factor of $O(n^4)$. 
Semidefinite Programming: Our runtime is $\tilde{O}(n(n^2+m^{\omega}+S))$, improving upon the previous best of $\tilde{O}(n(n^{\omega}+m^{\omega}+S))$ for the regime where the number of nonzeros $S$ is small.

Sam Chiu-wai Wong

Papers

A Faster Cutting Plane Method and its Implications for Combinatorial and Convex Optimization

Magnetic resonance imaging assisted cryosurgery

An improved cutting plane method for convex optimization, convex-concave games, and its applications

Two-sided Online Bipartite Matching and Vertex Cover: Beating the Greedy Algorithm

A Faster Cutting Plane Method and its Implications for Combinatorial and Convex Optimization