/pdf/convex-analysisnoer-san-nojin-zhan-nituite-5dl2rbmoyr.pdf

Convex Analysisの二,三の進展について

This monograph is about a class of optimization algorithms called proximal algorithms. Much like Newton's method is a standard tool for solving unconstrained smooth optimization problems of modest size, proximal algorithms can be viewed as an analogous tool for nonsmooth, constrained, large-scale, or distributed versions of these problems. They are very generally applicable, but are especially well-suited to problems of substantial recent interest involving large or high-dimensional datasets. Proximal methods sit at a higher level of abstraction than classical algorithms like Newton's method: the base operation is evaluating the proximal operator of a function, which itself involves solving a small convex optimization problem. These subproblems, which generalize the problem of projecting a point onto a convex set, often admit closed-form solutions or can be solved very quickly with standard or simple specialized methods. Here, we discuss the many different interpretations of proximal operators and algorithms, describe their connections to many other topics in optimization and applied mathematics, survey some popular algorithms, and provide a large number of examples of proximal operators that commonly arise in practice.

Proximal Algorithms

We propose a new first-order splitting algorithm for solving jointly the primal and dual formulations of large-scale convex minimization problems involving the sum of a smooth function with Lipschitzian gradient, a nonsmooth proximable function, and linear composite functions. This is a full splitting approach, in the sense that the gradient and the linear operators involved are applied explicitly without any inversion, while the nonsmooth functions are processed individually via their proximity operators. This work brings together and notably extends several classical splitting schemes, like the forward–backward and Douglas–Rachford methods, as well as the recent primal–dual method of Chambolle and Pock designed for problems with linear composite terms.

/pdf/a-primal-dual-splitting-method-for-convex-optimization-21pmpnr51c.pdf

A primal-dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms

We introduce a first-order method for solving very large convex cone programs. The method uses an operator splitting method, the alternating directions method of multipliers, to solve the homogeneous self-dual embedding, an equivalent feasibility problem involving finding a nonzero point in the intersection of a subspace and a cone. This approach has several favorable properties. Compared to interior-point methods, first-order methods scale to very large problems, at the cost of requiring more time to reach very high accuracy. Compared to other first-order methods for cone programs, our approach finds both primal and dual solutions when available or a certificate of infeasibility or unboundedness otherwise, is parameter free, and the per-iteration cost of the method is the same as applying a splitting method to the primal or dual alone. We discuss efficient implementation of the method in detail, including direct and indirect methods for computing projection onto the subspace, scaling the original problem data, and stopping criteria. We describe an open-source implementation, which handles the usual (symmetric) nonnegative, second-order, and semidefinite cones as well as the (non-self-dual) exponential and power cones and their duals. We report numerical results that show speedups over interior-point cone solvers for large problems, and scaling to very large general cone programs.

/pdf/conic-optimization-via-operator-splitting-and-homogeneous-1fxr7pcmul.pdf

Conic Optimization via Operator Splitting and Homogeneous Self-Dual Embedding

In this work we present a novel method for the challenging problem of depth image up sampling. Modern depth cameras such as Kinect or Time-of-Flight cameras deliver dense, high quality depth measurements but are limited in their lateral resolution. To overcome this limitation we formulate a convex optimization problem using higher order regularization for depth image up sampling. In this optimization an an isotropic diffusion tensor, calculated from a high resolution intensity image, is used to guide the up sampling. We derive a numerical algorithm based on a primal-dual formulation that is efficiently parallelized and runs at multiple frames per second. We show that this novel up sampling clearly outperforms state of the art approaches in terms of speed and accuracy on the widely used Middlebury 2007 datasets. Furthermore, we introduce novel datasets with highly accurate ground truth, which, for the first time, enable to benchmark depth up sampling methods using real sensor data.

/pdf/image-guided-depth-upsampling-using-anisotropic-total-2o0jkxsrb8.pdf

Image Guided Depth Upsampling Using Anisotropic Total Generalized Variation

We generalize the primal-dual hybrid gradient (PDHG) algorithm proposed by Zhu and Chan in [An Efficient Primal-Dual Hybrid Gradient Algorithm for Total Variation Image Restoration, CAM Report 08-34, UCLA, Los Angeles, CA, 2008] to a broader class of convex optimization problems. In addition, we survey several closely related methods and explain the connections to PDHG. We point out convergence results for a modified version of PDHG that has a similarly good empirical convergence rate for total variation (TV) minimization problems. We also prove a convergence result for PDHG applied to TV denoising with some restrictions on the PDHG step size parameters. We show how to interpret this special case as a projected averaged gradient method applied to the dual functional. We discuss the range of parameters for which these methods can be shown to converge. We also present some numerical comparisons of these algorithms applied to TV denoising, TV deblurring, and constrained $l_1$ minimization problems.

A General Framework for a Class of First Order Primal-Dual Algorithms for Convex Optimization in Imaging Science

We propose an extended full-waveform inversion formulation that includes general convex constraints on the model. Though the full problem is highly nonconvex, the overarching optimization scheme arrives at geologically plausible results by solving a sequence of relaxed and warm-started constrained convex subproblems. The combination of box, total variation, and successively relaxed asymmetric total variation constraints allows us to steer free from parasitic local minima while keeping the estimated physical parameters laterally continuous and in a physically realistic range. For accurate starting models, numerical experiments carried out on the challenging 2004 BP velocity benchmark demonstrate that bound and total variation constraints improve the inversion result significantly by removing inversion artifacts, related to source encoding, and by clearly improved delineation of top, bottom, and flanks of a high-velocity high-contrast salt inclusion. The experiments also show that for poor starting models t...

Total Variation Regularization Strategies in Full-Waveform Inversion

Given appropriate data acquisition, processing to remove nonprimary arrivals, and use of an accurate migration algorithm, it is the quality of the subsurface velocity model that typically controls the quality of imaging that can be obtained from salt-affected seismic data. Full-waveform inversion has the potential to improve the accuracy, resolution, repeatability, and speed with which such velocity models can be generated, but, in the absence of an accurate starting model, that potential is difficult to realize in practice. Presented are successful inversion results, obtained from synthetic subsalt models, using a robust full-waveform inversion code that includes constraints upon the set of allowable earth models. These constraints include limitations on the total variation of the velocity of the model and, most significantly, on the asymmetric variation of velocity with depth such that negative velocity excursions are limited. During the iteration, these constraints are relaxed progressively so that the final model is driven principally by the seismic data, but the constraints act to steer the inversion path away from local minima in its early stages. This methodology is applied to portions of the 2004 BP benchmark and Phase I SEAM salt models, recovering an accurate model of the salt body, including its base and flanks, and an accurate model of the subsalt velocity structure, starting from one-dimensional velocity models that are severely cycle skipped. This approach removes entirely the requirement to pick salt boundaries from migrated seismic data, and acts as a form of automatic salt and sediment flooding during full-waveform inversion.

Constrained waveform inversion for automatic salt flooding

Time-lapse seismic is a powerful technology for monitoring a variety of subsurface changes due to reservoir fluid flow. However, the practice can be technically challenging when one seeks to acquire colocated time-lapse surveys with high degrees of replicability among the shot locations. We have determined that under “ideal” circumstances, in which we ignore errors related to taking measurements off the grid, high-quality prestack data can be obtained from randomized subsampled measurements that are observed from surveys in which we choose not to revisit the same randomly subsampled on-the-grid shot locations. Our acquisition is low cost because our measurements are subsampled. We have found that the recovered finely sampled prestack baseline and monitor data actually improve significantly when the same on-the-grid shot locations are not revisited. We achieve this result by using the fact that different time-lapse data share information and that nonreplicated (on-the-grid) acquisitions can add inf...

Low-cost time-lapse seismic with distributed compressive sensing — Part 1: Exploiting common information among the vintages

Summary We present a novel adaptation of a recently developed relatively simple iterative algorithm to solve large-scale sparsity-promoting optimization problems. Our algorithm is particularly suitable to large-scale geophysical inversion problems, such as sparse least-squares reverse-time migration or Kirchoff migration since it allows for a tradeoff between parallel computations, memory allocation, and turnaround times, by working on subsets of the data with different sizes. Comparison of the proposed method for sparse least-squares imaging shows a performance that rivals and even exceeds the performance of state-of-the art one-norm solvers that are able to carry out least-squares migration at the cost of a single migration with all data.

Ernie Esser

Papers

A General Framework for a Class of First Order Primal-Dual Algorithms for Convex Optimization in Imaging Science

Total Variation Regularization Strategies in Full-Waveform Inversion

Constrained waveform inversion for automatic salt flooding

Low-cost time-lapse seismic with distributed compressive sensing — Part 1: Exploiting common information among the vintages

Fast "Online" Migration with Compressive Sensing