Playing with Duality: An overview of recent primal?dual approaches for solving large-scale optimization problems

TLDR: In this article, the authors present the principles of primal?dual approaches while providing an overview of the numerical methods that have been proposed in different contexts, including convex analysis, discrete optimization, parallel processing, and nonsmooth optimization with an emphasis on sparsity issues.

Abstract: Optimization methods are at the core of many problems in signal/image processing, computer vision, and machine learning. For a long time, it has been recognized that looking at the dual of an optimization problem may drastically simplify its solution. However, deriving efficient strategies that jointly bring into play the primal and dual problems is a more recent idea that has generated many important new contributions in recent years. These novel developments are grounded in the recent advances in convex analysis, discrete optimization, parallel processing, and nonsmooth optimization with an emphasis on sparsity issues. In this article, we aim to present the principles of primal?dual approaches while providing an overview of the numerical methods that have been proposed in different contexts. Last but not least, primal?dual methods lead to algorithms that are easily parallelizable. Today, such parallel algorithms are becoming increasingly important for efficiently handling high-dimensional problems.