Efficient Schemes for Total Variation Minimization Under Constraints in Image Processing

TLDR: New fast algorithms to minimize total variation and more generally $l^1$-norms under a general convex constraint and a recent advance in convex optimization proposed by Yurii Nesterov are presented.

Abstract: This paper presents new fast algorithms to minimize total variation and more generally $l^1$-norms under a general convex constraint. Such problems are standards of image processing. The algorithms are based on a recent advance in convex optimization proposed by Yurii Nesterov. Depending on the regularity of the data fidelity term, we solve either a primal problem or a dual problem. First we show that standard first-order schemes allow one to get solutions of precision $\epsilon$ in $O(\frac{1}{\epsilon^2})$ iterations at worst. We propose a scheme that allows one to obtain a solution of precision $\epsilon$ in $O(\frac{1}{\epsilon})$ iterations for a general convex constraint. For a strongly convex constraint, we solve a dual problem with a scheme that requires $O(\frac{1}{\sqrt{\epsilon}})$ iterations to get a solution of precision $\epsilon$. Finally we perform some numerical experiments which confirm the theoretical results on various problems of image processing.

Figures

Figure 3: Cartoon + texture decompositions. Top : Meyer’s model. Bottom : Osher-SoléVese’s model

Figure 6: Total variation VS number of iterations for µ ∈ {1e − 2, 5e − 3, 1e − 3}. PSD stands for Projected subgradient descent. PGD stands for projected gradient descent.

Figure 4: Cartoon + texture decompositions. Top : BV − l1 model. Bottom : result of minimizing the l1-norm of the g field in (3.45)

Figure 1: Example of image decompression - TL : Original image - TR : Compressed image using Daubechies 9-7 wavelet transform (the quantization coefficients are chosen by hand to outline the model features) - BL : Solution of (2.17) using µ = 0.01 - BR : Solution of (2.17) using µ = 0.001

Full text

HAL Id: inria-00166096
https://hal.inria.fr/inria-00166096v3
Submitted on 7 Mar 2008
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entic research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diusion de documents
scientiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
Ecient schemes for total variation minimization under
constraints in image processing
Pierre Weiss, Gilles Aubert, Laure Blanc-Féraud
To cite this version:
Pierre Weiss, Gilles Aubert, Laure Blanc-Féraud. Ecient schemes for total variation minimization
under constraints in image processing. [Research Report] RR-6260, INRIA. 2007, pp.36. �inria-
00166096v3�

apport 

de recherche
ISSN 0249-6399 ISRN INRIA/RR--6260--FR+ENG
Thème COG
INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET EN AUTOMATIQUE
Efficient schemes for total variation minimization
under constraints in image processing
Pierre Weiss — Gilles Aubert — Laure Blanc-Féraud
N° 6260
July 2007

Unité de recherche INRIA Sophia Antipolis
2004, route des Lucioles, BP 93, 06902 Sophia Antipolis Cedex (France)
Téléphone : +33 4 92 38 77 77 — Télécopie : +33 4 92 38 77 65
Efficient schemes for total variation minimization under
constraints in image processing
Pierre Weiss , Gilles Aubert , Laure Blanc-Féraud
Thème COG — Systèmes cognitifs
Projets Ariana
Rapp ort de recherche n° 6260 — July 2007 — 36 pages
Abstract: This paper presents new algorithms to minimize total variation and more g e n-
erally l
1
-norms under a general convex constraint. The algorithms are based on a r e c ent
advance in convex optimization proposed by Yurii Nesterov [34]. Depending on the regu-
larity of the data fidelity term, we solve either a primal problem, either a dual problem.
First we show that standard first order schemes allow to get solutions of precision ǫ in O(
1
ǫ
2
)
iterations at worst. For a genera l convex c onstraint, we propose a scheme that allows to
obtain a solution of precision ǫ in O(
1
ǫ
) iterations. For a strongly convex constraint, we solve
a dual problem with a scheme that requires O(
1
√
ǫ
) iterations to get a solution o f precision
ǫ. Thus, depending on the regularity of the data term, we gain from one to two order s of
magnitude in the convergence rates with respect to standard schemes. Finally we perform
some numerical ex periments which confirm the theoretical results on various problems.
Key-words: l
1
-norm minimization , total variation minimization, l
p
-norms, duality, gra-
dient and subgradient descents, Nesterov scheme, bounded and non-bounded noises, tex-
ture+geometry decomposition, complexity.

Algorithmes efficaces pour la minimisation de la
variation totale sous contraintes
Résumé : Ce papier présente de nouveaux algorithmes pour minimiser la variation to-
tale, et plus généralement des normes l
1
, sous des contraintes convexes. Ces algo rithmes
proviennent d’une avancée récente en optimisation convexe proposée par Yurii Nesterov.
Suivant la régularité de l’attache a ux données, nous résolvons soit un problème primal, so it
un problème dual. Première ment, nous montrons que les schémas standard de premier ordre
permettent d’obtenir des solutions de précision ǫ en O(
1
ǫ
2
) itérations au pire des cas. Pour
une contrainte convexe quelconque, nous proposons un schéma qui permet d’obtenir une
solution de précision ǫ en O(
1
ǫ
) itérations. Pour une contrainte fortement convexe, no us
résolvons un problème dual avec un schéma qui demande O(
1
√
ǫ
) itérations pour obtenir une
solution de précision ǫ. Suivant la contrainte, nous gagnons donc un à deux ordres dans
la rapidité de convergence par rapport à des approches standard. Finalement, nous faisons
quelques expériences numériques qui confirment les résultats théoriques sur de nombreux
problèmes.
Mots-clés : norme l
1
, minimisation variation totale, bruits bo rnés, bruits de compr e ssion,
décomposition texture + géométrie, dualité, s ous-gradient projeté, complexité, contraintes
convexes

Frequently asked questions

Q1. What contributions have the authors mentioned in the paper "Efficient schemes for total variation minimization under constraints in image processing" ?

This paper presents new algorithms to minimize total variation and more generally l-norms under a general convex constraint. First the authors show that standard first order schemes allow to get solutions of precision ǫ in O ( 1 ǫ2 ) iterations at worst. For a general convex constraint, the authors propose a scheme that allows to obtain a solution of precision ǫ in O ( 1ǫ ) iterations. Finally the authors perform some numerical experiments which confirm the theoretical results on various problems. 

Q2. What are some of the techniques that have been proposed to solve convex problems?

Those include subgradient descents [28], Newton-like methods [25], second order cone programming [21], interior point methods [19], or graph based approaches [15, 10]. 

Q3. How many iterations do the authors need to get the same result?

After 200 iterations, very little perceptual modifications are observed in all experiments, while a projected gradient descent requires around 1000 iterations to get the same result. 

Q4. What is the cost per iteration of this model?

For a 256 × 256 image, the cost per iteration is 0.2 seconds (we used the fft2 function of Matlab and implemented a C code with Matlab mex compiler for the projections). 

Q5. What is the main purpose of this paper?

In this paper the authors concentrate on the use of B = ∇, ||Bu||1 corresponding to the total variation (see the appendix for the discretization of differential operators). 

Q6. What is the definition of a function that is differentiable?

A function F defined on K is said to be L-Lipschitz differentiable if it is differentiableon K and that |∇F (u1) −∇F (u2)|2 ≤ L|u1 − u2|2 for any (u1, u2) ∈ K2.Definition 2.7. [Strongly convex differentiable function] 

Q7. What is the straightforward algorithm to solve for general convex function F?

Maybe the most straightforward algorithm to solve (1.1) for general convex function F , is the projected subgradient descent algorithm. 

Q8. What is the way to deblur the image?

To retrieve the original image, the authors can solve the following problem :inf u∈X,|h⋆u−f |2≤α (J(u)) (2.18)In view of Shannon’s theorem, one could think that the best way to zoom an image is to use a zero-padding technique. 

Q9. What is the way to retrieve u from f?

Then it can be shown using the Bayes rule that the "best" way to retrieve u from f is to solve the following problem :inf u∈X,|u−f |p≤α (J(u)) (2.14)with p = 1 for impulse noise [2, 35, 11, 15], p = 2 for Gaussian noise [41], p = ∞ for uniform noise [45], and α a parameter depending on the variance of the noise. 

Q10. How many iterations of the problem are there?

Convergence rate (3.24) and bound (3.30) tell us that after k iterations of a Nesterovscheme, the worst case precision is 2||div||22d(ū)µk2 + nµ 2 . 

Q11. What is the way to solve the BV l1 problem?

Having the previous section in mind, a straightforward approach to solve (1.2) is to smooth the total variation, if F is non-smooth, it can be smoothed too, and then one just needs to use a fast scheme like (3.23) adapted to the unconstrained minimization of Lipschitz differentiable functions [33]. 

Q12. What is the amplitude of the rr?

It can be shown that the amplitude of theRR n° 6260Figure 2: Image to be decomposedtexture (v component) is bounded by a parameter depending linearly on α in (3.45). 

Q13. What is the method for a subgradient descent?

The second is a projected subgradient descent with optimal step :{u0 = f uk+1 = ΠK(u k − tk η k ||ηk||2 ) (4.82)ηk must belong to ∂J(uk) for convergence. 

Q14. How much computational effort is the Nesterov technique taking?

Let us precise that the computational effort per iteration of the Nesterov technique is between one and twice that of the projected gradient descent. 

Q15. What is the algorithm for this problem?

In [34], Y. Nesterov presents an O( 1√ ǫ ) algorithm adapted to the problem:inf u∈Q E(u) (3.22)where E is any convex, L − Lipschitz − differentiable function, and Q is any convex, closed set. 

Q16. How many iterations can be done to solve a dual problem?

the authors show that when dealing with a strongly convex function F , the resolution of a dual problem can be done with an O( 1√ǫ ) algorithm.