scispace - formally typeset
Search or ask a question
Journal ArticleDOI

Curve evolution implementation of the Mumford-Shah functional for image segmentation, denoising, interpolation, and magnification

01 Aug 2001-IEEE Transactions on Image Processing (IEEE TRANSACTION ON IMAGE PROCESSING)-Vol. 10, Iss: 8, pp 1169-1186
TL;DR: The resulting active contour model offers a tractable implementation of the original Mumford-Shah model to simultaneously segment and smoothly reconstruct the data within a given image in a coupled manner and leads to a novel PDE-based approach for simultaneous image magnification, segmentation, and smoothing.
Abstract: We first address the problem of simultaneous image segmentation and smoothing by approaching the Mumford-Shah (1989) paradigm from a curve evolution perspective. In particular, we let a set of deformable contours define the boundaries between regions in an image where we model the data via piecewise smooth functions and employ a gradient flow to evolve these contours. Each gradient step involves solving an optimal estimation problem for the data within each region, connecting curve evolution and the Mumford-Shah functional with the theory of boundary-value stochastic processes. The resulting active contour model offers a tractable implementation of the original Mumford-Shah model (i.e., without resorting to elliptic approximations which have traditionally been favored for greater ease in implementation) to simultaneously segment and smoothly reconstruct the data within a given image in a coupled manner. Various implementations of this algorithm are introduced to increase its speed of convergence. We also outline a hierarchical implementation of this algorithm to handle important image features such as triple points and other multiple junctions. Next, by generalizing the data fidelity term of the original Mumford-Shah functional to incorporate a spatially varying penalty, we extend our method to problems in which data quality varies across the image and to images in which sets of pixel measurements are missing. This more general model leads us to a novel PDE-based approach for simultaneous image magnification, segmentation, and smoothing, thereby extending the traditional applications of the Mumford-Shah functional which only considers simultaneous segmentation and smoothing.

Content maybe subject to copyright    Report

Citations
More filters
Proceedings ArticleDOI
01 Jul 2000
TL;DR: A novel algorithm for digital inpainting of still images that attempts to replicate the basic techniques used by professional restorators, and does not require the user to specify where the novel information comes from.
Abstract: Inpainting, the technique of modifying an image in an undetectable form, is as ancient as art itself. The goals and applications of inpainting are numerous, from the restoration of damaged paintings and photographs to the removal/replacement of selected objects. In this paper, we introduce a novel algorithm for digital inpainting of still images that attempts to replicate the basic techniques used by professional restorators. After the user selects the regions to be restored, the algorithm automatically fills-in these regions with information surrounding them. The fill-in is done in such a way that isophote lines arriving at the regions' boundaries are completed inside. In contrast with previous approaches, the technique here introduced does not require the user to specify where the novel information comes from. This is automatically done (and in a fast way), thereby allowing to simultaneously fill-in numerous regions containing completely different structures and surrounding backgrounds. In addition, no limitations are imposed on the topology of the region to be inpainted. Applications of this technique include the restoration of old photographs and damaged film; removal of superimposed text like dates, subtitles, or publicity; and the removal of entire objects from the image like microphones or wires in special effects.

3,830 citations

Journal ArticleDOI
TL;DR: A new multiphase level set framework for image segmentation using the Mumford and Shah model, for piecewise constant and piecewise smooth optimal approximations, and validated by numerical results for signal and image denoising and segmentation.
Abstract: We propose a new multiphase level set framework for image segmentation using the Mumford and Shah model, for piecewise constant and piecewise smooth optimal approximations. The proposed method is also a generalization of an active contour model without edges based 2-phase segmentation, developed by the authors earlier in T. Chan and L. Vese (1999. In Scale-Space'99, M. Nilsen et al. (Eds.), LNCS, vol. 1682, pp. 141–151) and T. Chan and L. Vese (2001. IEEE-IP, 10(2):266–277). The multiphase level set formulation is new and of interest on its own: by construction, it automatically avoids the problems of vacuum and overlaps it needs only log n level set functions for n phases in the piecewise constant cases it can represent boundaries with complex topologies, including triple junctionss in the piecewise smooth case, only two level set functions formally suffice to represent any partition, based on The Four-Color Theorem. Finally, we validate the proposed models by numerical results for signal and image denoising and segmentation, implemented using the Osher and Sethian level set method.

2,649 citations


Cites methods or result from "Curve evolution implementation of t..."

  • ...Most of the models need three level set functions, as in Zhao et al. (1996) and Samson et al....

    [...]

  • ...A standard rescaling can be made, as in Zhao et al. (1996), by replacing δε(φ) by |∇φ|, giving the following equations, already introduced in Osher and Sethian (1988) in the context of the level set theory:...

    [...]

  • ...We would like to mention that ideas very similar with those from the above case, have been also developed by Tsai et al. (2001), independently and contemporaneously....

    [...]

  • ...14 and 15 have been obtained independently and contemporaneously by Tsai et al. (2001). Finally, we show in Fig....

    [...]

  • ...14 and 15 have been obtained independently and contemporaneously by Tsai et al. (2001)....

    [...]

Journal ArticleDOI
TL;DR: This work proposes a region-based active contour model that draws upon intensity information in local regions at a controllable scale to cope with intensity inhomogeneity and shows desirable performances of this model.
Abstract: Intensity inhomogeneities often occur in real-world images and may cause considerable difficulties in image segmentation. In order to overcome the difficulties caused by intensity inhomogeneities, we propose a region-based active contour model that draws upon intensity information in local regions at a controllable scale. A data fitting energy is defined in terms of a contour and two fitting functions that locally approximate the image intensities on the two sides of the contour. This energy is then incorporated into a variational level set formulation with a level set regularization term, from which a curve evolution equation is derived for energy minimization. Due to a kernel function in the data fitting term, intensity information in local regions is extracted to guide the motion of the contour, which thereby enables our model to cope with intensity inhomogeneity. In addition, the regularity of the level set function is intrinsically preserved by the level set regularization term to ensure accurate computation and avoids expensive reinitialization of the evolving level set function. Experimental results for synthetic and real images show desirable performances of our method.

1,630 citations


Cites background from "Curve evolution implementation of t..."

  • ...For an image on the image domain , they propose to minimize the following energy: (2) where and represent the regions outside and inside the contour , respectively, and and are two constants that approximate the image intensity in and ....

    [...]

  • ...In [29], Vese and Chan introduced an energy functional on a level set function and two smooth functions and that are defined on the regions outside and inside the zero level contour of a level set function , respectively....

    [...]

  • ...Obviously, the involved computation in PS model is expensive, which limits its applications in practice....

    [...]

Journal ArticleDOI
TL;DR: A novel region-based method for image segmentation, which is able to simultaneously segment the image and estimate the bias field, and the estimated bias field can be used for intensity inhomogeneity correction (or bias correction).
Abstract: Intensity inhomogeneity often occurs in real-world images, which presents a considerable challenge in image segmentation. The most widely used image segmentation algorithms are region-based and typically rely on the homogeneity of the image intensities in the regions of interest, which often fail to provide accurate segmentation results due to the intensity inhomogeneity. This paper proposes a novel region-based method for image segmentation, which is able to deal with intensity inhomogeneities in the segmentation. First, based on the model of images with intensity inhomogeneities, we derive a local intensity clustering property of the image intensities, and define a local clustering criterion function for the image intensities in a neighborhood of each point. This local clustering criterion function is then integrated with respect to the neighborhood center to give a global criterion of image segmentation. In a level set formulation, this criterion defines an energy in terms of the level set functions that represent a partition of the image domain and a bias field that accounts for the intensity inhomogeneity of the image. Therefore, by minimizing this energy, our method is able to simultaneously segment the image and estimate the bias field, and the estimated bias field can be used for intensity inhomogeneity correction (or bias correction). Our method has been validated on synthetic images and real images of various modalities, with desirable performance in the presence of intensity inhomogeneities. Experiments show that our method is more robust to initialization, faster and more accurate than the well-known piecewise smooth model. As an application, our method has been used for segmentation and bias correction of magnetic resonance (MR) images with promising results.

1,201 citations


Cites background from "Curve evolution implementation of t..."

  • ...Z. Ding and J. C. Gore are with the Institute of Imaging Science, Vanderbilt University, Nashville, TN 37232 USA (e-mail: zhaohua.ding@vanderbilt.edu; john.gore@vanderbilt.edu)....

    [...]

  • ...J. C. Gatenby was with the Institute of Imaging Science, Vanderbilt University, Nashville, TN 37232 USA....

    [...]

  • ...C. Li was with the Institute of Imaging Science, Vanderbilt University, Nashville, TN 37232 USA....

    [...]

Journal ArticleDOI
TL;DR: A survey of a specific class of region-based level set segmentation methods and how they can all be derived from a common statistical framework is presented.
Abstract: Since their introduction as a means of front propagation and their first application to edge-based segmentation in the early 90's, level set methods have become increasingly popular as a general framework for image segmentation. In this paper, we present a survey of a specific class of region-based level set segmentation methods and clarify how they can all be derived from a common statistical framework. Region-based segmentation schemes aim at partitioning the image domain by progressively fitting statistical models to the intensity, color, texture or motion in each of a set of regions. In contrast to edge-based schemes such as the classical Snakes, region-based methods tend to be less sensitive to noise. For typical images, the respective cost functionals tend to have less local minima which makes them particularly well-suited for local optimization methods such as the level set method. We detail a general statistical formulation for level set segmentation. Subsequently, we clarify how the integration of various low level criteria leads to a set of cost functionals. We point out relations between the different segmentation schemes. In experimental results, we demonstrate how the level set function is driven to partition the image plane into domains of coherent color, texture, dynamic texture or motion. Moreover, the Bayesian formulation allows to introduce prior shape knowledge into the level set method. We briefly review a number of advances in this domain.

1,117 citations


Cites background or methods from "Curve evolution implementation of t..."

  • ...The probabilistic formulation of the segmentation problem presented in the following extends the statistical approaches pioneered in Leclerc (1989), Zhu and Yuille (1996), Paragios and Deriche (2002) and Tsai et al. (2001)....

    [...]

  • ...Tsai et al. (2001, 2003) proposed a very efficient implementation of shape-driven level set segmentation by directly optimizing in the linear subspace spanned by the principal components....

    [...]

References
More filters
Journal ArticleDOI
TL;DR: This work uses snakes for interactive interpretation, in which user-imposed constraint forces guide the snake near features of interest, and uses scale-space continuation to enlarge the capture region surrounding a feature.
Abstract: A snake is an energy-minimizing spline guided by external constraint forces and influenced by image forces that pull it toward features such as lines and edges. Snakes are active contour models: they lock onto nearby edges, localizing them accurately. Scale-space continuation can be used to enlarge the capture region surrounding a feature. Snakes provide a unified account of a number of visual problems, including detection of edges, lines, and subjective contours; motion tracking; and stereo matching. We have used snakes successfully for interactive interpretation, in which user-imposed constraint forces guide the snake near features of interest.

18,095 citations

Journal ArticleDOI
TL;DR: The PSC algorithm as mentioned in this paper approximates the Hamilton-Jacobi equations with parabolic right-hand-sides by using techniques from the hyperbolic conservation laws, which can be used also for more general surface motion problems.

13,020 citations

Journal ArticleDOI
TL;DR: A new definition of scale-space is suggested, and a class of algorithms used to realize a diffusion process is introduced, chosen to vary spatially in such a way as to encourage intra Region smoothing rather than interregion smoothing.
Abstract: A new definition of scale-space is suggested, and a class of algorithms used to realize a diffusion process is introduced. The diffusion coefficient is chosen to vary spatially in such a way as to encourage intraregion smoothing rather than interregion smoothing. It is shown that the 'no new maxima should be generated at coarse scales' property of conventional scale space is preserved. As the region boundaries in the approach remain sharp, a high-quality edge detector which successfully exploits global information is obtained. Experimental results are shown on a number of images. Parallel hardware implementations are made feasible because the algorithm involves elementary, local operations replicated over the image. >

12,560 citations


"Curve evolution implementation of t..." refers methods in this paper

  • ...For image smoothing, the technique of anisotropic diffusion has become a widespread field of research ranging from techniques based upon the original formulation of Perona and Malik [29], [30] to curve and surface evolution methods based upon geometric scale spaces [13], [16], [17], [35] and to a number of recent techniques for color imagery and other forms of vector-valued data [39], [40], [44]–[46], [50]....

    [...]

Journal ArticleDOI
TL;DR: A new model for active contours to detect objects in a given image, based on techniques of curve evolution, Mumford-Shah (1989) functional for segmentation and level sets is proposed, which can detect objects whose boundaries are not necessarily defined by the gradient.
Abstract: We propose a new model for active contours to detect objects in a given image, based on techniques of curve evolution, Mumford-Shah (1989) functional for segmentation and level sets. Our model can detect objects whose boundaries are not necessarily defined by the gradient. We minimize an energy which can be seen as a particular case of the minimal partition problem. In the level set formulation, the problem becomes a "mean-curvature flow"-like evolving the active contour, which will stop on the desired boundary. However, the stopping term does not depend on the gradient of the image, as in the classical active contour models, but is instead related to a particular segmentation of the image. We give a numerical algorithm using finite differences. Finally, we present various experimental results and in particular some examples for which the classical snakes methods based on the gradient are not applicable. Also, the initial curve can be anywhere in the image, and interior contours are automatically detected.

10,404 citations


"Curve evolution implementation of t..." refers background or methods or result in this paper

  • ...Obviously, this is not possible ith lower dimensional models (such as ones based on mean ntensities [9])....

    [...]

  • ...In particular, it naturally generalizes the recent work of Chan and Vese in [9] who consider piecewise constant generalization of the Mumford–Shah functional within a level set framework.2 We note that region-based approaches in general, enjoy a number of attractive properties including greater robustness to noise (by avoiding derivatives of the image intensity) and initial contour placement (by being less local than most edge-based approaches)....

    [...]

  • ...Consider the following vector version of the Mumford–Shah functional: 4Chan and Vese, who have considered the piecewise constant version of the Mumford–Shah functional [9], have also extended their framework to vectorvalued data in “Active Contours Without Edges for Vector-Valued Images” (see http://www.math.ucla.edu/applied/cam). where and denote theth component of the-dimensional vector-valued observed data and its smooth estimate, respectively....

    [...]

  • ...In contrast to most other region based techniques however (including our own previous work [48], [49] and that of Chan-Vese [9] and Paragios-Deriche [27]), which assume highly constrained parametric models for pixel intensities within each region, our approach employs the statistical model directly implied by the Mumford–Shah functional....

    [...]

  • ...This development may be regarded as an extension of several recent region-based approaches to curve evolution [9], [27], [48]....

    [...]

Journal ArticleDOI
TL;DR: In this article, the authors introduce and study the most basic properties of three new variational problems which are suggested by applications to computer vision, and study their application in computer vision.
Abstract: : This reprint will introduce and study the most basic properties of three new variational problems which are suggested by applications to computer vision. In computer vision, a fundamental problem is to appropriately decompose the domain R of a function g (x,y) of two variables. This problem starts by describing the physical situation which produces images: assume that a three-dimensional world is observed by an eye or camera from some point P and that g1(rho) represents the intensity of the light in this world approaching the point sub 1 from a direction rho. If one has a lens at P focusing this light on a retina or a film-in both cases a plane domain R in which we may introduce coordinates x, y then let g(x,y) be the strength of the light signal striking R at a point with coordinates (x,y); g(x,y) is essentially the same as sub 1 (rho) -possibly after a simple transformation given by the geometry of the imaging syste. The function g(x,y) defined on the plane domain R will be called an image. What sort of function is g? The light reflected off the surfaces Si of various solid objects O sub i visible from P will strike the domain R in various open subsets R sub i. When one object O1 is partially in front of another object O2 as seen from P, but some of object O2 appears as the background to the sides of O1, then the open sets R1 and R2 will have a common boundary (the 'edge' of object O1 in the image defined on R) and one usually expects the image g(x,y) to be discontinuous along this boundary. (JHD)

5,516 citations


"Curve evolution implementation of t..." refers background or methods in this paper

  • ...We note that one of the most widely studied mathematical models in image processing and computer vision addresses both goals simultaneously, namely that of Mumford and Shah [22], [23] who presented the variational problem of minimizing a functional involving a piecewise smooth representation of an image....

    [...]

  • ...This follows from the theory of junctions as presented in [22]....

    [...]

  • ...The estimates and that minimize (2) satisfy (decoupled) PDEs which can be obtained using standard variational methods [22]....

    [...]

  • ...in which denotes the smooth, closed segmenting curve, denotes the observed data, denotes the piecewise smooth approximation to with discontinuities only along , and denotes the image domain [22], [23]....

    [...]

  • ...For the rest of the paper, we will refer to this gradient flow, which is also derived in [22], as the Mumford–Shah flow....

    [...]