Constant Time Weighted Median Filtering for Stereo Matching and Beyond
01 Dec 2013-pp 49-56
TL;DR: It is discovered that with this refinement, even the simple box filter aggregation achieves comparable accuracy with various sophisticated aggregation methods (with the same refinement), revealing that the previously overlooked refinement can be at least as crucial as aggregation.
Abstract: Despite the continuous advances in local stereo matching for years, most efforts are on developing robust cost computation and aggregation methods. Little attention has been seriously paid to the disparity refinement. In this work, we study weighted median filtering for disparity refinement. We discover that with this refinement, even the simple box filter aggregation achieves comparable accuracy with various sophisticated aggregation methods (with the same refinement). This is due to the nice weighted median filtering properties of removing outlier error while respecting edges/structures. This reveals that the previously overlooked refinement can be at least as crucial as aggregation. We also develop the first constant time algorithm for the previously time-consuming weighted median filter. This makes the simple combination ``box aggregation + weighted median'' an attractive solution in practice for both speed and accuracy. As a byproduct, the fast weighted median filtering unleashes its potential in other applications that were hampered by high complexities. We show its superiority in various applications such as depth up sampling, clip-art JPEG artifact removal, and image stylization.
Citations
More filters
06 Sep 2014
TL;DR: A new framework to filter images with the complete control of detail smoothing under a scale measure is proposed, based on a rolling guidance implemented in an iterative manner that converges quickly and achieves realtime performance and produces artifact-free results.
Abstract: Images contain many levels of important structures and edges. Compared to masses of research to make filters edge preserving, finding scale-aware local operations was seldom addressed in a practical way, albeit similarly vital in image processing and computer vision. We propose a new framework to filter images with the complete control of detail smoothing under a scale measure. It is based on a rolling guidance implemented in an iterative manner that converges quickly. Our method is simple in implementation, easy to understand, fully extensible to accommodate various data operations, and fast to produce results. Our implementation achieves realtime performance and produces artifact-free results in separating different scale structures. This filter also introduces several inspiring properties different from previous edge-preserving ones.
532 citations
Cites background from "Constant Time Weighted Median Filte..."
...They include representatives of anisotropic diffusion [20], bilateral filter (BF) [26], guided filter (GF) [13], geodesic filters [7,11], weighted median filters [18,34], to name a few....
[...]
...This class mainly includes mode filter [27,15], median filter [28,15], and weighted median filter [18,34]....
[...]
Book•
30 May 2015TL;DR: This tutorial presents a hands-on view of the field of multi-view stereo with a focus on practical algorithms, describing in detail its main two ingredients: robust implementations of photometric consistency measures, and efficient optimization algorithms.
Abstract: This tutorial presents a hands-on view of the field of multi-view stereo with a focus on practical algorithms. Multi-view stereo algorithms are able to construct highly detailed 3D models from images alone. They take a possibly very large set of images and construct a 3D plausible geometry that explains the images under some reasonable assumptions, the most important being scene rigidity. The tutorial frames the multiview stereo problem as an image/geometry consistency optimization problem. It describes in detail its main two ingredients: robust implementations of photometric consistency measures, and efficient optimization algorithms. It then presents how these main ingredients are used by some of the most successful algorithms, applied into real applications, and deployed as products in the industry. Finally it describes more advanced approaches exploiting domain-specific knowledge such as structural priors, and gives an overview of the remaining challenges and future research directions.
459 citations
Cites background from "Constant Time Weighted Median Filte..."
...Although bilateral filters are the most common, other anisotropic filters can be used for stereo such as a weighted median filter [137]....
[...]
07 Jun 2015
TL;DR: This paper introduces an algorithm that accurately estimates depth maps using a lenslet light field camera and estimates the multi-view stereo correspondences with sub-pixel accuracy using the cost volume using the phase shift theorem.
Abstract: This paper introduces an algorithm that accurately estimates depth maps using a lenslet light field camera. The proposed algorithm estimates the multi-view stereo correspondences with sub-pixel accuracy using the cost volume. The foundation for constructing accurate costs is threefold. First, the sub-aperture images are displaced using the phase shift theorem. Second, the gradient costs are adaptively aggregated using the angular coordinates of the light field. Third, the feature correspondences between the sub-aperture images are used as additional constraints. With the cost volume, the multi-label optimization propagates and corrects the depth map in the weak texture regions. Finally, the local depth map is iteratively refined through fitting the local quadratic function to estimate a non-discrete depth map. Because micro-lens images contain unexpected distortions, a method is also proposed that corrects this error. The effectiveness of the proposed algorithm is demonstrated through challenging real world examples and including comparisons with the performance of advanced depth estimation algorithms.
436 citations
TL;DR: A survey of optical flow estimation classifying the main principles elaborated during this evolution, with a particular concern given to recent developments is proposed.
368 citations
Cites methods from "Constant Time Weighted Median Filte..."
...The combination of the three ingredients has led to the development of competitive optical flow estimation methods based on pure feature matching locally filtered [122,172,238]....
[...]
...The filtering is achieved in [122] by guided filtering [111] and in [172] by weighted median....
[...]
08 Oct 2016
TL;DR: A novel algorithm for edge-aware smoothing that combines the flexibility and speed of simple filtering approaches with the accuracy of domain-specific optimization algorithms, fast, robust, straightforward to generalize to new domains, and simple to integrate into deep learning pipelines.
Abstract: We present the bilateral solver, a novel algorithm for edge-aware smoothing that combines the flexibility and speed of simple filtering approaches with the accuracy of domain-specific optimization algorithms. Our technique is capable of matching or improving upon state-of-the-art results on several different computer vision tasks (stereo, depth superresolution, colorization, and semantic segmentation) while being 10–1000\(\times \) faster than baseline techniques with comparable accuracy, and producing lower-error output than techniques with comparable runtimes. The bilateral solver is fast, robust, straightforward to generalize to new domains, and simple to integrate into deep learning pipelines.
336 citations
References
More filters
01 Dec 2001
TL;DR: A machine learning approach for visual object detection which is capable of processing images extremely rapidly and achieving high detection rates and the introduction of a new image representation called the "integral image" which allows the features used by the detector to be computed very quickly.
Abstract: This paper describes a machine learning approach for visual object detection which is capable of processing images extremely rapidly and achieving high detection rates. This work is distinguished by three key contributions. The first is the introduction of a new image representation called the "integral image" which allows the features used by our detector to be computed very quickly. The second is a learning algorithm, based on AdaBoost, which selects a small number of critical visual features from a larger set and yields extremely efficient classifiers. The third contribution is a method for combining increasingly more complex classifiers in a "cascade" which allows background regions of the image to be quickly discarded while spending more computation on promising object-like regions. The cascade can be viewed as an object specific focus-of-attention mechanism which unlike previous approaches provides statistical guarantees that discarded regions are unlikely to contain the object of interest. In the domain of face detection the system yields detection rates comparable to the best previous systems. Used in real-time applications, the detector runs at 15 frames per second without resorting to image differencing or skin color detection.
18,620 citations
04 Jan 1998
TL;DR: In contrast with filters that operate on the three bands of a color image separately, a bilateral filter can enforce the perceptual metric underlying the CIE-Lab color space, and smooth colors and preserve edges in a way that is tuned to human perception.
Abstract: Bilateral filtering smooths images while preserving edges, by means of a nonlinear combination of nearby image values. The method is noniterative, local, and simple. It combines gray levels or colors based on both their geometric closeness and their photometric similarity, and prefers near values to distant values in both domain and range. In contrast with filters that operate on the three bands of a color image separately, a bilateral filter can enforce the perceptual metric underlying the CIE-Lab color space, and smooth colors and preserve edges in a way that is tuned to human perception. Also, in contrast with standard filtering, bilateral filtering produces no phantom colors along edges in color images, and reduces phantom colors where they appear in the original image.
8,738 citations
"Constant Time Weighted Median Filte..." refers background in this paper
...Thanks to the constant time algorithm for weighted median filtering, it is feasible for us to study its performance for refining local stereo results....
[...]
...We show its superiority in various applications such as depth upsampling, clip-art JPEG artifact removal, and image stylization....
[...]
TL;DR: This paper has designed a stand-alone, flexible C++ implementation that enables the evaluation of individual components and that can easily be extended to include new algorithms.
Abstract: Stereo matching is one of the most active research areas in computer vision. While a large number of algorithms for stereo correspondence have been developed, relatively little work has been done on characterizing their performance. In this paper, we present a taxonomy of dense, two-frame stereo methods designed to assess the different components and design decisions made in individual stereo algorithms. Using this taxonomy, we compare existing stereo methods and present experiments evaluating the performance of many different variants. In order to establish a common software platform and a collection of data sets for easy evaluation, we have designed a stand-alone, flexible C++ implementation that enables the evaluation of individual components and that can be easily extended to include new algorithms. We have also produced several new multiframe stereo data sets with ground truth, and are making both the code and data sets available on the Web.
7,458 citations
TL;DR: The guided filter is a novel explicit image filter derived from a local linear model that can be used as an edge-preserving smoothing operator like the popular bilateral filter, but it has better behaviors near edges.
Abstract: In this paper, we propose a novel explicit image filter called guided filter. Derived from a local linear model, the guided filter computes the filtering output by considering the content of a guidance image, which can be the input image itself or another different image. The guided filter can be used as an edge-preserving smoothing operator like the popular bilateral filter [1], but it has better behaviors near edges. The guided filter is also a more generic concept beyond smoothing: It can transfer the structures of the guidance image to the filtering output, enabling new filtering applications like dehazing and guided feathering. Moreover, the guided filter naturally has a fast and nonapproximate linear time algorithm, regardless of the kernel size and the intensity range. Currently, it is one of the fastest edge-preserving filters. Experiments show that the guided filter is both effective and efficient in a great variety of computer vision and computer graphics applications, including edge-aware smoothing, detail enhancement, HDR compression, image matting/feathering, dehazing, joint upsampling, etc.
4,730 citations
"Constant Time Weighted Median Filte..." refers background in this paper
...Thanks to the constant time algorithm for weighted median filtering, it is feasible for us to study its performance for refining local stereo results....
[...]
...We show its superiority in various applications such as depth upsampling, clip-art JPEG artifact removal, and image stylization....
[...]
TL;DR: A new technique for the display of high-dynamic-range images, which reduces the contrast while preserving detail, is presented, based on a two-scale decomposition of the image into a base layer.
Abstract: We present a new technique for the display of high-dynamic-range images, which reduces the contrast while preserving detail. It is based on a two-scale decomposition of the image into a base layer,...
1,715 citations
"Constant Time Weighted Median Filte..." refers background in this paper
...The weighted median filter inherits some desired properties from both median filtering and edge-aware averaging filtering (the guided filter)....
[...]
...…desired for refining local stereo aggregation results, which may generate erroneous disparity values in arbitrary ranges (see 3For a clearer summarization, in Table 1 we have ignored some operations applied for individual methods, including division, slicing, downsampling/quantizing/coarsening,…...
[...]