In the last decades several cost aggregation methods aimed at improving the robustness of stereo correspondence within local and global algorithms have been proposed. Given the recent developments and the lack of an appropriate comparison, in this paper we survey, classify and compare experimentally on a standard data set the main cost aggregation approaches proposed in literature. The experimental evaluation addresses both accuracy and computational requirements, so as to outline the best performing methods under these two criteria.

/pdf/classification-and-evaluation-of-cost-aggregation-methods-28tg93l8s6.pdf

Classification and evaluation of cost aggregation methods for stereo correspondence

Recent work has shown that convolutional neural networks (CNNs) can be applied successfully in disparity estimation, but these methods still suffer from errors in regions of low texture, occlusions, and reflections. Concurrently, deep learning for semantic segmentation has shown great progress in recent years. In this letter, we design a CNN architecture that combines these two tasks to improve the quality and accuracy of disparity estimation with the help of semantic segmentation. Specifically, we propose a network structure in which these two tasks are highly coupled. One key novelty of this approach is the two-stage refinement process. Initial disparity estimates are refined with an embedding learned from the semantic segmentation branch of the network. The proposed model is trained using an unsupervised approach, in which images from one half of the stereo pair are warped and compared against images from the other camera. Another key advantage of the proposed approach is that a single network is capable of outputting disparity estimates and semantic labels. These outputs are of great use in autonomous vehicle operation; with real-time constraints being key, such performance improvements increase the viability of driving applications. Experiments on KITTI and Cityscapes datasets show that our model can achieve state-of-the-art results and that leveraging embedding learned from semantic segmentation improves the performance of disparity estimation.

DispSegNet: Leveraging Semantics for End-to-End Learning of Disparity Estimation From Stereo Imagery

Adaptive support weight (ASW) methods represent the state of the art in local stereo matching, while the bilateral filter-based ASW method achieves outstanding performance. However, this method fails to resolve the ambiguity induced by nearby pixels at different disparities but with similar colors. In this paper, we introduce a novel trilateral filter (TF)-based ASW method that remedies such ambiguities by considering the possible disparity discontinuities through color discontinuity boundaries, i.e., the boundary strength between two pixels, which is measured by a local energy model. We also present a recursive TF-based ASW method whose computational complexity is $O(N)$ for the cost aggregation step, and $O(N{\rm Log}_{2}(N))$ for boundary detection, where $N$ denotes the input image size. This complexity is thus independent of the support window size. The recursive TF-based method is a nonlocal cost aggregation strategy. The experimental evaluation on the Middlebury benchmark shows that the proposed method, whose average error rate is 4.95%, outperforms other local methods in terms of accuracy. Equally, the average runtime of the proposed TF-based cost aggregation is roughly 260 ms on a 3.4-GHz Inter Core i7 CPU, which is comparable with state-of-the-art efficiency.

/pdf/a-fast-trilateral-filter-based-adaptive-support-weight-4ouml8fu46.pdf

A Fast Trilateral Filter-Based Adaptive Support Weight Method for Stereo Matching

Stereo matching has been widely used in various computer vision applications and it is still a challenging problem. Adaptive support weights (ASW) methods represent the state of the art in stereo matching and have achieved outstanding performance. However, the local ASW methods fail to resolve the matching ambiguity in low texture areas because their cost aggregation is limited within local fixed or adaptive support windows. On the other hand, the non-local ASW methods perform cost aggregation along a special tree, so that these methods are often sensitive to high texture areas since some useful connectivity constrains between adjacent pixels are broken during constructing the special tree. To solve these problems, in this paper, a novel and generic fusing ASW framework are proposed for stereo matching. In this framework, we establish dual support windows for each pixel, i.e., a local window and the whole image window. As such, the primitive connectivity between each pixel and its neighboring pixels in the local window can be maintained, and then each pixel not only gets appropriate supports from neighboring pixels within its local support window but also receives more adaptive supports from the other pixels outside the local window. Furthermore, a local edge-aware filter and a non-local edge-aware filter, whose kernel windows correspond to the dual support windows, are merged in order to achieve collaborative filtering of the cost volume. The performance evaluation on the Middlebury and KITTI datasets shows that the proposed stereo matching method outperforms the current state-of-the-art methods.

/pdf/stereo-matching-with-fusing-adaptive-support-weights-1yg8blyw82.pdf

Stereo Matching With Fusing Adaptive Support Weights

In this study, the authors present a new area-based stereo matching algorithm that computes dense disparity maps for a real-time vision system. Although many stereo matching algorithms have been proposed in recent years, correlation-based algorithms still have an edge because of speed and less memory requirements. The selection of appropriate shape and size of the matching window is a difficult problem for correlation-based algorithms. In the proposed approach, two correlation windows are used to improve the performance of the algorithm while maintaining its real-time suitability. The CPU implementation of the proposed algorithm computes more than 10 frame/s. Unlike other area-based stereo matching algorithms, this method works very well at disparity boundaries as well as in low textured image areas and computes a dense and sharp disparity map. Evaluations on the benchmark Middlebury stereo datasets have been performed to demonstrate the qualitative and quantitative performance of the proposed algorithm.

https://ietresearch.onlinelibrary.wiley.com/doi/epdf/10.1049/iet-cvi.2011.0077

Window-based approach for fast stereo correspondence

Window-based correlation algorithms are widely used for stereo matching due to their computational efficiency as compared to global algorithms. In this paper, a multiple window correlation algorithm for stereo matching is presented which addresses the problems associated with a fixed window size. The developed algo- rithm differs from the previous multiple window algorithms by intro- ducing a reliability test to select the most reliable window among multiple windows of increasing sizes. This ensures that at least one window is large enough to cover a region of adequate intensity variations while at the same time small enough to cover a constant depth region. A recursive computation procedure is also used to allow a computationally efficient implementation of the algorithm. The outcome obtained from a standard set of images with known disparity maps shows that the generated disparity maps are more accurate as compared to two popular stereo matching local algorithms. © 2007 SPIE and IS&T. DOI: 10.1117/1.2711817

/pdf/stereo-matching-via-selective-multiple-windows-48ih182ax2.pdf

Satyajit Anil Adhyapak

Papers

Stereo matching via selective multiple windows