Human pose estimation is a very active research field, stimulated by its important applications in robotics, entertainment or health and sports sciences, among others. Advances in convolutional networks triggered noticeable improvements in 2D pose estimation, leading modern 3D markerless motion capture techniques to an average error per joint of 20 mm. However, with the proliferation of methods, it is becoming increasingly difficult to make an informed choice. Here, we review the leading human pose estimation methods of the past five years, focusing on metrics, benchmarks and method structures. We propose a taxonomy based on accuracy, speed and robustness that we use to classify de methods and derive directions for future research.

A review of 3D human pose estimation algorithms for markerless motion capture

With the advent of convolutional neural networks, stereo matching algorithms have recently gained tremendous progress. However, it remains a great challenge to accurately extract disparities from real-world image pairs taken by consumer-level devices like smartphones, due to practical complicating factors such as thin structures, non-ideal rectification, camera module inconsistencies and various hard-case scenes. In this paper, we propose a set of innovative designs to tackle the problem of practical stereo matching: 1) to better recover fine depth details, we design a hierarchical network with recurrent refinement to update disparities in a coarse-to-fine manner, as well as a stacked cascaded architecture for inference; 2) we propose an adaptive group correlation layer to mitigate the impact of erroneous rectification; 3) we introduce a new synthetic dataset with special attention to difficult cases for better generalizing to real-world scenes. Our results not only rank 1st on both Middlebury and ETH3D benchmarks, outperforming existing state-of-the-art methods by a notable margin, but also exhibit high-quality details for real-life photos, which clearly demonstrates the efficacy of our contributions.

Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation

Real-time performance of stereo matching networks is important for many applications, such as automatic driving, robot navigation and augmented reality (AR). Although significant progress has been made in stereo matching networks in recent years, it is still challenging to balance real-time performance and accuracy. In this paper, we present a novel edge-preserving cost volume upsampling module based on the slicing operation in the learned bilateral grid. The slicing layer is parameter-free, which allows us to obtain a high quality cost volume of high resolution from a low-resolution cost volume under the guide of the learned guidance map efficiently. The proposed cost volume upsampling module can be seamlessly embedded into many existing stereo matching networks, such as GCNet, PSMNet, and GANet. The resulting networks are accelerated several times while maintaining comparable accuracy. Furthermore, we design a real-time network (named BGNet) based on this module, which outperforms existing published real-time deep stereo matching networks, as well as some complex networks on the KITTI stereo datasets. The code is available at https://github.com/YuhuaXu/BGNet.

/pdf/bilateral-grid-learning-for-stereo-matching-networks-3uotivcszm.pdf

Bilateral Grid Learning for Stereo Matching Networks

In this paper, we propose HESIC, an end-to-end trainable deep network for stereo image compression (SIC). To fully explore the mutual information across two stereo images, we use a deep regression model to estimate the homography matrix, i.e., H matrix. Then, the left image is spatially transformed by the H matrix, and only the residual information between the left and right images is encoded to save bitrates. A two-branch auto-encoder architecture is adopted in HESIC, corresponding to the left and right images, respectively. For entropy coding, we use two conditional stereo entropy models, i.e., Gaussian mixture model (GMM) based and context based entropy models, to fully explore the correlation between the two images to reduce the coding bit-rates. In decoding, a cross quality enhancement module is proposed to enhance the image quality based on inverse H matrix. Experimental results show that our HESIC outperforms state-of-the-art SIC methods on InStereo2K and KITTI datasets both quantitatively and qualitatively. Code is available at https://github.com/ywz978020607/HESIC.

/pdf/deep-homography-for-efficient-stereo-image-compression-4oeb6qqlcp.pdf

Deep Homography for Efficient Stereo Image Compression

U-Net model for multi-component digital rock modeling of shales based on CT and QEMSCAN images

Deep neural networks have shown great success in stereo matching in recent years. On the KITTI datasets, most top performing methods are based on neural networks. However, on the Middlebury datasets, these methods usually do not perform well. The KITTI datasets are collected in outdoor scenes while the Middlebury datasets are collected in indoor scenes. It is commonly believed that the community still lacks a large labelled dataset for stereo matching in indoor scenes. In this paper, we introduce a new stereo dataset called InStereo2K. It contains 2050 pairs of stereo images with highly accurate groundtruth disparity maps, including 2000 pairs for training and 50 pairs for test. Experimental results show that our dataset can significantly improve the performance of several latest networks (including StereoNet and PSMNet) on the Middlebury 2014 dataset. The large scale, high accuracy and rich diversity of the proposed InStereo2K dataset provide new opportunities to researchers in the area of stereo matching and beyond. It also takes end-to-end stereo matching methods a step towards practical applications.

InStereo2K: a large real dataset for stereo matching in indoor scenes

Noncontact deflection measurement for bridge through a multi-UAVs system

Compared with geometric stereo vision based on triangulation principle, photometric stereo method has advantages in recovering per-pixel surface details. In this paper, we present a practical 3D imaging system by combining the near-light photometric stereo and the speckle-based stereo matching method. The system is compact in structure and suitable for multi-albedo targets. The parameters (including position and intensity) of the light sources can be self-calibrated. To realize the auto-calibration, we first use the distant lighting model to estimate the initial surface albedo map, and then with the estimated albedo map and the normal vector field fixed, the parameters of the near lighting model are optimized. Next, with the optimized lighting model, we use the near-light photometric stereo method to re-compute the surface normal and fuse it with the coarse depth map from stereo vision to achieve high-quality depth map. Experimental results show that our system can realize high-quality reconstruction in general indoor environments.

A self-calibrated photo-geometric depth camera

For the problem of 3D line reconstruction in binocular or multiple view stereo vision, when there are no corresponding points on the line, the method called Direction-then-Point (DtP) can be used, and if there are two pairs of corresponding points on the line, the method called Two Points 3D coordinates (TPS) can be used. However, when there is only one pair of corresponding points on the line, can we get the better accuracy than DtP for 3D line reconstruction? In this paper, a linear and more accurate method called Point-then-Direction (PtD) is proposed. First, we used the intersection method to obtain the 3D point's coordinate from its corresponding image points. Then, we used this point as a position on the line to calculate the direction of the line by minimizing the image angle residual. PtD is also suitable for multiple camera systems. The simulation results demonstrate that PtD increases the accuracy of both the direction and the position of the 3D line compared to DtP. At the same time, PtD achieves a better result in direction of the 3D line than TPS, but has a lower accuracy in the position of 3D lines than TPS.

https://www.mdpi.com/1424-8220/21/2/658/pdf

Xiaohu Zhang

Papers

InStereo2K: a large real dataset for stereo matching in indoor scenes

Noncontact deflection measurement for bridge through a multi-UAVs system

A self-calibrated photo-geometric depth camera

An Accurate Linear Method for 3D Line Reconstruction for Binocular or Multiple View Stereo Vision.

SkeletonFusion: Reconstruction and tracking of human body in real-time