Important new developments have appeared since the most recent direct survey on shape correspondence published almost a decade ago. Our survey covers the period from 2011, their stopping point, to 2019, inclusive. The goal is to present the recent updates on correspondence computation between surfaces or point clouds embedded in 3D. Two tables summarizing and classifying the prominent, to our knowledge, papers of this period, and a large section devoted to their discussion lay down the foundation of our survey. The discussion is carried out in chronological order to reveal the distribution of various types of correspondence methods per year. We also explain our classification criteria along with the most basic solution examples. We finish with conclusions and future research directions.

Recent advances in shape correspondence

Electron microscopy (EM) image segmentation plays an important role in computer-aided diagnosis of specific pathogens or disease. However, EM image segmentation is a laborious task and needs to impose experts knowledge, which can take up valuable time from research. Convolutional neural network (CNN)-based methods have been proposed for EM image segmentation and achieved considerable progress. Among those CNN-based methods, UNet is regarded as the state-of-the-art method. However, the UNet usually has millions of parameters to increase training difficulty and is limited by the issue of vanishing gradients. To address those problems, the authors present a novel highly parameter efficient method called DenseUNet, which is inspired by the approach that takes particular advantage of recent advances in both UNet and DenseNet. In addition, they successfully apply the weighted loss, which enables us to boost the performance of segmentation. They conduct several comparative experiments on the ISBI 2012 EM dataset. The experimental results show that their method can achieve state-of-the-art results on EM image segmentation without any further post-processing module or pre-training. Moreover, due to smart design of the model, their approach has much less parameters than currently published encoder-decoder architecture variants for this dataset.

https://ietresearch.onlinelibrary.wiley.com/doi/epdf/10.1049/iet-ipr.2019.1527

DenseUNet: densely connected UNet for electron microscopy image segmentation

We present NeuroMorph, a new neural network architecture that takes as input two 3D shapes and produces in one go, i.e. in a single feed forward pass, a smooth interpolation and point-to-point correspondences between them. The interpolation, expressed as a deformation field, changes the pose of the source shape to resemble the target, but leaves the object identity unchanged. NeuroMorph uses an elegant architecture combining graph convolutions with global feature pooling to extract local features. During training, the model is incentivized to create realistic deformations by approximating geodesics on the underlying shape space manifold. This strong geometric prior allows to train our model end-to-end and in a fully unsupervised manner without requiring any manual correspondence annotations. NeuroMorph works well for a large variety of input shapes, including non-isometric pairs from different object categories. It obtains state-of-the-art results for both shape correspondence and interpolation tasks, matching or surpassing the performance of recent unsupervised and supervised methods on multiple benchmarks.

/pdf/neuromorph-unsupervised-shape-interpolation-and-1fsqnan49y.pdf

NeuroMorph: Unsupervised Shape Interpolation and Correspondence in One Go

The traditional visual SLAM systems take the monocular or stereo camera as input sensor, with complex map initialization and map point triangulation steps needed for 3D map reconstruction, which are easy to fail, computationally complex and can cause noisy measurements. The emergence of RGB-D camera which provides RGB image together with depth information breaks this situation. While a number of RGB-D SLAM systems have been proposed in recent years, the current classification research on RGB-D SLAM is very lacking, and their advantages and shortcomings remain unclear regarding different applications and perturbations, such as illumination transformation, noise and rolling shutter effect of sensors. In this paper, we mainly introduced the basic concept and structure of the RGB-D SLAM system, and then introduced the differences between the various RGB-D SLAM systems in the three aspects of tracking, mapping, and loop detection, and we make a classification study on different RGB-D SLAM algorithms according to the three aspect. Furthermore, we discuss some advanced topics and open problems of RGB-D SLAM, hoping that it will help for future exploring. In the end, we conducted a large number of evaluation experiments on multiple RGB-D SLAM systems, and analyzed their advantages and disadvantages, as well as performance differences in different application scenarios, and provided references for researchers and developers.

/pdf/survey-and-evaluation-of-rgb-d-slam-5xqjewgpzw.pdf

Survey and Evaluation of RGB-D SLAM

We propose a novel 3D shape correspondence method based on the iterative alignment of so-called smooth shells. Smooth shells define a series of coarse-to-fine shape approximations designed to work well with multiscale algorithms. The main idea is to first align rough approximations of the geometry and then add more and more details to refine the correspondence. We fuse classical shape registration with Functional Maps by embedding the input shapes into an intrinsic-extrinsic product space. Moreover, we disambiguate intrinsic symmetries by applying a surrogate based Markov chain Monte Carlo initialization. Our method naturally handles various types of noise that commonly occur in real scans, like non-isometry or incompatible meshing. Finally, we demonstrate state-of-the-art quantitative results on several datasets and show that our pipeline produces smoother, more realistic results than other automatic matching methods in real world applications.

Smooth Shells: Multi-Scale Shape Registration with Functional Maps

Non-rigid registration is challenging because it is ill-posed with high degrees of freedom and is thus sensitive to noise and outliers. We propose a robust non-rigid registration method using reweighted sparsities on position and transformation to estimate the deformations between 3-D shapes. We formulate the energy function with position and transformation sparsity on both the data term and the smoothness term, and define the smoothness constraint using local rigidity. The double sparsity based non-rigid registration model is enhanced with a reweighting scheme, and solved by transferring the model into four alternately-optimized subproblems which have exact solutions and guaranteed convergence. Experimental results on both public datasets and real scanned datasets show that our method outperforms the state-of-the-art methods and is more robust to noise and outliers than conventional non-rigid registration methods.

/pdf/robust-non-rigid-registration-with-reweighted-position-and-gje9sj63kj.pdf

Robust Non-Rigid Registration with Reweighted Position and Transformation Sparsity

SHREC’20: Shape correspondence with non-isometric deformations

We present a novel global non-rigid registration method for dynamic 3D objects. Our method allows objects to undergo large non-rigid deformations and achieves high-quality results even with substantial pose change or camera motion between views. In addition, our method does not require a template prior and uses less raw data than tracking-based methods since only a sparse set of scans is needed. We simultaneously compute the deformations of all the scans by optimizing a global alignment problem to avoid the well-known loop closure problem and use an as-rigid-as-possible constraint to eliminate the shrinkage problem of the deformed shapes, especially near open boundaries of scans. To cope with large-scale problems, we design a coarse-to-fine multi-resolution scheme, which also avoids the optimization being trapped into local minima. The proposed method is evaluated on public datasets and real datasets captured by an RGB-D sensor. The experimental results demonstrate that the proposed method obtains better results than several state-of-the-art methods.

/pdf/global-3d-non-rigid-registration-of-deformable-objects-using-3eu8di8wwo.pdf

Global 3D Non-Rigid Registration of Deformable Objects Using a Single RGB-D Camera

/pdf/shape-correspondence-with-isometric-and-non-isometric-qbgm5xkbqg.pdf

Shape Correspondence with Isometric and Non-Isometric Deformations

The invention belongs to the field of computer application, and aims at realizing simultaneous constraining of position and transformation and acquiring a batter registration result so that the adopted technical scheme is a non-rigid surface registration method based on weighted double sparse constraint. A corresponding point in a target point set of each point in a template point set is found, and a double sparse L1 norm is utilized to constrain a non-rigid registration energy equation so that non-rigid registration is performed on three-dimensional surface data with deformation. The non-rigid surface registration method based on the weighted double sparse constraint is mainly applied to acquire a complete model of a three-dimensional object.

Daoliang Guo

Papers

Robust Non-Rigid Registration with Reweighted Position and Transformation Sparsity

SHREC’20: Shape correspondence with non-isometric deformations

Global 3D Non-Rigid Registration of Deformable Objects Using a Single RGB-D Camera

Shape Correspondence with Isometric and Non-Isometric Deformations

Non-rigid surface registration method based on weighted double sparse constraint