Joint Object Pose Estimation and Shape Reconstruction in Urban Street Scenes Using 3D Shape Priors

TLDR: This work proposes a novel approach for using compact shape manifolds of the shape within an object class for object segmentation, pose and shape estimation and demonstrates that the shape manifold alignment method yields improved results over the initial stereo reconstruction and object detection method in depth and pose accuracy.

Abstract: Estimating the pose and 3D shape of a large variety of instances within an object class from stereo images is a challenging problem, especially in realistic conditions such as urban street scenes. We propose a novel approach for using compact shape manifolds of the shape within an object class for object segmentation, pose and shape estimation. Our method first detects objects and estimates their pose coarsely in the stereo images using a state-of-the-art 3D object detection method. An energy minimization method then aligns shape and pose concurrently with the stereo reconstruction of the object. In experiments, we evaluate our approach for detection, pose and shape estimation of cars in real stereo images of urban street scenes. We demonstrate that our shape manifold alignment method yields improved results over the initial stereo reconstruction and object detection method in depth and pose accuracy.