Real-time single image depth perception in the wild with handheld devices

TLDR: A thorough evaluation of real-time, depth-aware augmented reality networks highlights the ability of such fast networks to generalize well to new environments, a crucial feature required to tackle the extremely varied contexts faced in real applications.

Abstract: Depth perception is paramount to tackle real-world problems, ranging from autonomous driving to consumer applications For the latter, depth estimation from a single image represents the most versatile solution, since a standard camera is available on almost any handheld device Nonetheless, two main issues limit its practical deployment: i) the low reliability when deployed in-the-wild and ii) the demanding resource requirements to achieve real-time performance, often not compatible with such devices Therefore, in this paper, we deeply investigate these issues showing how they are both addressable adopting appropriate network design and training strategies -- also outlining how to map the resulting networks on handheld devices to achieve real-time performance Our thorough evaluation highlights the ability of such fast networks to generalize well to new environments, a crucial feature required to tackle the extremely varied contexts faced in real applications Indeed, to further support this evidence, we report experimental results concerning real-time depth-aware augmented reality and image blurring with smartphones in-the-wild