Proceedings ArticleDOI
KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera
Shahram Izadi,David Kim,Otmar Hilliges,David Molyneaux,Richard Newcombe,Pushmeet Kohli,Jamie Shotton,Steve Hodges,Dustin Freeman,Andrew J. Davison,Andrew Fitzgibbon +10 more
- pp 559-568
Reads0
Chats0
TLDR
Novel extensions to the core GPU pipeline demonstrate object segmentation and user interaction directly in front of the sensor, without degrading camera tracking or reconstruction, to enable real-time multi-touch interactions anywhere.Abstract:Â
KinectFusion enables a user holding and moving a standard Kinect camera to rapidly create detailed 3D reconstructions of an indoor scene. Only the depth data from Kinect is used to track the 3D pose of the sensor and reconstruct, geometrically precise, 3D models of the physical scene in real-time. The capabilities of KinectFusion, as well as the novel GPU-based pipeline are described in full. Uses of the core system for low-cost handheld scanning, and geometry-aware augmented reality and physics-based interactions are shown. Novel extensions to the core GPU pipeline demonstrate object segmentation and user interaction directly in front of the sensor, without degrading camera tracking or reconstruction. These extensions are used to enable real-time multi-touch interactions anywhere, allowing any planar or non-planar reconstructed physical surface to be appropriated for touch.read more
Citations
More filters
Patent
System and methods for depth regularization and semiautomatic interactive matting using rgb-d images
TL;DR: In this paper, an image processing pipeline application directs a processor to receive (i) an image (ii) an initial depth map corresponding to the depths of pixels within the image, regularize the original depth map into a dense depth map using depth values of known pixels to compute depth value of unknown pixels, determine an object of interest to be extracted from the image and generate an initial trimap using the sparse depth map and the object of the interest to extract from the original image, and apply color image matting to unknown regions of the initial trimmedap to generate a matte for
Proceedings ArticleDOI
Filter Forests for Learning Data-Dependent Convolutional Kernels
Sean Fanello,Cem Keskin,Pushmeet Kohli,Shahram Izadi,Jamie Shotton,Antonio Criminisi,Ugo Pattacini,Tim Paek +7 more
TL;DR: It is demonstrated how filter forests can be used to learn optimal denoising filters for natural images as well as for other tasks such as depth image refinement, and 1D signal magnitude estimation.
Proceedings ArticleDOI
Super Generalized 4PCS for 3D Registration
TL;DR: This work shows through experimentation that the number of 4-point bases decreases, sometimes exponentially, with a non-coplanar base, and proposes the Super Generalized 4PCS algorithm which can exhibit a significant speed-up of up to 6.5x over the Super 4 PCS algorithm as demonstrated experimentally.
Proceedings ArticleDOI
MUWA: Multi-field universal wheel for air-land vehicle with quad variable-pitch propellers
TL;DR: This article clarifies the behavioral objectives, structural design, basic control mechanism of the ring-shaped robot, and examples of 3D measurements of the vehicle.
Patent
Depth from time-of-flight using machine learning
TL;DR: In this article, a depth detection system with a trained machine learning component has been described, which is configured to compute in a single stage, for an item of the stored raw time-of-flight sensor data, a depth value of a surface depicted by the item, by pushing the item through the trained ML component.
References
More filters
Journal ArticleDOI
A method for registration of 3-D shapes
Paul J. Besl,H.D. McKay +1 more
TL;DR: In this paper, the authors describe a general-purpose representation-independent method for the accurate and computationally efficient registration of 3D shapes including free-form curves and surfaces, based on the iterative closest point (ICP) algorithm, which requires only a procedure to find the closest point on a geometric entity to a given point.
Book
Multiple view geometry in computer vision
Richard Hartley,Andrew Zisserman +1 more
TL;DR: In this article, the authors provide comprehensive background material and explain how to apply the methods and implement the algorithms directly in a unified framework, including geometric principles and how to represent objects algebraically so they can be computed and applied.
Multiple View Geometry in Computer Vision.
TL;DR: This book is referred to read because it is an inspiring book to give you more chance to get experiences and also thoughts and it will show the best book collections and completed collections.
Book
Level Set Methods and Dynamic Implicit Surfaces
Stanley Osher,Ronald Fedkiw +1 more
TL;DR: A student or researcher working in mathematics, computer graphics, science, or engineering interested in any dynamic moving front, which might change its topology or develop singularities, will find this book interesting and useful.
Proceedings ArticleDOI
KinectFusion: Real-time dense surface mapping and tracking
Richard Newcombe,Shahram Izadi,Otmar Hilliges,David Molyneaux,David Kim,Andrew J. Davison,Pushmeet Kohi,Jamie Shotton,Steve Hodges,Andrew Fitzgibbon +9 more
TL;DR: A system for accurate real-time mapping of complex and arbitrary indoor scenes in variable lighting conditions, using only a moving low-cost depth camera and commodity graphics hardware, which fuse all of the depth data streamed from a Kinect sensor into a single global implicit surface model of the observed scene in real- time.