Showing papers by "Brian Curless published in 2012"

3D puppetry: a kinect-based interface for 3D animation

Abstract: We present a system for producing 3D animations using physical objects (i.e., puppets) as input. Puppeteers can load 3D models of familiar rigid objects, including toys, into our system and use them as puppets for an animation. During a performance, the puppeteer physically manipulates these puppets in front of a Kinect depth sensor. Our system uses a combination of image-feature matching and 3D shape matching to identify and track the physical puppets. It then renders the corresponding 3D models into a virtual set. Our system operates in real time so that the puppeteer can immediately see the resulting animation and make adjustments on the fly. It also provides 6D virtual camera \\rev{and lighting} controls, which the puppeteer can adjust before, during, or after a performance. Finally our system supports layered animations to help puppeteers produce animations in which several characters move at the same time. We demonstrate the accessibility of our system with a variety of animations created by puppeteers with no prior animation experience.

DuploTrack: a real-time system for authoring and guiding duplo block assembly

Abstract: We demonstrate a realtime system which infers and tracks the assembly process of a snap-together block model using a Kinect® sensor. The inference enables us to build a virtual replica of the model at every step. Tracking enables us to provide context specific visual feedback on a screen by augmenting the rendered virtual model aligned with the physical model. The system allows users to author a new model and uses the inferred assembly process to guide its recreation by others. We propose a novel way of assembly guidance where the next block to be added is rendered in blinking mode with the tracked virtual model on screen. The system is also able to detect any mistakes made and helps correct them by providing appropriate feedback. We focus on assemblies of Duplo® blocks. We discuss the shortcomings of existing methods of guidance - static figures or recorded videos - and demonstrate how our method avoids those shortcomings. We also report on a user study to compare our system with standard figure-based guidance methods found in user manuals. The results of the user study suggest that our method is able to aid users' structural perception of the model better, leads to fewer assembly errors, and reduces model construction time.

Schematic surface reconstruction

Abstract: This paper introduces a schematic representation for architectural scenes together with robust algorithms for reconstruction from sparse 3D point cloud data. The schematic models architecture as a network of transport curves, approximating a floorplan, with associated profile curves, together comprising an interconnected set of swept surfaces. The representation is extremely concise, composed of a handful of planar curves, and easily interpretable by humans. The approach also provides a principled mechanism for interpolating a dense surface, and enables filling in holes in the data, by means of a pipeline that employs a global optimization over all parameters. By incorporating a displacement map on top of the schematic surface, it is possible to recover fine details. Experiments show the ability to reconstruct extremely clean and simple models from sparse structure-from-motion point clouds of complex architectural scenes.

Frequency-space decomposition and acquisition of light transport under spatially varying illumination

Abstract: We show that, under spatially varying illumination, the light transport of diffuse scenes can be decomposed into direct, near-range (subsurface scattering and local inter-reflections) and far-range transports (diffuse inter-reflections) We show that these three component transports are redundant either in the spatial or the frequency domain and can be separated using appropriate illumination patterns We propose a novel, efficient method to sequentially separate and acquire the component transports First, we acquire the direct transport by extending the direct-global separation technique from floodlit images to full transport matrices Next, we separate and acquire the near-range transport by illuminating patterns sampled uniformly in the frequency domain Finally, we acquire the far-range transport by illuminating low-frequency patterns We show that theoretically, our acquisition method achieves the lower bound our model places on the required number of patterns We quantify the savings in number of patterns over the brute force approach We validate our observations and acquisition method with rendered and real examples throughout

Photo Tours

Abstract: This paper describes an effort to automatically create ``tours'' of thousands of the world's landmarks from geo-tagged user-contributed photos on the Internet. These photo tours take you through each site's most popular viewpoints on a tour that maximizes visual quality and traversal efficiency. This planning problem is framed as a form of the Traveling Salesman Problem on a graph with photos as nodes and transition costs on edges and pairs of edges, permitting efficient solution even for large graphs containing thousands of photos. Our approach is highly scalable and is the basis for the Photo Tours feature in Google Maps, which can be viewed at http://maps.google.com/phototours.

Refractive height fields from single and multiple images

Abstract: We propose a novel framework for reconstructing homogenous, transparent, refractive height-fields from a single viewpoint. The height-field is imaged against a known planar background, or sequence of backgrounds. Unlike existing approaches that do a point-by-point reconstruction — which is known to have intractable ambiguities — our method estimates and optimizes for the entire height-field at the same time. The formulation supports shape recovery from measured distortions (deflections) or directly from the images themselves, including from a single image. We report results for a variety of refractive height-fields showing significant improvement over prior art.