Showing papers by "Matthew Turk published in 2017"
TL;DR: This work presents a new approach to rendering a geometrically-correct user-perspective view for a magic lens interface, based on leveraging the gradients in the real world scene, and couples a recent gradient-domain image-based rendering method with a novel semi-dense stereo matching algorithm.
Abstract: We present a new approach to rendering a geometrically-correct user-perspective view for a magic lens interface, based on leveraging the gradients in the real world scene. Our approach couples a recent gradient-domain image-based rendering method with a novel semi-dense stereo matching algorithm. Our stereo algorithm borrows ideas from PatchMatch, and adapts them to semi-dense stereo. This approach is implemented in a prototype device build from off-the-shelf hardware, with no active depth sensing. Despite the limited depth data, we achieve high-quality rendering for the user-perspective magic lens.
14 citations
20 Sep 2017
TL;DR: A prototype of a smartphone-based virtual reality (VR) viewer, which can cover nearly the full human field-of-view (FOV) and suggests that such extensions are feasible ways to significantly expand the FOV of standard VR viewers.
Abstract: We present a prototype of a smartphone-based virtual reality (VR) viewer, which can cover nearly the full human field-of-view (FOV). The prototype suggests that such extensions are feasible ways to significantly expand the FOV of standard VR viewers. The concept can be employed for future VR viewers or it can even be retrofitted to some existing ones.
9 citations
07 Jun 2017
TL;DR: Novel optics and head-mounted display prototypes, which have the widest reported field-of-view (FOV), and which can cover the full human FOV or even beyond, are presented.
Abstract: We present novel optics and head-mounted display (HMD) prototypes, which have the widest reported field-of-view (FOV), and which can cover the full human FOV or even beyond They are based on lenses and screens which are curved around the eyes While this is still work-in-progress, the HMD prototypes and user tests suggest a feasible approach to significantly expand the FOV of HMDs
6 citations
08 Nov 2017
TL;DR: Experimental results found that the snapping-to-photos interfaces are preferred over the baseline fully constrained- to-photos interface, that there exist differences between indoor and outdoor scenes, and that users preferred and were able to reach target photos better with click-to -snap point-of-interest snapping compared to automatic point- of-view snapping.
Abstract: Navigating through a virtual, 3D reconstructed scene has recently become very important in many applications. A popular approach is to virtually travel to the photos used in reconstructing the scene; such an approach may be generally termed a "snapping-to-photos" virtual travel interface. While previous work has either used fully constrained interfaces (always at the photos) or minimally constrained interfaces (free-flight navigation), in this paper we introduce new snapping-to-photos interfaces that lie in between these two extremes. Our snapping-to-photos interfaces snap the view to a photo in 3D based on viewpoint similarity and optionally the user's mouse cursor or finger-tap position. Experimental results, with both indoor and outdoor scene reconstructions, found that our snapping-to-photos interfaces are preferred over the baseline fully constrained-to-photos interface, that there exist differences between indoor and outdoor scenes, and that users preferred and were able to reach target photos better with click-to-snap point-of-interest snapping compared to automatic point-of-view snapping.
4 citations
01 Sep 2017
TL;DR: ANSAC as discussed by the authors is a RANSAC-based estimator that accounts for noise by adaptively using more than the minimal number of correspondences required to generate a hypothesis.
Abstract: While RANSAC-based methods are robust to incorrect image correspondences (outliers), their hypothesis generators are not robust to correct image correspondences (inliers) with positional error (noise). This slows down their convergence because hypotheses drawn from a minimal set of noisy inliers can deviate significantly from the optimal model. This work addresses this problem by introducing ANSAC, a RANSAC-based estimator that accounts for noise by adaptively using more than the minimal number of correspondences required to generate a hypothesis. ANSAC estimates the inlier ratio (the fraction of correct correspondences) of several ranked subsets of candidate correspondences and generates hypotheses from them. Its hypothesis-generation mechanism prioritizes the use of subsets with high inlier ratio to generate high-quality hypotheses. ANSAC uses an early termination criterion that keeps track of the inlier ratio history and terminates when it has not changed significantly for a period of time. The experiments show that ANSAC finds good homography and fundamental matrix estimates in a few iterations, consistently outperforming state-of-the-art methods.
4 citations
Posted Content•
TL;DR: ANSAC as discussed by the authors is a RANSAC-based estimator that accounts for noise by adaptively using more than the minimal number of correspondences required to generate a hypothesis.
Abstract: While RANSAC-based methods are robust to incorrect image correspondences (outliers), their hypothesis generators are not robust to correct image correspondences (inliers) with positional error (noise). This slows down their convergence because hypotheses drawn from a minimal set of noisy inliers can deviate significantly from the optimal model. This work addresses this problem by introducing ANSAC, a RANSAC-based estimator that accounts for noise by adaptively using more than the minimal number of correspondences required to generate a hypothesis. ANSAC estimates the inlier ratio (the fraction of correct correspondences) of several ranked subsets of candidate correspondences and generates hypotheses from them. Its hypothesis-generation mechanism prioritizes the use of subsets with high inlier ratio to generate high-quality hypotheses. ANSAC uses an early termination criterion that keeps track of the inlier ratio history and terminates when it has not changed significantly for a period of time. The experiments show that ANSAC finds good homography and fundamental matrix estimates in a few iterations, consistently outperforming state-of-the-art methods.
2 citations
24 Mar 2017
TL;DR: This work takes semi-dense depth maps and converts them into a dense scene model, suitable for rendering plausible novel views of the scene using conventional image-based rendering, and densifies depth maps at the rate they are generated, and enables to generate novel Views of live scenes with no pre-capture or preprocessing.
Abstract: In this paper, we consider the problem of rendering novel views of a live unprepared scene from video input, important to many application scenarios (such as telepresence and remote collaboration). We present an optimization approach to improving incomplete scene reconstructions captured in real time with a single moving monocular camera. We take semi-dense depth maps and convert them into a dense scene model, suitable for rendering plausible novel views of the scene using conventional image-based rendering. Our implementation densifies depth maps at the rate they are generated, and enables us to generate novel views of live scenes with no pre-capture or preprocessing. In evaluations comparing with other approaches, our method performs well even on difficult scenes, and results in higherquality novel views.
1 citations