scispace - formally typeset
Search or ask a question
Proceedings ArticleDOI

A Novel Algebaric Variety Based Model for High Quality Free-Viewpoint View Synthesis on a Krylov Subspace

TL;DR: A new depth-image-based rendering algorithm for free-viewpoint 3DTV applications that effectively handles artifacts appear in wide-baseline spatial view interpolation and arbitrary camera movements and results excel with state-of-the-art methods in quantitative and qualitative evaluation.
Abstract: This paper presents a new depth-image-based rendering algorithm for free-viewpoint 3DTV applications. The cracks, holes, ghost countors caused by visibility, disocclusion, resampling problems associated with 3D warping lead to serious rendering artifacts in synthesized virtual views. This challenging problem of hole filling is formulated as an algebraic matrix completion problem on a higher dimensional space of monomial features described by a novel variety model. The high-level idea of this work is to exploit the linear or nonlinear structures of the data and interpolate missing values by solving algebraic varieties associated with Hankel matrices as a member of Krylov subspace. The proposed model effectively handles artifacts appear in wide-baseline spatial view interpolation and arbitrary camera movements. Our model has a low runtime and results excel with state-of-the-art methods in quantitative and qualitative evaluation.
Citations
More filters
Book
01 Jan 2017
TL;DR: This book discusses Graph Theory concepts and definitions used in Image Processing and Analysis, and the role of Graphs in Matching Shapes and in Categorization in GED Computation Applications of GED.
Abstract: Graph Theory Concepts and Definitions Used in Image Processing and Analysis, O. Lezoray and L. Grady Introduction Basic Graph Theory Graph Representation Paths, Trees, and Connectivity Graph Models in Image Processing and Analysis Graph Cuts-Combinatorial Optimization in Vision, H. Ishikawa Introduction Markov Random Field Basic Graph Cuts: Binary Labels Multi-Label Minimization Examples Higher-Order Models in Computer Vision, P. Kohli and C. Rother Introduction Higher-Order Random Fields Patch and Region-Based Potentials Relating Appearance Models and Region-Based Potentials Global Potentials Maximum a Posteriori Inference A Parametric Maximum Flow Approach for Discrete Total Variation Regularization, A. Chambolle and J. Darbon Introduction Idea of the approach Numerical Computations Applications Targeted Image Segmentation Using Graph Methods, L. Grady The Regularization of Targeted Image Segmentation Target Specification Conclusion A Short Tour of Mathematical Morphology on Edge and Vertex Weighted Graphs, L. Najman and F. Meyer Introduction Graphs and lattices Neighborhood Operations on Graphs Filters Connected Operators and Filtering with the Component Tree Watershed Cuts MSF Cut Hierarchy and Saliency Maps Optimization and the Power Watershed Partial Difference Equations on Graphs for Local and Nonlocal Image Processing, A. Elmoataz, O. Lezoray, V.-T. Ta, and S. Bougleux Introduction Difference Operators on Weighted Graphs Construction of Weighted Graphs p-Laplacian Regularization on Graphs Examples Image Denoising with Nonlocal Spectral Graph Wavelets, D.K. Hammond, L. Jacques, and P. Vandergheynst Introduction Spectral Graph Wavelet Transform Nonlocal Image Graph Hybrid Local/Nonlocal Image Graph Scaled Laplacian Model Applications to Image Denoising Conclusions Acknowledgments Image and Video Matting, J. Wang Introduction Graph Construction for Image Matting Solving Image Matting Graphs Data Set Video Matting Optimal Simultaneous Multisurface and Multiobject Image Segmentation, X. Wu, M.K. Garvin, and M. Sonka Introduction Motivation and Problem Description Methods for Graph-Based Image Segmentation Case Studies Conclusion Acknowledgments Hierarchical Graph Encodings, L. Brun and W. Kropatsch Introduction Regular Pyramids Irregular Pyramids Parallel construction schemes Irregular Pyramids and Image properties Graph-Based Dimensionality Reduction, J.A. Lee and M. Verleysen Summary Introduction Classical methods Nonlinearity through Graphs Graph-Based Distances Graph-Based Similarities Graph embedding Examples and comparisons Graph Edit Distance-Theory, Algorithms, and Applications, M. Ferrer and H. Bunke Introduction Definitions and Graph Matching Theoretical Aspects of GED GED Computation Applications of GED The Role of Graphs in Matching Shapes and in Categorization, B. Kimia Introduction Using Shock Graphs for Shape Matching Using Proximity Graphs for Categorization Conclusion Acknowledgment 3D Shape Registration Using Spectral Graph Embedding and Probabilistic Matching, A. Sharma, R. Horaud, and D. Mateus Introduction Graph Matrices Spectral Graph Isomorphism Graph Embedding and Dimensionality Reduction Spectral Shape Matching Experiments and Results Discussion Appendix: Permutation and Doubly- stochastic Matrices Appendix: The Frobenius Norm Appendix: Spectral Properties of the Normalized Laplacian Modeling Images with Undirected Graphical Models, M.F. Tappen Introduction Background Graphical Models for Modeling Image Patches Pixel-Based Graphical Models Inference in Graphical Models Learning in Undirected Graphical Models Tree-Walk Kernels for Computer Vision, Z. Harchaoui and F. Bach Introduction Tree-Walk Kernels as Graph Kernels The Region Adjacency Graph Kernel as a Tree-Walk Kernel The Point Cloud Kernel as a Tree-Walk Kernel Experimental Results Conclusion Acknowledgments

8 citations

Journal ArticleDOI
TL;DR: The depth estimation problem is revisits, avoiding the explicit stereo matching step using a simple two-tower convolutional neural network, and the proposed algorithm is entitled 2T-UNet, which surpasses state-of-the-art monocular and stereo depth estimation methods on the challenging Scene dataset.
Abstract: —Stereo correspondence matching is an essential part of the multi-step stereo depth estimation process. This paper revisits the depth estimation problem, avoiding the explicit stereo matching step using a simple two-tower convolutional neural network. The proposed algorithm is entitled as 2T-UNet. The idea behind 2T-UNet is to replace cost volume construction with twin convolution towers. These towers have an allowance for different weights between them. Additionally, the input for twin encoders in 2T-UNet are different compared to the existing stereo methods. Generally, a stereo network takes a right and left image pair as input to determine the scene geometry. However, in the 2T-UNet model, the right stereo image is taken as one input and the left stereo image along with its monocular depth clue information, is taken as the other input. Depth clues provide complementary suggestions that help enhance the quality of predicted scene geometry. The 2T-UNet surpasses state-of-the-art monocular and stereo depth estimation methods on the challenging Scene flow dataset, both quantitatively and qualitatively. The architecture performs incredibly well on complex natural scenes, highlight- ing its usefulness for various real-time applications. Pretrained weights and code will be made readily available.
References
More filters
Proceedings ArticleDOI
23 Oct 1995
TL;DR: Texture correction, which uses an estimate of a non-Lambertian reflectivity model, is done in such a way to simulate the migration of reflections due to the change of viewpoint, and has been tested on real images, producing realistic synthesized views.
Abstract: A technique for the synthesis of virtual views of a 3D scene, starting from images taken by a calibrated multicamera system, is proposed and tested. Surface interpolation is performed over a set of 3D edges, computed with stereometric algorithms, and additional curvature-tuning points, scattered where the reflectivity model is sufficiently reliable. The 3D coordinates of the tuning points are computed by minimizing the MSE between the available real views and the corresponding synthetic views. Synthesis is finally carried out by reprojecting on the new image plane the estimated object surface over which texture-mapping of the reflection-corrected luminance function has been performed. Texture correction, which uses an estimate of a non-Lambertian reflectivity model, is done in such a way to simulate the migration of reflections due to the change of viewpoint. The technique has been tested on real images, producing realistic synthesized views.

4 citations


"A Novel Algebaric Variety Based Mod..." refers background in this paper

  • ...Employing more views and doing dense view interpolation can help model such effects [32]....

    [...]

Journal ArticleDOI
TL;DR: An equivalent condition for the Hankel matrix to be a Krylov matrix of C T by the notion of compatibility is discussed, and new determinant formulas for such Hankel matrices in terms of eigenvalues and eigenvectors of C respectively are derived.

2 citations


"A Novel Algebaric Variety Based Mod..." refers background in this paper

  • ...[28] describe close relation between Hankel matrices and the Krylov matrices....

    [...]

Posted Content
09 Sep 2018
TL;DR: Two novel algorithms for spectral clustering of a subset of the graph vertices (target subset) based on the theory of model order reduction rely on realizations of a reduced order model (ROM) that accurately approximates the diffusion transfer function of the original graph for inputs and outputs restricted to the target subset.
Abstract: Clustering via graph-Laplacian spectral imbedding is ubiquitous in data science and machine learning. However, it becomes less efficient for large data sets due to two factors. First, computing the partial eigendecomposition of the graph-Laplacian typically requires a large Krylov subspace. Second, after the spectral imbedding is complete, the clustering is typically performed with various relaxations of k-means, which may become prone to getting stuck in local minima and scale poorly in terms of computational cost for large data sets. Here we propose two novel algorithms for spectral clustering of a subset of the graph vertices (target subset) based on the theory of model order reduction. They rely on realizations of a reduced order model (ROM) that accurately approximates the diffusion transfer function of the original graph for inputs and outputs restricted to the target subset. While our focus is limited to this subset, our algorithms produce its clustering that is consistent with the overall structure of the graph. Moreover, working with a small target subset reduces greatly the required dimension of Krylov subspace and allows to exploit the approximations of k-means in the regimes when they are most robust and efficient, as verified by the numerical experiments. There are several uses for our algorithms. First, they can be employed on their own to clusterize a representative subset in cases when the full graph clustering is either infeasible or not required. Second, they may be used for quality control. Third, as they drastically reduce the problem size, they enable the application of more powerful approximations of k-means like those based on semi-definite programming (SDP) instead of the conventional Lloyd's algorithm. Finally, they can be used as building blocks of a divide-and-conquer algorithm for the full graph clustering. The latter will be reported in a separate article.

2 citations


"A Novel Algebaric Variety Based Mod..." refers background or methods in this paper

  • ...The clustering performance is notably improved employing Krylov subspace model reduction [21]....

    [...]

  • ...Thus, we improve efficiency in depth map clusterization by working on a subset of the graph vertices and projecting the graph Laplacian on a Krylov subspace, as described by [21]....

    [...]

  • ...Moreover, working with a small feasible vertices subset reduces greatly Krylov subspace dimension compared to conventional eigensolvers and approximate k-means in the regimes when most robust and efficient, as verified by the numerical experiments [21]....

    [...]

  • ...The conventional k-means becomes less precise and not cost efficient [20, 21]....

    [...]

Journal ArticleDOI
28 Apr 2014
TL;DR: In this paper, it was shown that sets of varieties, tied to a given Hankel matrix, resemble a set of hyperplanes as regards the number of points of their intersections.
Abstract: Let be a class of Hankel matrices whose entries, depending on a given matrix , are linear forms in variables with coefficients in a finite field . For every matrix in , it is shown that the varieties specified by the leading minors of orders from 1 to have the same number of points in . Further properties are derived, which show that sets of varieties, tied to a given Hankel matrix, resemble a set of hyperplanes as regards the number of points of their intersections.

1 citations


"A Novel Algebaric Variety Based Mod..." refers background in this paper

  • ...It is possible to uniquely recover missing entries by leveraging induced linear structure on a Krylov space [29]....

    [...]