Relightable Buildings from Images

TLDR: A complete image-based process that facilitates recovery of both gross-scale geometry and local surface structure to create highly detailed 3D models of building facades from photographs, imparting the illusion of measured reflectance is proposed.

Abstract: We propose a complete image-based process that facilitates recovery of both gross-scale geometry and local surface structure to create highly detailed 3D models of building facades from photographs. We approximate both albedo and sufficient local geometric structure to compute complex self-shadowing effects, and fuse this with a gross-scale 3D model. Our approach yields a perceptually high-quality model, imparting the illusion of measured reflectance. The requirements of our approach are that image capture must be performed under diffuse lighting and surfaces in the images must be predominantly Lambertian. Exemplars of materials are obtained through surface depth hallucination, and our novel method matches these with multi-view image sequences that are also used to automatically recover 3D geometry. In this paper we illustrate the effictiveness of our approach through visually realistic reconstructions of historic buildings in two test cases, together with a table showing the breakdown of effort required to reconstruct each.