Decomposing Single Images for Layered Photo Retouching
read more
Citations
Learning to reconstruct shape and spatially-varying reflectance from a single image
Single-image SVBRDF capture with a rendering-aware deep network
Material Editing Using a Physically Based Rendering Network
Self-Supervised Intrinsic Image Decomposition
References
ImageNet Classification with Deep Convolutional Neural Networks
U-Net: Convolutional Networks for Biomedical Image Segmentation
Gradient-based learning applied to document recognition
ImageNet classification with deep convolutional neural networks
U-Net: Convolutional Networks for Biomedical Image Segmentation
Related Papers (5)
Frequently Asked Questions (12)
Q2. What have the authors stated for future works in "Decomposing single images for layered photo retouching" ?
Future work could investigate other decompositions such as global and direct illumination, subsurface-scattering or directional illumination or other inputs, such as videos.
Q3. What is the deconvolution part of the network?
The deconvolution part of the network consists of blocks performing a resize-convolution (upsampling followed by a stride-one convolution), cross-linking and a stride-one convolution.
Q4. How many random instances of ShapeNet are there?
Surface geometry consists of about 2,000 random instances from ShapeNet [C∗15] coming from the top-level classes, selected from ShapeNetCore semi-automatically.
Q5. What is the main step in the decomposition of a single image?
The decomposition has two main steps: (i) producing training data (Sec. 3.2) and (ii) a convolutional neural network to decompose single images into editable layers (Sec. 3.3).
Q6. How do professional photographers decompose their photos into one image?
They do so, by ‘decomposing’ the scene into individual layers, e. g., by changing the scenes physical illumination, manipulating the individual layers (e. g., typically using a software such as Adobe Photoshop), and then composing them into a single image.
Q7. How many components are used to produce C?
To produce C, the authors compute four individual components, that can be composed into Eq. 1 or further into directions according to Eq. 2 as per-pixel normals are known at render time.
Q8. What is the main part of the approach?
Their approach has two main parts: an imaging model that describes a decomposition of a single photo into layers for individual editing and a method to perform this decomposition.
Q9. What is the way to perform a decomposition of a single image?
For single images, a more classic approach is to perform intrinsic decomposition into shading (irradiance) and diffuse reflectance (albedo) [BT78, GMLMG12, BBS14], possibly supported by a dedicated UI for images [BPD09, BBPD12], using annotated data [BBS14, ZKE15, ZIKF15], or videos [BST∗14, YGL∗14].
Q10. What is the effect of editing pixels corresponding to negative SH contributions?
the effect of editing pixels corresponding to negative SH contributions is not easily evident to the user; 2. the strong overlap between basis functions makes it difficult to apply desired edits for individual spatial directions only.
Q11. How many images can be produced in eight hours?
Overall the authors produce 300,000 unique samples in a resolution of 256×256 (ca. 14 GB) in eight hours on a current PC with a higher-end GPU.
Q12. How many layers can be combined using a compositing software?
Using this basis, Equation (2) produces 14 layers from an input image – where twelve are directionally-dependent and two are not (ρ and Oa) – that can be combined using any compositing software.