Whitened Expectation Propagation: Non-Lambertian Shape from Shading and Shadow

TLDR: This work proposes a variation of EP that exploits regularities in natural scene statistics to achieve run times that are linear in both number of pixels and clique size, and uses large, non-local cliques to exploit cast shadow, which is traditionally ignored in shape from shading.

Abstract: For problems over continuous random variables, MRFs with large cliques pose a challenge in probabilistic inference. Difficulties in performing optimization efficiently have limited the probabilistic models explored in computer vision and other fields. One inference technique that handles large cliques well is Expectation Propagation. EP offers run times independent of clique size, which instead depend only on the rank, or intrinsic dimensionality, of potentials. This property would be highly advantageous in computer vision. Unfortunately, for grid-shaped models common in vision, traditional Gaussian EP requires quadratic space and cubic time in the number of pixels. Here, we propose a variation of EP that exploits regularities in natural scene statistics to achieve run times that are linear in both number of pixels and clique size. We test these methods on shape from shading, and we demonstrate strong performance not only for Lambertian surfaces, but also on arbitrary surface reflectance and lighting arrangements, which requires highly non-Gaussian potentials. Finally, we use large, non-local cliques to exploit cast shadow, which is traditionally ignored in shape from shading.

Figures

Figure 2. Results of whitened EP under several reflectances and lighting conditions. The left column shows example potential functions φR(p, q|i). In each case, potentials are highly non-Gaussian. The potential φR(p, q|i) differs at each pixel depending on intensity; here the chosen intensity is given by the blue dot on each sphere. For each reflectance, inferred surfaces are shown for benchmark SfS images. Mean-squared image error is reported for each case.

Figure 1. Results of the Whitened EP algorithm for Lambertian SfS, in comparison with other methods. Subfigure c) shows the results of linear constraint node BP [17]. Subfigure d) shows the results of EP using a full covariance matrix. Each of these methods is able to satisfy the Lambertian constraint accurately. Whitened EP is able to perform comparably in a substantially less time than other methods and with greater flexibility towards dense factor graphs or large cliques. Subfigure e) shows the results of diagonal EP without whitening.