# Metric-aware processing of spherical imagery

Abstract: Processing spherical images is challenging. Because no spherical parameterization is globally uniform, an accurate solver must account for the spatially varying metric. We present the first efficie...

## Summary (3 min read)

### 1 Introduction

- Spherical images are often used to represent photographed or rendered environments.
- One limitation is that a uniform grid in the parametric domain oversamples the sphere near the poles.
- (2) The other strategy is to define a metric-aware system, relying on the bounded distortion to provide good conditioning.
- Thereby inheriting the advantages of differentiability, axial symmetry, and regularity.the authors.
- The authors demonstrate the new representation and multigrid solver using two quite different applications: reaction-diffusion simulation (for texture synthesis) and spherical gradient-domain image processing (for panorama stitching and enhancement).

### 3 Motivation and Approach

- Many interesting image processing operations can be expressed in the gradient domain, including image stitching, dynamic range reduction, removal of shadows or reflections, and gradient-based sharpening [Agrawal and Raskar 2007].
- The basic strategy is to extract gradient fields from one or more images, modify the gradient values, and find the new image that best approximates the desired gradients.
- Even for parameterizations like HEALPix [Górski et al. 2005] and Rhombic Dodecahedron [Fu et al. 2009], which are specifically designed to have low distortion, the spatial nonuniformity of the finite differences/elements introduces significant errors.
- In the discretization of the metric-aware finite element system, the coefficients of matrices L and D in (3) are expensive to compute because they require inner products of functions defined on the sphere.

### 4 Geometry-Aware Finite Element Solution

- In this section the authors describe the basic approach using a regular tessellation of the equirectangular domain.
- The design of the system incorporates many of the basic building blocks that the authors leverage in their well-conditioned adapted discretization, described in Section 5.
- The main tasks in defining a metric-aware spherical solver are to choose the set of spherical finite elements that span the subspace of functions, and to compute the coefficients of the linear system using the sphere’s metric.
- The authors first show how an equirectangular parameterization can be used to transform Bsplines defined on a planar domain into B-spline-like functions on the sphere.
- Using these functions, the authors derive the coefficients of the linear system.

### 4.1 Choosing the Finite Elements

- Second-order B-splines have many benefits, including a tensorproduct structure that simplifies computation, continuous first derivatives, a small support that results in sparse linear systems, and a nesting structure compatible with a multigrid solver.
- Next the authors create test functions near the domain boundaries in accordance with the spherical topology.

### 4.2 Defining the Linear System

- (5) Because the functions FI are tensor products of B-splines, computing the system coefficients reduces to integrating products of polynomials with trigonometric functions.
- For the cosecant term arising in the Laplace-Beltrami coefficients,∫ xn/sin(x)dx, the authors can compute the integral by evaluating the polylogarithm function (e.g. using the Cephes Library [Cephes 1995]).

### 4.4 Efficient System Solution

- Creating the linear system using an equirectangular parameterization provides two efficiency advantages.
- Thus, the authors can re-use the computation performed in defining the row of the system matrix corresponding to element Fi,0 to set the values for all other elements.
- The second advantage is that the resulting matrix L is banded, because elements in one image row only overlap elements in nearby rows.
- Thus, a θ -major data ordering lets us implement a streaming solver akin to that described in [Kazhdan and Hoppe 2008].

### 5 Adapted Elements for Better Conditioning

- While the finite element system developed in the previous section offers a multiresolution hierarchy, it is difficult to directly leverage this hierarchy in a classical multigrid solver.
- To overcome this, the authors use the equirectangular map, but adaptively discretize its domain to bound the anisotropy of the resulting elements .
- Note that, as in [Grinspun et al. 2002], their implementation has the B-splines centered on the facets of the parameterization and defined independently of the size and shape of adjacent faces.
- Thus, the function-space does not exhibit any T-junction discontinuities.
- The key to establishing the transition between the adapted and nonadapted system is the definition of a refinement operator R that expresses elements in the adapted system as linear combinations of elements in the non-adapted system from Section 4.

### 6 Results

- The design of their adapted equirectangular finite-element system is motivated by several key requirements.
- From the theoretical perspective, their goal is to design a system that correctly incorporates information about the geometry of the sphere.
- From the numerical perspective, their goal is to design a multigrid solver that exhibits the convergence properties of standard multigrid solvers defined over regular grids.
- And finally, from a practical perspective, their goal is to design a solver that is both fast and scalable, supporting image processing over large spherical panoramas.

### 6.1 Geometry Awareness

- As discussed in Section 3, even when the parameterization of the sphere is chosen to be either area-preserving or approximately area- and angle-preserving (Rhombic Dodecahedron), the obtained solution can deviate significantly from the true solution.
- This is demonstrated in the table in Figure 3 which gives the RMS difference between the correct (spherical harmonic) solution and the one returned by the different solvers.

### 6.2 Multigrid Convergence

- To address the anisotropy of the regular equirectangular parameterization at the poles, the authors have proposed adapted equirectangular elements that define an approximately uniform partition of the sphere.
- As the table in Figure 5 shows, their adapted elements resolve the difficulty with anisotropy, providing a multigrid solver that quickly converges to the correct solution.
- Furthermore, this table demonstrates that as the number of V-cycles is increased, the solver exhibits the exponential decay in residual norm typical of multigrid solvers defined over regular domains.

### 6.3 Reaction-diffusion processing on the sphere

- To demonstrate the practical implications of geometry awareness and multigrid convergence, the authors use the different parameterizations to perform the semi-implicit time-stepping in a reaction-diffusion process, shown in Figure 6.
- As observed by Turk [1991] and Witkin et al. [1991], failure to accommodate the geometry in a reaction-diffusion process can result in undesirable artifacts, and the authors see these in the synthesis of Turing’s “spots” texture [1952].
- For TOAST and Cube-Map, which do not preserve area, the spots have non-uniform sizes.
- By comparison, their efficient metric-aware solver obtains nicely uniform spots.

### 6.4 Spherical Image Processing

- The authors apply their adapted equirectangular elements to two problems in spherical image processing: stitching and sharpening.
- The authors do this by first constructing a target vector-field representing the approximate gradients of the seamless image and then solving the Poisson equation to fit an image to this vector-field [Pérez et al.
- It can be seen that in regions of low-frequency, where the gradients are small, the value interpolation constraints dominate and the sharpened image preserves the original colors.
- As discussed in [Kazhdan et al. 2010], metric-unaware solvers can still provide convincing solutions for image-processing applications such as stitching and sharpening.
- The authors have confirmed the accuracy of their solver by comparing the above results with “ground-truth” results obtained using a solver running five V-cycles, with ten Gauss-Seidel iterations per level, and using double-precision floating point values.

### 7 Conclusion

- The authors have introduced an adapted finite element system that supports the geometry-aware processing of spherical imagery.
- Using this system, the authors have shown how to extend traditional image processing techniques to the sphere, without sacrificing either speed or accuracy.
- In doing so, the authors have maintained essential regularity of the parameterization, enabling the design of a streaming multigrid solver that supports the processing of huge images.
- In addition, because this representation is a simple extension of a commonly used spherical parameterization, the equirectangular map, their technique can be readily applied to existing datasets without any lossy resampling or expensive conversion.

Did you find this useful? Give us your feedback

...read more

##### Citations

197 citations

177 citations

14 citations

10 citations

4 citations

##### References

9,015 citations

4,869 citations

4,063 citations

1,214 citations

1,062 citations

##### Related Papers (5)

##### Frequently Asked Questions (2)

###### Q2. What future works have the authors mentioned in the paper "Metric-aware processing of spherical imagery" ?

Using this system, the authors have shown how to extend traditional ( planar ) image processing techniques to the sphere, without sacrificing either speed or accuracy. In addition, because this representation is a simple extension of a commonly used spherical parameterization, the equirectangular map, their technique can be readily applied to existing datasets without any lossy resampling or expensive conversion.