In the last 10 years, development in robotics, computer vision, and sensor technology has provided new spectral remote sensing tools to capture unprecedented ultra-high spatial and high spectral resolution with unmanned aerial vehicles (UAVs). This development has led to a revolution in geospatial data collection in which not only few specialist data providers collect and deliver remotely sensed data, but a whole diverse community is potentially able to gather geospatial data that fit their needs. However, the diversification of sensing systems and user applications challenges the common application of good practice procedures that ensure the quality of the data. This challenge can only be met by establishing and communicating common procedures that have had demonstrated success in scientific experiments and operational demonstrations. In this review, we evaluate the state-of-the-art methods in UAV spectral remote sensing and discuss sensor technology, measurement procedures, geometric processing, and radiometric calibration based on the literature and more than a decade of experimentation. We follow the 'journey' of the reflected energy from the particle in the environment to its representation as a pixel in a 2D or 2.5D map, or 3D spectral point cloud. Additionally, we reflect on the current revolution in remote sensing, and identify trends, potential opportunities, and limitations.

/pdf/quantitative-remote-sensing-at-ultra-high-resolution-with-1fowvgc7qt.pdf

Quantitative Remote Sensing at Ultra-High Resolution with UAV Spectroscopy: A Review of Sensor Technology, Measurement Procedures, and Data Correction Workflows

Mono-CCD color cameras acquire only one color component at each pixel by means of their color filter array (CFA) covering the CCD sensor. To obtain a color image, a procedure - called demosaicing - is then necessary to estimate the other two missing color components at each pixel. This chapter deals with the quality of color images generated in such a way. We attempt to determine which demosaicing method provides the best results according to several comparison criteria, particularly for subsequent needs of color image analyses like edge detection.

/pdf/comparison-of-color-demosaicing-methods-l2576lwwwm.pdf

Comparison of color demosaicing methods

Many algorithms for computer vision rely on locating interest points, or keypoints in each image, and calculating a feature description from the pixel region surrounding the interest point. This is in contrast to methods such as correlation, where a larger rectangular pattern is stepped over the image at pixel intervals and the correlation is measured at each location. The interest point is the, and often provides the scale, rotational, and illumination invariance attributes for the descriptor; the descriptor adds more detail and more invariance attributes. Groups of interest points and descriptors together describe the actual objects.

https://link.springer.com/content/pdf/10.1007%2F978-1-4302-5930-5_6.pdf

Interest Point Detector and Feature Descriptor Survey

Hyperspectral imaging is useful for applications ranging from medical diagnostics to agricultural crop monitoring; however, traditional scanning hyperspectral imagers are prohibitively slow and expensive for widespread adoption. Snapshot techniques exist but are often confined to bulky benchtop setups or have low spatio-spectral resolution. In this paper, we propose a novel, compact, and inexpensive computational camera for snapshot hyperspectral imaging. Our system consists of a tiled spectral filter array placed directly on the image sensor and a diffuser placed close to the sensor. Each point in the world maps to a unique pseudorandom pattern on the spectral filter array, which encodes multiplexed spatio-spectral information. By solving a sparsity-constrained inverse problem, we recover the hyperspectral volume with sub-super-pixel resolution. Our hyperspectral imaging framework is flexible and can be designed with contiguous or non-contiguous spectral filters that can be chosen for a given application. We provide theory for system design, demonstrate a prototype device, and present experimental results with high spatio-spectral resolution.

Spectral DiffuserCam: lensless snapshot hyperspectral imaging with a spectral filter array

Color image demosaicking for the Bayer color filter array is an essential image processing operation for acquiring high-quality color images. Recently, residual interpolation (RI)-based algorithms have demonstrated superior demosaicking performance over conventional color difference interpolation-based algorithms. In this paper, we propose adaptive residual interpolation (ARI) that improves existing RI-based algorithms by adaptively combining two RI-based algorithms and selecting a suitable iteration number at each pixel. These are performed based on a unified criterion that evaluates the validity of an RI-based algorithm. Experimental comparisons using standard color image datasets demonstrate that ARI can improve existing RI-based algorithms by more than 0.6 dB in the color peak signal-to-noise ratio and can outperform state-of-the-art algorithms based on training images. We further extend ARI for a multispectral filter array, in which more than three spectral bands are arrayed, and demonstrate that ARI can achieve state-of-the-art performance also for the task of multispectral image demosaicking.

Adaptive Residual Interpolation for Color and Multispectral Image Demosaicking.

Single-sensor color cameras, which classically use a color filter array to sample RGB channels, have recently been extended to the multispectral domain. To sample more than three wavelength bands, such systems use a multispectral filter array that provides a raw image in which a single channel value is available at each pixel. A demosaicing procedure is then needed to estimate a fully defined multispectral image. In this paper, we review multispectral demosaicing methods and propose a new one based on the pseudo-panchromatic image (PPI). Pixel values in the PPI are computed as the average spectral values. Experimental results show that our method provides estimated images of better quality than classical ones.

/pdf/multispectral-demosaicing-using-pseudo-panchromatic-image-18x99y79np.pdf

Multispectral Demosaicing Using Pseudo-Panchromatic Image

Texture description is a challenging problem with color images. Despite some attempts to include colors in local binary patterns (LBPs), no proposal has emerged as a color counterpart of grayscale LBPs. This is because colors are defined by vectors that are not naturally ordered and several ways exist to compare them. We propose an LBP extension that takes the vector information of color into account due to a color order. As several color orders are available and the selection of the most suitable one is difficult, we combine two of them in a texture descriptor called “mixed color order LBPs.” This small-size feature provides good performance on several benchmark databases for two classification problems with regard to larger-size LBP-based features of color textures.

Color local binary patterns: compact descriptors for texture classification

Single-sensor color cameras, which classically use a color filter array (CFA) to sample RGB channels, have recently been extended to the multispectral domain. To sample more than three wavelength bands, such systems use a multispectral filter array (MSFA) that provides a raw image in which a single channel level is available at each pixel. A demosaicing procedure is then needed to estimate a multispectral image with full spectral resolution. In this paper, we propose a new demosaicing method that takes spectral and spatial correlations into account by estimating the level for each channel. Experimental results show that it provides estimated images of better quality than classical methods.

/pdf/multispectral-demosaicing-using-intensity-based-spectral-1g1hm5gxas.pdf

Olivier Losson

Papers

Comparison of color demosaicing methods

Multispectral Demosaicing Using Pseudo-Panchromatic Image

Color local binary patterns: compact descriptors for texture classification

Multispectral demosaicing using intensity-based spectral correlation

Color texture analysis using CFA chromatic co-occurrence matrices