We propose the concept of a generalized assorted pixel (GAP) camera, which enables the user to capture a single image of a scene and, after the fact, control the tradeoff between spatial resolution, dynamic range and spectral detail. The GAP camera uses a complex array (or mosaic) of color filters. A major problem with using such an array is that the captured image is severely under-sampled for at least some of the filter types. This leads to reconstructed images with strong aliasing. We make four contributions in this paper: 1) we present a comprehensive optimization method to arrive at the spatial and spectral layout of the color filter array of a GAP camera. 2) We develop a novel algorithm for reconstructing the under-sampled channels of the image while minimizing aliasing artifacts. 3) We demonstrate how the user can capture a single image and then control the tradeoff of spatial resolution to generate a variety of images, including monochrome, high dynamic range (HDR) monochrome, RGB, HDR RGB, and multispectral images. 4) Finally, the performance of our GAP camera has been verified using extensive simulations that use multispectral images of real world scenes. A large database of these multispectral images has been made available at http://wwwl.cs.columbia.edu/ CAVE/projects/gap_camera/ for use by the research community.

/pdf/generalized-assorted-pixel-camera-postcapture-control-of-4mrfe9pr06.pdf

Generalized Assorted Pixel Camera: Postcapture Control of Resolution, Dynamic Range, and Spectrum

Defining Color. Describing Color. Measuring Color. Measuring Color Quality. Colorants. Producing Colors. Back to Principles. Appendix. Bibliography. Index.

Billmeyer and Saltzman's Principles of Color Technology

Computational color constancy is the task of estimating the true reflectances of visible surfaces in an image. In this paper we follow a line of research that assumes uniform illumination of a scene, and that the principal step in estimating reflectances is the estimation of the scene illuminant. We review recent approaches to illuminant estimation, firstly those based on formulae for normalisation of the reflectance distribution in an image - so-called grey-world algorithms, and those based on a Bayesian formulation of image formation. In evaluating these previous approaches we introduce a new tool in the form of a database of 568 high-quality, indoor and outdoor images, accurately labelled with illuminant, and preserved in their raw form, free of correction or normalisation. This has enabled us to establish several properties experimentally. Firstly automatic selection of grey-world algorithms according to image properties is not nearly so effective as has been thought. Secondly, it is shown that Bayesian illuminant estimation is significantly improved by the improved accuracy of priors for illuminant and reflectance that are obtained from the new dataset.

/pdf/bayesian-color-constancy-revisited-1w73puwppn.pdf

Bayesian color constancy revisited

Thanks to some technical progress in interferencefilter design based on different technologies, we can finally successfully implement the concept of multispectral filter array-based sensors. This article provides the relevant state-of-the-art for multispectral imaging systems and presents the characteristics of the elements of our multispectral sensor as a case study. The spectral characteristics are based on two different spatial arrangements that distribute eight different bandpass filters in the visible and near-infrared area of the spectrum. We demonstrate that the system is viable and evaluate its performance through sensor spectral simulation.

Multispectral Filter Arrays: Recent Advances and Practical Implementation

Imaging device analysis systems and imaging device analysis methods are described. According to one embodiment, an imaging device analysis system includes a light source configured to output light for use in analyzing at least one imaging component of an imaging device, wherein the imaging device is configured to generate images responsive to received light, and processing circuitry coupled with the light source and configured to control the light source to optically communicate the light to the imaging device, wherein the processing circuitry is further configured to access image data generated by the imaging device responsive to the reception, by the imaging device, of the light from the light source and to process the image data to analyze an operational status of the at least one imaging component.

Imaging device analysis systems and imaging device analysis methods

A filter optimization was investigated to design a set of filters for a five channel multi-spectral camera, three of which result in high colorimetric performance when used alone, and the full set having high quality spectral performance. Each candidate filter was selected from a set of 33 glass filters with three different thicknesses where filters may be combined in optical series. The effectiveness of the filter selection is demonstrated by computer simulation taking into account the spectral sensitivity of the imaging system, and a comprehensive database of natural and man-made object spectral reflectances.

Digital camera filter design for colorimetric and spectral accuracy

To evaluate and optimally design spectral sensitivity functions for color input devices, a metric that incorporates practical, significant requirements is desired. The candidate metrics are Vora-Trussell's p-factor, a metric based on geometrical difference, and the proposed Unified Measure of Goodness, or UMG, which simultaneously considers the imaging noise and its propagation, colorimetric reproduction accuracy and multi-illuminant color correction. A systematic approach is presented to searching for an optimal set of spectral sensitivity functions from among the complete combinations of the given filter components. Comparative computation results show that μ-factor is not a competent metric for the optimal design of camera spectral sensitivity functions while UMG is able to pick out the optimum successfully. Furthermore, the ultimate optimal set has been obtained by selecting the set with highest μ-factor value from the sub-optimal collection obtained with UMG. This hierarchical approach comprehensively considers the advantages of both quality metrics. The candidates of the optimal sets based on the given filter components are experimentally tested and presented in the end of the article.

Unified Measure of Goodness and Optimal Design of Spectral Sensitivity Functions

In the evaluation and optimal design of digital camera spectral sensitivity (SS), it is useful to have a metric of goodness that agrees closely with the perceived color accuracy of human visual system. In this paper, Vora et al’s µ-factor based on the orthonormal color matching function space and a universal measure of goodness (UMG) based on CIE L * a * b * space with noise consideration were introduced to evaluate and optimize the camera spectral sensitivities. General physical constraints on spectral sensitivities were discussed in the paper and three feasible approaches to the optimal design of camera spectral sensitivities were proposed, compared and verified with simulated experiments.

Optimization of Camera Spectral Sensitivities

A method for optimizing a selection of spectral sensitivities for an imaging device includes determining a first set (52) of spectral sensitivities from an initial set (50) of spectral sensitivities based on an analysis of one of a universal measure of goodness (52), a mµ-Factor (54), and RMS noise (56). A second set (54) of spectral sensitivities from the first set of spectral sensitivities based on an analysis of a different one of a universal measure of goodness, a mµ-Factor, and RMS noise. The second set of spectral sensitivities is an optimized set of spectral sensitivities.

Shuxue Quan

Papers

Digital camera filter design for colorimetric and spectral accuracy

Unified Measure of Goodness and Optimal Design of Spectral Sensitivity Functions

Optimization of Camera Spectral Sensitivities

Method and system for optimizing a selection of spectral sensitivities

Optimal Design of Camera Spectral Sensitivity Functions Based on Practical Filter Components.