Multispectral Filter-Wheel Cameras: Geometric Distortion Model and Compensation Algorithms
TL;DR: A mathematical model of the distortions of the optical path is derived and it is shown that the color fringes vanish completely after application of two different algorithms for compensation.
Abstract: Multispectral image acquisition considerably improves color accuracy in comparison to RGB technology. A common multispectral camera design concept features a filter-wheel consisting of six or more optical bandpass filters. By shifting the filters sequentially into the optical path, the electromagnetic spectrum is acquired through the channels, thus making an approximate reconstruction of the spectrum feasible. However, since the optical filters exhibit different thicknesses, refraction indices and may not be aligned in a perfectly coplanar manner, geometric distortions occur in each spectral channel: The reconstructed RGB images thus show rainbow-like color fringes. To compensate for these, we analyze the optical path and derive a mathematical model of the distortions. Based on this model we present two different algorithms for compensation and show that the color fringes vanish completely after application of our algorithms. We also evaluate our compensation algorithms in terms of accuracy and execution time.
Citations
More filters
Patent•
28 Sep 2011TL;DR: In this paper, a processing module can process multi-spectral video images of individuals and detect skin blobs within different images of the multiscale video images, which can then be converted into time series signals and processed with a band pass filter.
Abstract: An embodiment generally relates to systems and methods for estimating heart rates of individuals using non-contact imaging. A processing module can process multi-spectral video images of individuals and detect skin blobs within different images of the multi-spectral video images. The skin blobs can be converted into time series signals and processed with a band pass filter. Further, the time series signals can be processed to separate pulse signals from unnecessary signals. The heart rate of the individual can be estimated according to the resulting time series signal processing.
86 citations
Patent•
17 Mar 2011
TL;DR: In this article, an iterative light transport based inversion algorithm was applied for correcting the intensity of the spectral images from the geometry/coupling effect as well as from the scattering amplitude distortions.
Abstract: Methods and apparatus for video rate or near video rate quantitative imaging of tissue physiological and morphological properties from visible/NIR light spectral images obtain rapid multi-spectral reflectance images by illuminating with a series of spectra containing multiple narrow wavelength bands. An iterative light- transport based inversion algorithm may be applied for correcting the intensity of the spectral images from the geometry/coupling effect as well as from the scattering amplitude distortions. The method can produce video rate absorption as well as scattering spectral images that can be further analyzed very rapidly, using matrix-based rapid inversion algorithms to produce more detailed quantitative images containing information relevant to tissue physiology and morphology.
85 citations
TL;DR: The state of the art of such acquisition systems and their applications for the study of cultural heritage are presented and existing registration techniques that can be used to obtain 3D models with multispectral texture are described.
75 citations
Cites background from "Multispectral Filter-Wheel Cameras:..."
...Various algorithms can be used to compensate geometric distortion [4, 5], longitudinal aberrations [8] and ghosting [9] introduced by the lter-wheel....
[...]
TL;DR: A new measurement method utilizing a random noise test target with markers is developed, which allows for direct comparison between prototype and image and presents comprehensive results for the PSF estimation using the authors' multispectral camera and provides deconvolution results.
Abstract: Conventional point spread function (PSF) measurement methods often use parametric models for the estimation
of the PSF. This limits the shape of the PSF to a specific form provided by the model. However, there are
unconventional imaging systems like multispectral cameras with optical bandpass filters, which produce an, e.g.,
unsymmetric PSF. To estimate such PSFs we have developed a new measurement method utilizing a random noise
test target with markers: After acquisition of this target, a synthetic prototype of the test target is geometrically
transformed to match the acquired image with respect to its geometric alignment. This allows us to estimate the
PSF by direct comparison between prototype and image. The noise target allows us to evaluate all frequencies
due to the approximately "white" spectrum of the test target - we are not limited to a specifically shaped PSF.
The registration of the prototype pattern gives us the opportunity to take the specific spectrum into account
and not just a "white" spectrum, which might be a weak assumption in small image regions. Based on the PSF
measurement, we perform a deconvolution. We present comprehensive results for the PSF estimation using our
multispectral camera and provide deconvolution results.
57 citations
TL;DR: In this article, a new measure called normalized total gradient is proposed for multispectral image registration, which is based on the assumption that the gradient of the difference between two aligned band images is sparser than that between two misaligned ones.
Abstract: Image registration is a fundamental issue in multispectral image processing, and is challenged by two main characteristics of multispectral images. First, the regional intensities can be essentially different between band images. Second, the local contrasts of two difference band images are inconsistent or even reversed. Conventional measures can align images with different regional intensity levels, but may fail in the circumstance of severe local intensity variation. In this paper, a new measure called normalized total gradient is proposed for multispectral image registration. The measure is based on the key assumption (observation) that the gradient of the difference between two aligned band images is sparser than that between two misaligned ones. A registration framework, which incorporates image pyramid and global/local optimization, is further introduced for affine transform. Experimental results validate that the proposed method is not only effective for multispectral image registration, but also applicable to general unimodal/multimodal image registration tasks. It performs better than or comparable to the existing methods, both quantitatively and qualitatively.
42 citations
References
More filters
TL;DR: A method is described for the minimization of a function of n variables, which depends on the comparison of function values at the (n 41) vertices of a general simplex, followed by the replacement of the vertex with the highest value by another point.
Abstract: A method is described for the minimization of a function of n variables, which depends on the comparison of function values at the (n 41) vertices of a general simplex, followed by the replacement of the vertex with the highest value by another point. The simplex adapts itself to the local landscape, and contracts on to the final minimum. The method is shown to be effective and computationally compact. A procedure is given for the estimation of the Hessian matrix in the neighbourhood of the minimum, needed in statistical estimation problems.
27,271 citations
TL;DR: New results are derived on the minimum number of landmarks needed to obtain a solution, and algorithms are presented for computing these minimum-landmark solutions in closed form that provide the basis for an automatic system that can solve the Location Determination Problem under difficult viewing.
Abstract: A new paradigm, Random Sample Consensus (RANSAC), for fitting a model to experimental data is introduced. RANSAC is capable of interpreting/smoothing data containing a significant percentage of gross errors, and is thus ideally suited for applications in automated image analysis where interpretation is based on the data provided by error-prone feature detectors. A major portion of this paper describes the application of RANSAC to the Location Determination Problem (LDP): Given an image depicting a set of landmarks with known locations, determine that point in space from which the image was obtained. In response to a RANSAC requirement, new results are derived on the minimum number of landmarks needed to obtain a solution, and algorithms are presented for computing these minimum-landmark solutions in closed form. These results provide the basis for an automatic system that can solve the LDP under difficult viewing
23,396 citations
TL;DR: The results demonstrate that subvoxel accuracy with respect to the stereotactic reference solution can be achieved completely automatically and without any prior segmentation, feature extraction, or other preprocessing steps which makes this method very well suited for clinical applications.
Abstract: A new approach to the problem of multimodality medical image registration is proposed, using a basic concept from information theory, mutual information (MI), or relative entropy, as a new matching criterion. The method presented in this paper applies MI to measure the statistical dependence or information redundancy between the image intensities of corresponding voxels in both images, which is assumed to be maximal if the images are geometrically aligned. Maximization of MI is a very general and powerful criterion, because no assumptions are made regarding the nature of this dependence and no limiting constraints are imposed on the image content of the modalities involved. The accuracy of the MI criterion is validated for rigid body registration of computed tomography (CT), magnetic resonance (MR), and photon emission tomography (PET) images by comparison with the stereotactic registration solution, while robustness is evaluated with respect to implementation issues, such as interpolation and optimization, and image content, including partial overlap and image degradation. Our results demonstrate that subvoxel accuracy with respect to the stereotactic reference solution can be achieved completely automatically and without any prior segmentation, feature extraction, or other preprocessing steps which makes this method very well suited for clinical applications.
4,773 citations
TL;DR: An overview is presented of the medical image processing literature on mutual-information-based registration, an introduction for those new to the field, an overview for those working in the field and a reference for those searching for literature on a specific application.
Abstract: An overview is presented of the medical image processing literature on mutual-information-based registration. The aim of the survey is threefold: an introduction for those new to the field, an overview for those working in the field, and a reference for those searching for literature on a specific application. Methods are classified according to the different aspects of mutual-information-based registration. The main division is in aspects of the methodology and of the application. The part on methodology describes choices made on facets such as preprocessing of images, gray value interpolation, optimization, adaptations to the mutual information measure, and different types of geometrical transformations. The part on applications is a reference of the literature available on different modalities, on interpatient registration and on different anatomical objects. Comparison studies including mutual information are also considered. The paper starts with a description of entropy and mutual information and it closes with a discussion on past achievements and some future challenges.
3,121 citations
"Multispectral Filter-Wheel Cameras:..." refers background in this paper
...Registration algorithms for medical images, therefore, often use mutual information (MI) as similarity criterion [36], [37], which addresses the mentioned problems....
[...]
TL;DR: It is shown that for downsampled images the signal power in the phase correlation is not concentrated in a single peak, but rather in several coherent peaks mostly adjacent to each other.
Abstract: In this paper, we have derived analytic expressions for the phase correlation of downsampled images. We have shown that for downsampled images the signal power in the phase correlation is not concentrated in a single peak, but rather in several coherent peaks mostly adjacent to each other. These coherent peaks correspond to the polyphase transform of a filtered unit impulse centered at the point of registration. The analytic results provide a closed-form solution to subpixel translation estimation, and are used for detailed error analysis. Excellent results have been obtained for subpixel translation estimation of images of different nature and across different spectral bands.
966 citations
"Multispectral Filter-Wheel Cameras:..." refers background in this paper
...After the ray is refracted, its angle with respect to the normal follows Snell’s law (4) where is the refraction index of air and the refraction index of the optical filter....
[...]