scispace - formally typeset
Search or ask a question
Author

Lidan Miao

Other affiliations: Microsoft
Bio: Lidan Miao is an academic researcher from University of Tennessee. The author has contributed to research in topics: Multispectral image & Demosaicing. The author has an hindex of 10, co-authored 19 publications receiving 1145 citations. Previous affiliations of Lidan Miao include Microsoft.

Papers
More filters
Journal ArticleDOI
TL;DR: A novel method without the pure-pixel assumption is presented, referred to as the minimum volume constrained nonnegative matrix factorization (MVC-NMF), for unsupervised endmember extraction from highly mixed image data, which outperforms several other advanced endmember detection approaches.
Abstract: Endmember extraction is a process to identify the hidden pure source signals from the mixture. In the past decade, numerous algorithms have been proposed to perform this estimation. One commonly used assumption is the presence of pure pixels in the given image scene, which are detected to serve as endmembers. When such pixels are absent, the image is referred to as the highly mixed data, for which these algorithms at best can only return certain data points that are close to the real endmembers. To overcome this problem, we present a novel method without the pure-pixel assumption, referred to as the minimum volume constrained nonnegative matrix factorization (MVC-NMF), for unsupervised endmember extraction from highly mixed image data. Two important facts are exploited: First, the spectral data are nonnegative; second, the simplex volume determined by the endmembers is the minimum among all possible simplexes that circumscribe the data scatter space. The proposed method takes advantage of the fast convergence of NMF schemes, and at the same time eliminates the pure-pixel assumption. The experimental results based on a set of synthetic mixtures and a real image scene demonstrate that the proposed method outperforms several other advanced endmember detection approaches

870 citations

Journal ArticleDOI
TL;DR: Experimental results support that MSFA technique can be applied to multispectral imaging with unique advantages, and a binary tree based generic demosaicking method is presented.
Abstract: In this paper, we extend the idea of using mosaicked color filter array (CFA) in color imaging, which has been widely adopted in the digital color camera industry, to the use of multispectral filter array (MSFA) in multispectral imaging. The filter array technique can help reduce the cost, achieve exact registration, and improve the robustness of the imaging system. However, the extension from CFA to MSFA is not straightforward. First, most CFAs only deal with a few bands (3 or 4) within the narrow visual spectral region, while the design of MSFA needs to handle the arrangement of multiple bands (more than 3) across a much wider spectral range. Second, most existing CFA demosaicking algorithms assume the fixed Bayer CFA and are confined to properties only existed in the color domain. Therefore, they cannot be directly applied to multispectral demosaicking. The main challenges faced in multispectral demosaicking is how to design a generic algorithm that can handle the more diversified MSFA patterns, and how to improve performance with a coarser spatial resolution and a less degree of spectral correlation. In this paper, we present a binary tree based generic demosaicking method. Two metrics are used to evaluate the generic algorithm, including the root mean-square error (RMSE) for reconstruction performance and the classification accuracy for target discrimination performance. Experimental results show that the demosaicked images present low RMSE (less than 7) and comparable classification performance as original images. These results support that MSFA technique can be applied to multispectral imaging with unique advantages

116 citations

Journal ArticleDOI
TL;DR: This paper addresses the importance of the maximum entropy principle for mixed-pixel decomposition from a geometric point of view and demonstrates that when the given data present strong noise or when the endmember signatures are close to each other, the proposed method has the potential of providing more accurate estimates than the popular least-squares methods.
Abstract: Due to the wide existence of mixed pixels, the derivation of constituent components (endmembers) and their fractional proportions (abundances) at the subpixel scale has been given a lot of attention. The entire process is often referred to as mixed-pixel decomposition or spectral unmixing. Although various algorithms have been proposed to solve this problem, two potential issues still need to be further investigated. First, assuming the endmembers are known, the abundance estimation is commonly performed by employing a least-squares error criterion, which, however, makes the estimation sensitive to noise and outliers. Second, the mathematical intractability of the abundance non-negative constraint results in computationally expensive numerical approaches. In this paper, we propose an unsupervised decomposition method based on the classic maximum entropy principle, termed the gradient descent maximum entropy (GDME), aiming at robust and effective estimates. We address the importance of the maximum entropy principle for mixed-pixel decomposition from a geometric point of view and demonstrate that when the given data present strong noise or when the endmember signatures are close to each other, the proposed method has the potential of providing more accurate estimates than the popular least-squares methods (e.g., fully constrained least squares). We apply the proposed GDME to the subject of unmixing multispectral and hyperspectral data. The experimental results obtained from both simulated and real images show the effectiveness of the proposed method

82 citations

Journal ArticleDOI
TL;DR: It is demonstrated that most of the CFAs currently used by the industry can be derived as special cases of MSFAs generated using the generic algorithm, and two metrics, static coefficient and consistency coefficient, are designed to measure these two parameters.
Abstract: The technology of color filter arrays (CFA) has been widely used in the digital camera industry since it provides several advantages like low cost, exact registration, and strong robustness. The same motivations also drive the design of multispectral filter arrays (MSFA), in which more than three spectral bands are used. Although considerable research has been reported to optimally reconstruct the full-color image using various demosaicking algorithms, studies on the intrinsic properties of these filter arrays as well as the underlying design principles have been very limited. Given a set of representative spectral bands, the design of an MSFA involves two issues: the selection of tessellation mechanisms and the arrangement/layout of different spectral bands. We develop a generic MSFA generation method starting from a checkerboard pattern. We show, through case studies, that most of the CFAs currently used by the industry can be derived as special cases of MSFAs generated using the generic algorithm. The performance of different MSFAs are evaluated based on their intrinsic properties, namely, the spatial uniformity and the spectral consistency. We design two metrics, static coefficient and consistency coefficient, to measure these two parameters, respectively. The experimental results demonstrate that the generic algorithm can generate optimal or near-optimal MSFAs in both the rectangular and the hexagonal domains

78 citations

Proceedings ArticleDOI
24 Oct 2004
TL;DR: This paper develops a generic MSFA generation method starting from a checkerboard pattern with both rectangular and hexagonal tessellations, and designs a metric, referred as the static coefficient (SC), to measure the uniformity of MSFAs.
Abstract: The technology of color filter arrays (CFA) has been widely used in the digital camera industry since it provides several advantages like low cost, exact registration, and strong robustness. The same motivations also drive the design of multi-spectral filter arrays (MSFA), in which more than three color bands are used (e.g. visible and infrared). Although considerable research has been reported to optimally reconstruct the full-color image using various interpolation algorithms, studies on the intrinsic properties of these filter arrays as well as the underlying design principles have been very limited. In this paper, we identify the properties a CFA should possess and extend the design philosophy to MSFA. Based on these discussions, we develop a generic MSFA generation method starting from a checkerboard pattern with both rectangular and hexagonal tessellations. By manipulating this pattern through a combination of decomposition and subsampling steps, we can generate MSFAs that satisfy all the design requirements. We show, through case studies, that most of the CFAs currently used by the industry can be derived as special cases. To evaluate the performance of MSFAs, we design a metric, referred as the static coefficient (SC), to measure the uniformity of MSFAs.

28 citations


Cited by
More filters
Journal ArticleDOI
TL;DR: This paper presents an overview of un Mixing methods from the time of Keshava and Mustard's unmixing tutorial to the present, including Signal-subspace, geometrical, statistical, sparsity-based, and spatial-contextual unmixed algorithms.
Abstract: Imaging spectrometers measure electromagnetic energy scattered in their instantaneous field view in hundreds or thousands of spectral channels with higher spectral resolution than multispectral cameras. Imaging spectrometers are therefore often referred to as hyperspectral cameras (HSCs). Higher spectral resolution enables material identification via spectroscopic analysis, which facilitates countless applications that require identifying materials in scenarios unsuitable for classical spectroscopic analysis. Due to low spatial resolution of HSCs, microscopic material mixing, and multiple scattering, spectra measured by HSCs are mixtures of spectra of materials in a scene. Thus, accurate estimation requires unmixing. Pixels are assumed to be mixtures of a few materials, called endmembers. Unmixing involves estimating all or some of: the number of endmembers, their spectral signatures, and their abundances at each pixel. Unmixing is a challenging, ill-posed inverse problem because of model inaccuracies, observation noise, environmental conditions, endmember variability, and data set size. Researchers have devised and investigated many models searching for robust, stable, tractable, and accurate unmixing algorithms. This paper presents an overview of unmixing methods from the time of Keshava and Mustard's unmixing tutorial to the present. Mixing models are first discussed. Signal-subspace, geometrical, statistical, sparsity-based, and spatial-contextual unmixing algorithms are described. Mathematical problems and potential solutions are described. Algorithm characteristics are illustrated experimentally.

2,373 citations

Posted Content
TL;DR: An overview of unmixing methods from the time of Keshava and Mustard's tutorial as mentioned in this paper to the present can be found in Section 2.2.1].
Abstract: Imaging spectrometers measure electromagnetic energy scattered in their instantaneous field view in hundreds or thousands of spectral channels with higher spectral resolution than multispectral cameras. Imaging spectrometers are therefore often referred to as hyperspectral cameras (HSCs). Higher spectral resolution enables material identification via spectroscopic analysis, which facilitates countless applications that require identifying materials in scenarios unsuitable for classical spectroscopic analysis. Due to low spatial resolution of HSCs, microscopic material mixing, and multiple scattering, spectra measured by HSCs are mixtures of spectra of materials in a scene. Thus, accurate estimation requires unmixing. Pixels are assumed to be mixtures of a few materials, called endmembers. Unmixing involves estimating all or some of: the number of endmembers, their spectral signatures, and their abundances at each pixel. Unmixing is a challenging, ill-posed inverse problem because of model inaccuracies, observation noise, environmental conditions, endmember variability, and data set size. Researchers have devised and investigated many models searching for robust, stable, tractable, and accurate unmixing algorithms. This paper presents an overview of unmixing methods from the time of Keshava and Mustard's unmixing tutorial [1] to the present. Mixing models are first discussed. Signal-subspace, geometrical, statistical, sparsity-based, and spatial-contextual unmixing algorithms are described. Mathematical problems and potential solutions are described. Algorithm characteristics are illustrated experimentally.

1,808 citations

Journal ArticleDOI
TL;DR: A tutorial/overview cross section of some relevant hyperspectral data analysis methods and algorithms, organized in six main topics: data fusion, unmixing, classification, target detection, physical parameter retrieval, and fast computing.
Abstract: Hyperspectral remote sensing technology has advanced significantly in the past two decades. Current sensors onboard airborne and spaceborne platforms cover large areas of the Earth surface with unprecedented spectral, spatial, and temporal resolutions. These characteristics enable a myriad of applications requiring fine identification of materials or estimation of physical parameters. Very often, these applications rely on sophisticated and complex data analysis methods. The sources of difficulties are, namely, the high dimensionality and size of the hyperspectral data, the spectral mixing (linear and nonlinear), and the degradation mechanisms associated to the measurement process such as noise and atmospheric effects. This paper presents a tutorial/overview cross section of some relevant hyperspectral data analysis methods and algorithms, organized in six main topics: data fusion, unmixing, classification, target detection, physical parameter retrieval, and fast computing. In all topics, we describe the state-of-the-art, provide illustrative examples, and point to future challenges and research directions.

1,604 citations

Journal ArticleDOI
TL;DR: The experimental results, conducted using both simulated and real hyperspectral data sets collected by the NASA Jet Propulsion Laboratory's Airborne Visible Infrared Imaging Spectrometer and spectral libraries publicly available from the U.S. Geological Survey, indicate the potential of SR techniques in the task of accurately characterizing the mixed pixels using the library spectra.
Abstract: Linear spectral unmixing is a popular tool in remotely sensed hyperspectral data interpretation. It aims at estimating the fractional abundances of pure spectral signatures (also called as endmembers) in each mixed pixel collected by an imaging spectrometer. In many situations, the identification of the end-member signatures in the original data set may be challenging due to insufficient spatial resolution, mixtures happening at different scales, and unavailability of completely pure spectral signatures in the scene. However, the unmixing problem can also be approached in semisupervised fashion, i.e., by assuming that the observed image signatures can be expressed in the form of linear combinations of a number of pure spectral signatures known in advance (e.g., spectra collected on the ground by a field spectroradiometer). Unmixing then amounts to finding the optimal subset of signatures in a (potentially very large) spectral library that can best model each mixed pixel in the scene. In practice, this is a combinatorial problem which calls for efficient linear sparse regression (SR) techniques based on sparsity-inducing regularizers, since the number of endmembers participating in a mixed pixel is usually very small compared with the (ever-growing) dimensionality (and availability) of spectral libraries. Linear SR is an area of very active research, with strong links to compressed sensing, basis pursuit (BP), BP denoising, and matching pursuit. In this paper, we study the linear spectral unmixing problem under the light of recent theoretical results published in those referred to areas. Furthermore, we provide a comparison of several available and new linear SR algorithms, with the ultimate goal of analyzing their potential in solving the spectral unmixing problem by resorting to available spectral libraries. Our experimental results, conducted using both simulated and real hyperspectral data sets collected by the NASA Jet Propulsion Laboratory's Airborne Visible Infrared Imaging Spectrometer and spectral libraries publicly available from the U.S. Geological Survey, indicate the potential of SR techniques in the task of accurately characterizing the mixed pixels using the library spectra. This opens new perspectives for spectral unmixing, since the abundance estimation process no longer depends on the availability of pure spectral signatures in the input data nor on the capacity of a certain endmember extraction algorithm to identify such pure signatures.

956 citations

Journal ArticleDOI
TL;DR: Simulations with various image data sets demonstrate that the CNMF algorithm can produce high-quality fused data both in terms of spatial and spectral domains, which contributes to the accurate identification and classification of materials observed at a high spatial resolution.
Abstract: Coupled nonnegative matrix factorization (CNMF) unmixing is proposed for the fusion of low-spatial-resolution hyperspectral and high-spatial-resolution multispectral data to produce fused data with high spatial and spectral resolutions. Both hyperspectral and multispectral data are alternately unmixed into end member and abundance matrices by the CNMF algorithm based on a linear spectral mixture model. Sensor observation models that relate the two data are built into the initialization matrix of each NMF unmixing procedure. This algorithm is physically straightforward and easy to implement owing to its simple update rules. Simulations with various image data sets demonstrate that the CNMF algorithm can produce high-quality fused data both in terms of spatial and spectral domains, which contributes to the accurate identification and classification of materials observed at a high spatial resolution.

847 citations