Dictionary learning technique and penalized maximum likelihood for extending measurement range of a Rayleigh lidar
30 Sep 2020-Journal of Applied Remote Sensing (SPIE-Intl Soc Optical Eng)-Vol. 14, Iss: 3, pp 034529
TL;DR: A method to retrieve atmospheric temperatures using the penalized maximum likelihood method after denoising the backscattered signal using the dictionary learning technique is presented and compared with the conventional method, which has the advantage of improving the measurement range and reducing the standard error in temperatures.
Abstract: Extending the temperature measurement range and sustaining the system performance are important requirements in Rayleigh lidars. While replacing hardware and increasing the power aperture product are ways of achieving these objectives, the data analysis approach is another means for achieving the same. A method to retrieve atmospheric temperatures using the penalized maximum likelihood method after denoising the backscattered signal using the dictionary learning technique is presented and compared with the conventional method. The proposed combination has the advantage of improving the measurement range and reducing the standard error (SE) in temperatures. The penalized maximum likelihood function is solved using the method of successive approximations, and the SE in temperature is calculated using Monte Carlo simulations. Observations from the Rayleigh lidar at the National Atmospheric Research Laboratory, India, are used for testing the approach. When compared with the conventional method, the SE in temperatures improved by 5K at 84 km, and the average height improvement was about 6 km.
Citations
More filters
19 Dec 2014
TL;DR: In this paper, the authors used an optimal estimation method (OEM) to estimate the temperature in the middle atmosphere with Rayleigh-scatter lidars, which allows a full uncertainty budget to be obtained on a per profile basis that includes, in addition to the statistical uncertainties, the smoothing error and uncertainties due to Rayleigh extinction, ozone absorption, lidar constant, nonlinearity in the counting system, variation of the Rayleigh scatter cross section with altitude, pressure, acceleration due to gravity, and the variation of mean molecular mass with altitude.
Abstract: The measurement of temperature in the middle atmosphere with Rayleigh-scatter lidars is an important technique for assessing atmospheric change. Current retrieval schemes for this temperature have several shortcomings, which can be overcome by using an optimal estimation method (OEM). Forward models are presented that completely characterize the measurement and allow the simultaneous retrieval of temperature, dead time, and background. The method allows a full uncertainty budget to be obtained on a per profile basis that includes, in addition to the statistical uncertainties, the smoothing error and uncertainties due to Rayleigh extinction, ozone absorption, lidar constant, nonlinearity in the counting system, variation of the Rayleigh-scatter cross section with altitude, pressure, acceleration due to gravity, and the variation of mean molecular mass with altitude. The vertical resolution of the temperature profile is found at each height, and a quantitative determination is made of the maximum height to which the retrieval is valid. A single temperature profile can be retrieved from measurements with multiple channels that cover different height ranges, vertical resolutions, and even different detection methods. The OEM employed is shown to give robust estimates of temperature, which are consistent with previous methods, while requiring minimal computational time. This demonstrated success of lidar temperature retrievals using an OEM opens new possibilities in atmospheric science for measurement integration between active and passive remote sensing instruments.
36 citations
TL;DR: In this paper , a wavelet transform and locally weighted scatterplot smoothing (LOWESS) based on ensemble empirical mode decomposition (EEMD) for Rayleigh lidar signal denoising was proposed.
Abstract: Lidar is an active remote sensing technology that has many advantages, but the echo lidar signal is extremely susceptible to noise and complex atmospheric environment, which affects the effective detection range and retrieval accuracy. In this paper, a wavelet transform (WT) and locally weighted scatterplot smoothing (LOWESS) based on ensemble empirical mode decomposition (EEMD) for Rayleigh lidar signal denoising was proposed. The WT method was used to remove the noise in the signal with a signal-to-noise ratio (SNR) higher than 16 dB. The EEMD method was applied to decompose the remaining signal into a series of intrinsic modal functions (IMFs), and then detrended fluctuation analysis (DFA) was conducted to determine the threshold for distinguishing whether noise or signal was the main component of the IMFs. Moreover, the LOWESS method was adopted to remove the noise in the IMFs component containing the signal, and thus, finely extract the signal. The simulation results showed that the denoising effect of the proposed WT-EEMD-LOWESS method was superior to EEMD-WT, EEMD-SVD and VMD-WOA. Finally, the use of WT-EEMD-LOWESS on the measured lidar signal led to significant improvement in retrieval accuracy. The maximum error of density and temperature retrievals was decreased from 1.36% and 125.79 K to 1.1% and 13.84 K, respectively.
3 citations
TL;DR: In this article , the authors proposed a dictionary learning based approach to discriminate the effect of Doppler shift due to background horizontal wind in Gravity Wave Signatures from temperature perturbations.
1 citations
01 May 2020
TL;DR: In this paper, the authors compare ground and ship-based lidar measurements of atmospheric temperature, ozone, and wind to similar measurements made from orbiting satellites, and demonstrate how comparing lidar and satellite measurements of events such as small-scale fast moving gravity waves over a particular geographic region can be affected by instrument averaging kernels.
Abstract:
The comparison of ground and ship-based lidar measurements of atmospheric temperature, ozone, and wind to similar measurements made from orbiting satellites is a unique challenge. In this talk we will discuss general challenges associated with (i) determining coincidence by compensating for geographic and temporal offsets, (ii) satellite-lidar sampling errors, and (iii) comparing results made by different techniques.
We will show that comparisons of absolute temperature improve when the ground based measurements are compared to a composite satellite profile, created by a weighted average of multiple profiles from one overpass, instead of comparing to the single satellite profile from the closest approach.
We discuss the importance of including the variation between consecutive satellite profiles for a given overpass in addition to the given satellite instrument uncertainty when calculating the error budget of the comparisons, even when comparing to single satellite profiles.
We demonstrate how comparing lidar and satellite measurements of events such as small-scale fast moving gravity waves over a particular geographic region can be affected by instrument averaging kernels.
Illustrative examples we will be showing include lidar measurements made during recent instrument validation campaigns at L’Observatoire de Haute Provence (OHP, 43.93 N, 5.71 E), La Réunion (21.17 S, 55.37 E), Hohenpeißenberg Meteorological Observatory (47.80 N, 11.00 E), and onboard the French Navy Research Ship Monge as well as satellite measurements from the Microwave Limb Sounder (MLS), the Sounding of the Atmosphere by Broadband Emission Radiometry instrument (SABER), Global Ozone Monitoring by Occultation of Stars (GOMOS), and Atmospheric Dynamics Mission Aeolus (Aeolus).
1 citations
TL;DR: A human activity recognition algorithm is proposed for wireless sensor networks that uses the singularity of intraclass divergence matrix and linear indivisibility of real samples encountered in metric learning to reduce the impact of small sample problem.
Abstract: Wireless sensor network is an ad hoc network with sensing capability. Usually, a large number of sensor nodes are randomly deployed in an unreachable environment or complex area for data collection and transmission, which can realize the perception and monitoring of the target area or specific objects and transmit the obtained data to the remote end of the system. Human health activity recognition algorithm is a hot topic in the field of computer. Based on the small sample problem and the linear indivisibility of real samples encountered in metric learning, this paper proposes a human activity recognition algorithm for wireless sensor networks. Human activity recognition algorithm for wireless sensor networks uses human activity recognition algorithm to solve the singularity of intraclass divergence matrix, so as to reduce the impact of small sample problem. The algorithm maps two different feature spaces to the high-dimensional linearly separable kernel space through the corresponding kernel function, calculates the distance between samples in the two projected feature subspaces to obtain two distance measurement functions, and finally linearly combines them with weights to obtain the final distance measurement function.
1 citations
References
More filters
Book•
01 Jan 1998
TL;DR: An introduction to a Transient World and an Approximation Tour of Wavelet Packet and Local Cosine Bases.
Abstract: Introduction to a Transient World. Fourier Kingdom. Discrete Revolution. Time Meets Frequency. Frames. Wavelet Zoom. Wavelet Bases. Wavelet Packet and Local Cosine Bases. An Approximation Tour. Estimations are Approximations. Transform Coding. Appendix A: Mathematical Complements. Appendix B: Software Toolboxes.
17,693 citations
TL;DR: A novel algorithm for adapting dictionaries in order to achieve sparse signal representations, the K-SVD algorithm, an iterative method that alternates between sparse coding of the examples based on the current dictionary and a process of updating the dictionary atoms to better fit the data.
Abstract: In recent years there has been a growing interest in the study of sparse representation of signals. Using an overcomplete dictionary that contains prototype signal-atoms, signals are described by sparse linear combinations of these atoms. Applications that use sparse representation are many and include compression, regularization in inverse problems, feature extraction, and more. Recent activity in this field has concentrated mainly on the study of pursuit algorithms that decompose signals with respect to a given dictionary. Designing dictionaries to better fit the above model can be done by either selecting one from a prespecified set of linear transforms or adapting the dictionary to a set of training signals. Both of these techniques have been considered, but this topic is largely still open. In this paper we propose a novel algorithm for adapting dictionaries in order to achieve sparse signal representations. Given a set of training signals, we seek the dictionary that leads to the best representation for each member in this set, under strict sparsity constraints. We present a new method-the K-SVD algorithm-generalizing the K-means clustering process. K-SVD is an iterative method that alternates between sparse coding of the examples based on the current dictionary and a process of updating the dictionary atoms to better fit the data. The update of the dictionary columns is combined with an update of the sparse representations, thereby accelerating convergence. The K-SVD algorithm is flexible and can work with any pursuit method (e.g., basis pursuit, FOCUSS, or matching pursuit). We analyze this algorithm and demonstrate its results both on synthetic tests and in applications on real image data
8,905 citations
TL;DR: This work proposes an entirely non-recursive variational mode decomposition model, where the modes are extracted concurrently and is a generalization of the classic Wiener filter into multiple, adaptive bands.
Abstract: During the late 1990s, Huang introduced the algorithm called Empirical Mode Decomposition, which is widely used today to recursively decompose a signal into different modes of unknown but separate spectral bands. EMD is known for limitations like sensitivity to noise and sampling. These limitations could only partially be addressed by more mathematical attempts to this decomposition problem, like synchrosqueezing, empirical wavelets or recursive variational decomposition. Here, we propose an entirely non-recursive variational mode decomposition model, where the modes are extracted concurrently. The model looks for an ensemble of modes and their respective center frequencies, such that the modes collectively reproduce the input signal, while each being smooth after demodulation into baseband. In Fourier domain, this corresponds to a narrow-band prior. We show important relations to Wiener filter denoising. Indeed, the proposed method is a generalization of the classic Wiener filter into multiple, adaptive bands. Our model provides a solution to the decomposition problem that is theoretically well founded and still easy to understand. The variational model is efficiently optimized using an alternating direction method of multipliers approach. Preliminary results show attractive performance with respect to existing mode decomposition models. In particular, our proposed model is much more robust to sampling and noise. Finally, we show promising practical decomposition results on a series of artificial and real data.
4,111 citations
Book•
01 Jan 2008
TL;DR: The central concept of sparsity is explained and applied to signal compression, noise reduction, and inverse problems, while coverage is given to sparse representations in redundant dictionaries, super-resolution and compressive sensing applications.
Abstract: Mallat's book is the undisputed reference in this field - it is the only one that covers the essential material in such breadth and depth. - Laurent Demanet, Stanford University The new edition of this classic book gives all the major concepts, techniques and applications of sparse representation, reflecting the key role the subject plays in today's signal processing. The book clearly presents the standard representations with Fourier, wavelet and time-frequency transforms, and the construction of orthogonal bases with fast algorithms. The central concept of sparsity is explained and applied to signal compression, noise reduction, and inverse problems, while coverage is given to sparse representations in redundant dictionaries, super-resolution and compressive sensing applications. Features: * Balances presentation of the mathematics with applications to signal processing * Algorithms and numerical examples are implemented in WaveLab, a MATLAB toolbox * Companion website for instructors and selected solutions and code available for students New in this edition * Sparse signal representations in dictionaries * Compressive sensing, super-resolution and source separation * Geometric image processing with curvelets and bandlets * Wavelets for computer graphics with lifting on surfaces * Time-frequency audio processing and denoising * Image compression with JPEG-2000 * New and updated exercises A Wavelet Tour of Signal Processing: The Sparse Way, third edition, is an invaluable resource for researchers and R&D engineers wishing to apply the theory in fields such as image processing, video processing and compression, bio-sensing, medical imaging, machine vision and communications engineering. Stephane Mallat is Professor in Applied Mathematics at cole Polytechnique, Paris, France. From 1986 to 1996 he was a Professor at the Courant Institute of Mathematical Sciences at New York University, and between 2001 and 2007, he co-founded and became CEO of an image processing semiconductor company. Companion website: A Numerical Tour of Signal Processing * Includes all the latest developments since the book was published in 1999, including its application to JPEG 2000 and MPEG-4 * Algorithms and numerical examples are implemented in Wavelab, a MATLAB toolbox * Balances presentation of the mathematics with applications to signal processing
2,600 citations
TL;DR: In der Theorie der Reihenentwicklung der reellen Funktionen spielen die sog. orthogonalen Funktionensysteme eine fuhrende Rolle.
Abstract: In der Theorie der Reihenentwicklung der reellen Funktionen spielen die sog. orthogonalen Funktionensysteme eine fuhrende Rolle. Man versteht darunter ein System von unendlichvielen Funktionen $\phi_1 (s), \phi_2 (s),\ldots$, die in bezug auf die beliebige mesbare Punktmenge $M$ die Orthogonalitatseigenschaft $\int_{(M)}\phi_p(s)\phi_q(s)ds=0$ ($p
eq q, p, q=1,2,\ldots$), $\int_{(M)}(\phi_p(s))^2ds=1$ ($p=1,2,\ldots$) besitzen, wobei die Integrale im Lebesgueschen Sinne genommen sind. acces pdf
1,877 citations