Showing papers on "Kernel adaptive filter published in 2017"

Learning Spatial-Aware Regressions for Visual Tracking

Abstract: In this paper, we analyze the spatial information of deep features, and propose two complementary regressions for robust visual tracking. First, we propose a kernelized ridge regression model wherein the kernel value is defined as the weighted sum of similarity scores of all pairs of patches between two samples. We show that this model can be formulated as a neural network and thus can be efficiently solved. Second, we propose a fully convolutional neural network with spatially regularized kernels, through which the filter kernel corresponding to each output channel is forced to focus on a specific region of the target. Distance transform pooling is further exploited to determine the effectiveness of each output channel of the convolution layer. The outputs from the kernelized ridge regression model and the fully convolutional neural network are combined to obtain the ultimate response. Experimental results on two benchmark datasets validate the effectiveness of the proposed method.

Kernel Risk-Sensitive Loss: Definition, Properties and Application to Robust Adaptive Filtering

Abstract: Nonlinear similarity measures defined in kernel space, such as correntropy, can extract higher order statistics of data and offer potentially significant performance improvement over their linear counterparts especially in non Gaussian signal processing and machine learning. In this paper, we propose a new similarity measure in kernel space, called the kernel risk-sensitive loss (KRSL), and provide some important properties. We apply the KRSL to adaptive filtering and investigate the robustness, and then develop the MKRSL algorithm and analyze the mean square convergence performance. Compared with correntropy, the KRSL can offer a more efficient performance surface, thereby enabling a gradient-based method to achieve faster convergence speed and higher accuracy while still maintaining the robustness to outliers. Theoretical analysis results and superior performance of the new algorithm are confirmed by simulation.

A Multisensor Multi-Bernoulli Filter

Abstract: In this paper, we derive a multisensor multi-Bernoulli (MS-MeMBer) filter for multitarget tracking. Measurements from multiple sensors are employed by the proposed filter to update a set of tracks modeled as a multi-Bernoulli random finite set. An exact implementation of the MS-MeMBer update procedure is computationally intractable. We propose an efficient approximate implementation by using a greedy measurement partitioning mechanism. The proposed filter allows for Gaussian mixture or particle filter implementations. Numerical simulations conducted for both linear-Gaussian and nonlinear models highlight the improved accuracy of the MS-MeMBer filter and its reduced computational load with respect to the multisensor cardinalized probability hypothesis density filter and the iterated-corrector cardinality-balanced multi-Bernoulli filter especially for low probabilities of detection.

Performance analysis of improved iterated cubature Kalman filter and its application to GNSS/INS.

Abstract: In order to improve the accuracy and robustness of GNSS/INS navigation system, an improved iterated cubature Kalman filter (IICKF) is proposed by considering the state-dependent noise and system uncertainty. First, a simplified framework of iterated Gaussian filter is derived by using damped Newton–Raphson algorithm and online noise estimator. Then the effect of state-dependent noise coming from iterated update is analyzed theoretically, and an augmented form of CKF algorithm is applied to improve the estimation accuracy. The performance of IICKF is verified by field test and numerical simulation, and results reveal that, compared with non-iterated filter, iterated filter is less sensitive to the system uncertainty, and IICKF improves the accuracy of yaw, roll and pitch by 48.9%, 73.1% and 83.3%, respectively, compared with traditional iterated KF.

Robust Student’s t-Based Stochastic Cubature Filter for Nonlinear Systems With Heavy-Tailed Process and Measurement Noises

Abstract: In this paper, a new robust Student’s t-based stochastic cubature filter (RSTSCF) is proposed for a nonlinear state–space model with heavy-tailed process and measurement noises. The heart of the RSTSCF is a stochastic Student’s t-spherical radial cubature rule (SSTSRCR), which is derived based on the third-degree unbiased spherical rule and the proposed third-degree unbiased radial rule. The existing stochastic integration rule is a special case of the proposed SSTSRCR when the degrees of freedom parameter tends to infinity. The proposed filter is applied to a maneuvering bearings-only tracking example, in which an agile target is tracked and the bearing is observed in clutter. Simulation results show that the proposed RSTSCF can achieve higher estimation accuracy than the existing Gaussian approximate filter, Gaussian sum filter, Huber-based nonlinear Kalman filter, maximum correntropy criterion-based Kalman filter, and robust Student’s t-based nonlinear filters, and is computationally much more efficient than the existing particle filter.

Fast Bilateral Filtering for Denoising Large 3D Images

Abstract: A fast implementation of bilateral filtering is presented, which is based on an optimal expansion of the filter kernel into a sum of factorized terms. These terms are computed by minimizing the expansion error in the mean-square-error sense. This leads to a simple and elegant solution in terms of eigenvectors of a square matrix. In this way, the bilateral filter is applied through computing a few Gaussian convolutions, for which very efficient algorithms are readily available. Moreover, the expansion functions are optimized for the histogram of the input image, leading to improved accuracy. It is shown that this further optimization it made possible by removing the commonly deployed constrain of shiftability of the basis functions. Experimental validation is carried out in the context of digital rock imaging. Results on large 3D images of rock samples show the superiority of the proposed method with respect to other fast approximations of bilateral filtering.

Transfer Learning in Adaptive Filters: The Nearest Instance Centroid-Estimation Kernel Least-Mean-Square Algorithm

Abstract: We propose a novel nearest-neighbors approach to organize and curb the growth of radial basis function network in kernel adaptive filtering (KAF). The nearest-instance-centroid-estimation (NICE) kernel least-mean-square (KLMS) algorithm provides an appropriate time-space tradeoff with good performance. Its centers in the input/feature space are organized by quasi-orthogonal regions for greatly simplified filter evaluation. Instead of using all centers to evaluate/update the function approximation at every new point, a linear search among the iteratively-updated centroids determines the partial function to be used, naturally forming locally-supported partial functionals. Under this framework, partial functionals that compose the adaptive filter are quickly stored/retrieved based on input, each corresponding to a specialized “spatial-band” subfilter. The filter evaluation becomes the update of one of the subfilters, creating a content addressable filter bank (CAFB). This CAFB is incrementally updated for new signal applications with mild constraints, always using the past-learned partial filter sums, opening the door for transfer learning and significant efficiency for new data scenarios, avoiding training from scratch as have been done since the invention of adaptive filtering. Using energy conservation relation, we show the sufficient condition for mean square convergence of the NICE-KLMS algorithm and establish the upper and lower bounds of steady-state excess-mean-square-error (EMSE). Simulations on chaotic time-series prediction demonstrate similar levels of accuracy as existing methods, but with much faster computation involving fewer input samples. Simulations on transfer learning using both synthetic and real-world data demonstrate that NICE CAFB can leverage previously learned knowledge to related task or domain.

An adaptive dynamically weighted median filter for impulse noise removal

Abstract: A new impulsive noise removal filter, adaptive dynamically weighted median filter (ADWMF), is proposed. A popular method for removing impulsive noise is a median filter whereas the weighted median filter and center weighted median filter were also investigated. ADWMF is based on weighted median filter. In ADWMF, instead of fixed weights, weightages of the filter are dynamically assigned with the results of noise detection. A simple and efficient noise detection method is also used to detect noise candidates and dynamically assign zero or small weights to the noise candidates in the window. This paper proposes an adaptive method which increases the window size according to the amounts of impulsive noise. Simulation results show that the AMWMF works better for both images with low and high density of impulsive noise than existing methods work.

Optimizing Filter Size in Convolutional Neural Networks for Facial Action Unit Recognition

Abstract: Recognizing facial action units (AUs) during spontaneous facial displays is a challenging problem. Most recently, Convolutional Neural Networks (CNNs) have shown promise for facial AU recognition, where predefined and fixed convolution filter sizes are employed. In order to achieve the best performance, the optimal filter size is often empirically found by conducting extensive experimental validation. Such a training process suffers from expensive training cost, especially as the network becomes deeper. This paper proposes a novel Optimized Filter Size CNN (OFS-CNN), where the filter sizes and weights of all convolutional layers are learned simultaneously from the training data along with learning convolution filters. Specifically, the filter size is defined as a continuous variable, which is optimized by minimizing the training loss. Experimental results on two AU-coded spontaneous databases have shown that the proposed OFS-CNN is capable of estimating optimal filter size for varying image resolution and outperforms traditional CNNs with the best filter size obtained by exhaustive search. The OFS-CNN also beats the CNN using multiple filter sizes and more importantly, is much more efficient during testing with the proposed forward-backward propagation algorithm.

A unified method to energy-to-peak filter design for networked Markov switched singular systems over a finite-time interval

Abstract: For a class of networked singular Markov switched discrete-time systems, this work investigates the finite-time energy-to-peak filtering problem. The system modes information is transmitted though an unreliable communication link in the systems under consideration, where the packet dropout phenomenon, modes information available to the filter and asynchronous phenomenon between filter modes and system modes are randomly occurring with a certain probability and described by some Bernoulli distributed white sequence variables. The objective is focused on designing a unified filter, which covers mode-independent filter, asynchronous filter and mode-dependent filter, so that the estimation error system is finite-time bounded while meets a fixed energy-to-peak performance requirement in the presence of those stochastic phenomena. By employing a probability-dependent Lyapunov–Krasovskii function, some sufficient criteria are established to make sure that there is a feasible solution to the addressed problem. Moreover, with the help of a novel simple matrix decoupling approach, the filter gains are obtained. In the end, we employ a numerical example with simulation to show the serviceability of the presented method.

The Kernel Mixture Network: A Nonparametric Method for Conditional Density Estimation of Continuous Random Variables

Abstract: This paper introduces the kernel mixture network, a new method for nonparametric estimation of conditional probability densities using neural networks. We model arbitrarily complex conditional densities as linear combinations of a family of kernel functions centered at a subset of training points. The weights are determined by the outer layer of a deep neural network, trained by minimizing the negative log likelihood. This generalizes the popular quantized softmax approach, which can be seen as a kernel mixture network with square and non-overlapping kernels. We test the performance of our method on two important applications, namely Bayesian filtering and generative modeling. In the Bayesian filtering example, we show that the method can be used to filter complex nonlinear and non-Gaussian signals defined on manifolds. The resulting kernel mixture network filter outperforms both the quantized softmax filter and the extended Kalman filter in terms of model likelihood. Finally, our experiments on generative models show that, given the same architecture, the kernel mixture network leads to higher test set likelihood, less overfitting and more diversified and realistic generated samples than the quantized softmax approach.

A Quantized Kernel Learning Algorithm Using a Minimum Kernel Risk-Sensitive Loss Criterion and Bilateral Gradient Technique

Abstract: Recently, inspired by correntropy, kernel risk-sensitive loss (KRSL) has emerged as a novel nonlinear similarity measure defined in kernel space, which achieves a better computing performance. After applying the KRSL to adaptive filtering, the corresponding minimum kernel risk-sensitive loss (MKRSL) algorithm has been developed accordingly. However, MKRSL as a traditional kernel adaptive filter (KAF) method, generates a growing radial basis functional (RBF) network. In response to that limitation, through the use of online vector quantization (VQ) technique, this article proposes a novel KAF algorithm, named quantized MKRSL (QMKRSL) to curb the growth of the RBF network structure. Compared with other quantized methods, e.g., quantized kernel least mean square (QKLMS) and quantized kernel maximum correntropy (QKMC), the efficient performance surface makes QMKRSL converge faster and filter more accurately, while maintaining the robustness to outliers. Moreover, considering that QMKRSL using traditional gradient descent method may fail to make full use of the hidden information between the input and output spaces, we also propose an intensified QMKRSL using a bilateral gradient technique named QMKRSL_BG, in an effort to further improve filtering accuracy. Short-term chaotic time-series prediction experiments are conducted to demonstrate the satisfactory performance of our algorithms.

Time series prediction using kernel adaptive filter with least mean absolute third loss function

Abstract: In this paper, a novel kernel adaptive filter, based on the least mean absolute third (LMAT) loss function, is proposed for time series prediction in various noise environments. Combining the benefits of the kernel method and the LMAT loss function, the proposed KLMAT algorithm performs robustly against noises with different probability densities. However, an important limitation of the KLMAT algorithm is a trade-off between the convergence rate and steady-state prediction error imposed by the selection of a certain value for the learning rate. Therefore, a variable learning rate version (VLR–KLMAT algorithm) is also proposed based on a Lorentzian function. We analyze the stability and convergence behavior of the KLMAT algorithm and derive a sufficient condition to predict its learning rate behavior. Moreover, a kernel recursive extension of the KLMAT algorithm is further proposed for performance improvement. Simulation results in the context of time series prediction verify the effectiveness of the proposed algorithms.

Event-triggered Kalman-consensus filter for two-target tracking sensor networks.

Abstract: This paper is concerned with the problem of event-triggered Kalman-consensus filter for two-target tracking sensor networks. According to the event-triggered protocol and the mean-square analysis, a suboptimal Kalman gain matrix is derived and a suboptimal event-triggered distributed filter is obtained. Based on the Kalman-consensus filter protocol, all sensors which only depend on its neighbors' information can track their corresponding targets. Furthermore, utilizing Lyapunov method and matrix theory, some sufficient conditions are presented for ensuring the stability of the system. Finally, a simulation example is presented to verify the effectiveness of the proposed event-triggered protocol.

Adaptive Estimation of Multiple Fading Factors for GPS/INS Integrated Navigation Systems.

Abstract: The Kalman filter has been widely applied in the field of dynamic navigation and positioning However, its performance will be degraded in the presence of significant model errors and uncertain interferences In the literature, the fading filter was proposed to control the influences of the model errors, and the H-infinity filter can be adopted to address the uncertainties by minimizing the estimation error in the worst case In this paper, a new multiple fading factor, suitable for the Global Positioning System (GPS) and the Inertial Navigation System (INS) integrated navigation system, is proposed based on the optimization of the filter, and a comprehensive filtering algorithm is constructed by integrating the advantages of the H-infinity filter and the proposed multiple fading filter Measurement data of the GPS/INS integrated navigation system are collected under actual conditions Stability and robustness of the proposed filtering algorithm are tested with various experiments and contrastive analysis are performed with the measurement data Results demonstrate that both the filter divergence and the influences of outliers are restrained effectively with the proposed filtering algorithm, and precision of the filtering results are improved simultaneously

Non-casual linear prediction based adaptive filter for removal of high density impulse noise from color images

Abstract: In this paper, a non-causal linear prediction based adaptive vector median filter is proposed for removal of high density impulse noise from color images. Generally, when an image is affected by high density of impulse noise, homogeneity amongst the pixels is distorted. In the proposed method, if the pixel under operation is found to be corrupted, the filtering operation will be carried out. The decision about a particular pixel of being corrupted or not depends on the linear prediction error calculated from the non-causal region around the pixel under operation. If the error of the central pixel of the kernel exceeds some predefined threshold value, adaptive window based vector median filtering operation will be performed. The size of adaptive window will depend on the level of error according to the predefined threshold. The proposed filter improves the peak signal to noise ratio (PSNR) than that of modified histogram based fuzzy filter (MHFC) by approximately 4.5 dB. The results of structural similarity index measure (SSIM) suggest that the image details are maintained significantly better in the proposed method as compared to earlier approaches. It may be observed from subjective evaluation that the proposed method outperforms some of the existing filters.

PIV Uncertainty Quantification and Beyond

Abstract: The fundamental properties of computed flow fields using particle imaging velocimetry (PIV) have been investigated, viewing PIV processing as a black box without going in detail into algorithmic details. PIV processing can be analyzed using a linear filter model, i.e. assuming that the computed displacement field is the result of some spatial filtering of the underlying true flow field given a particular shape of the filter function. From such a mathematical framework, relationships are derived between the underlying filter function, wavelength response function (MTF) and response to a step function, power spectral density, and spatial autocorrelation of filter function and noise. A definition of a spatial resolution is provided independent of some arbitrary threshold e.g of the wavelength response function and provides the user with a single number to appropriately set the parameters of the PIV algorithm required for detecting small velocity fluctuations. The most important error sources in PIV are discussed and an uncertainty quantification method based on correlation statistics is derived, which has been compared to other available UQ-methods in two recent publications (Sciacchitano et al. 2015; Boomsma et al. 2016) showing good sensitivity to a variety of error sources. Instantaneous local velocity uncertainties are propagated for derived instantaneous and statistical quantities like vorticity, averages, Reynolds stresses and others. For Stereo-PIV the uncertainties of the 2C-velocity fields of the two cameras are propagated into uncertainties of the computed final 3C-velocity field. A new anisotropic denoising scheme as a post-processing step is presented which uses the uncertainties comparing to the local flow gradients in order to devise an optimal filter kernel for reducing the noise without suppressing true small-scale flow fluctuations. For Stereo-PIV and volumetric PIV/PTV, an accurate perspective calibration is mandatory. A Stereo-PIV self-calibration technique is described to correct misalignment between the actual position of the light sheet and where it is supposed to be according to the initial calibration procedure. For volumetric PIV/PTV, a volumetric self-calibration (VSC) procedure is presented to correct local calibration errors everywhere in the measurement volume. Finally, an iterative method for reconstructing particles (IPR) in a volume is developed, which is the basis for the recently introduced Shake-the-Box (STB) technique (Schanz et al. 2016).

An Adaptive Speed Estimation Method Based on a Strong Tracking Extended Kalman Filter with a Least-Square Algorithm for Induction Motors

Abstract: To improve the performance of sensorless induction motor (IM) drives, an adaptive speed estimation method based on a strong tracking extended Kalman filter with a least-square algorithm (LS-STEKF) for induction motors is proposed in this paper. With this method, a fading factor is introduced into the covariance matrix of the predicted state, which forces the innovation sequence orthogonal to each other and tunes the gain matrix online. In addition, the estimation error is adjusted adaptively and the mutational state is tracked fast. Simultaneously, the fading factor can be continuously self-tuned with the least-square algorithm according to the innovation sequence. The application of the least-square algorithm guarantees that the information in the innovation sequence is extracted as much as possible and as quickly as possible. Therefore, the proposed method improves the model adaptability in terms of actual systems and environmental variations, and reduces the speed estimation error. The correctness and the effectiveness of the proposed method are verified by experimental results.