We propose a robust learning algorithm and apply it to recurrent neural networks. This algorithm is based on filtering outliers from the data and then estimating parameters from the filtered data. The filtering removes outliers from both the target function and the inputs of the neural network. The filtering is soft in that some outliers are neither completely rejected nor accepted. To show the need for robust recurrent networks, we compare the predictive ability of least squares estimated recurrent networks on synthetic data and on the Puget Power Electric Demand time series. These investigations result in a class of recurrent neural networks, NARMA(p,q), which show advantages over feedforward neural networks for time series with a moving average component. Conventional least squares methods of fitting NARMA(p,q) neural network models are shown to suffer a lack of robustness towards outliers. This sensitivity to outliers is demonstrated on both the synthetic and real data sets. Filtering the Puget Power Electric Demand time series is shown to automatically remove the outliers due to holidays. Neural networks trained on filtered data are then shown to give better predictions than neural networks trained on unfiltered time series. >

/pdf/recurrent-neural-networks-and-robust-time-series-prediction-10igzaep85.pdf

Recurrent neural networks and robust time series prediction

In recent years there has been much interest in robustness issues in general and in robust signal processing schemes in particular. Robust schemes are useful in situations where imprecise a priori knowledge of input characteristics makes the sensitivity of performance to deviations from assumed conditions an important factor in the design of good signal processing schemes. In this survey we discuss the minimax approach for the design of robust methods for signal processing. This has proven to be a very useful approach because it leads to constructive procedures for designing robust schemes. Our emphasis is on the contributions which have been made in robust signal processing, although key results of other robust statistical procedures are also considered. Most of the results we survey have been obtained in the past fifteen years, although some interesting earlier ideas for minimax signal processing are also mentioned. This survey is organized into five main parts, which deal separately with robust linear filters for signal estimation, robust linear filters for signal detection and related applications, nonlinear methods for robust signal detection, nonlinear methods for robust estimation, and robust data quantization. The interrelationships among many of these results are also discussed in the survey.

Robust techniques for signal processing: A survey

A family of nonlinear filters based on order statistics is presented. A mathematical tool derived through robust estimation theory, order statistics has allowed engineers to develop nonlinear filters with excellent robustness properties. These filters are well suited to digital image processing because they preserve the edges and the fine details of an image much better than conventional linear filters. The probabilistic and deterministic properties of the best known and most widely used filter in this family, the median filter, are discussed. In addition, the authors consider filters that, while not based on order statistics, are related to them through robust estimation theory. A table that ranks nonlinear filters under a variety of performance criteria is included. Most of the topics treated are very active research areas, and the applications are varied, including HDTV, multichannel signal processing of geophysical and ECG/EEG data, and a variety of telecommunications applications. >

/pdf/order-statistics-in-digital-image-processing-1rhdihn5hi.pdf

Order statistics in digital image processing

Robust estimation of power spectra, coherences, and transfer functions is investigated in the context of geophysical data processing. The methods described are frequency-domain extensions of current techniques from the statistical literature and are applicable in cases where section-averaging methods would be used with data that are contaminated by local nonstationarity or isolated outliers. The paper begins with a review of robust estimation theory, emphasizing statistical principles and the maximum likelihood or M-estimators. These are combined with section-averaging spectral techniques to obtain robust estimates of power spectra, coherences, and transfer functions in an automatic, data-adaptive fashion. Because robust methods implicitly identify abnormal data, methods for monitoring the statistical behavior of the estimation process using quantile-quantile plots are also discussed. The results are illustrated using a variety of examples from electromagnetic geophysics.

http://basin.earth.ncu.edu.tw/download/courses/seminar_MSc/2009/1224-1_On%20the%20Robust%20Estimation%20of%20Power%20Spectra,%20Coherences,%20and%20Transfer%20Functions.pdf

On the robust estimation of power spectra, coherences, and transfer functions

Robust Inference

https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/j.2517-6161.1979.tb01087.x

B. Kleiner

Papers

Robust Estimation of Power Spectra