Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

On robust estimation of the location parameter

It is shown that, particularly for terrestrial cellular telephony, the interference-suppression feature of CDMA (code division multiple access) can result in a many-fold increase in capacity over analog and even over competing digital techniques. A single-cell system, such as a hubbed satellite network, is addressed, and the basic expression for capacity is developed. The corresponding expressions for a multiple-cell system are derived. and the distribution on the number of users supportable per cell is determined. It is concluded that properly augmented and power-controlled multiple-cell CDMA promises a quantum increase in current cellular capacity. >

On the capacity of a cellular CDMA system

Imaging spectrometers measure electromagnetic energy scattered in their instantaneous field view in hundreds or thousands of spectral channels with higher spectral resolution than multispectral cameras. Imaging spectrometers are therefore often referred to as hyperspectral cameras (HSCs). Higher spectral resolution enables material identification via spectroscopic analysis, which facilitates countless applications that require identifying materials in scenarios unsuitable for classical spectroscopic analysis. Due to low spatial resolution of HSCs, microscopic material mixing, and multiple scattering, spectra measured by HSCs are mixtures of spectra of materials in a scene. Thus, accurate estimation requires unmixing. Pixels are assumed to be mixtures of a few materials, called endmembers. Unmixing involves estimating all or some of: the number of endmembers, their spectral signatures, and their abundances at each pixel. Unmixing is a challenging, ill-posed inverse problem because of model inaccuracies, observation noise, environmental conditions, endmember variability, and data set size. Researchers have devised and investigated many models searching for robust, stable, tractable, and accurate unmixing algorithms. This paper presents an overview of unmixing methods from the time of Keshava and Mustard's unmixing tutorial to the present. Mixing models are first discussed. Signal-subspace, geometrical, statistical, sparsity-based, and spatial-contextual unmixing algorithms are described. Mathematical problems and potential solutions are described. Algorithm characteristics are illustrated experimentally.

/pdf/hyperspectral-unmixing-overview-geometrical-statistical-and-2lb11fr9l4.pdf

Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches

Stable non-Gaussian random processes , by G. Samorodnitsky and M. S. Taqqu. Pp. 632. £49.50. 1994. ISBN 0-412-05171-0 (Chapman and Hall).

Estimates of Higher Order Statistical quantities (such as the bicoherence) have higher variances than their second-order counterparts. Reliable estimates can be obtained by using longer data records, but in practice this is often not possible. In direct-method bicoherence estimation, estimates from shorter records can be highly dependent on measurement errors and background noise. To try to get around these problems, a new bicoherence measure based on the a-trimmed mean bispectrum is described. Simulations indicate how well this new measure performs compared to the standard bicoherence measure.

Robust Frequency-Domain Bispectrum Estimation

We present lower bounds on the two-dimensional capacity for two sets of symmetric and asymmetric M-ary runlength-limited constraints. The bounds extend and generalize our previous work on binary constraints. We also give sequential coding algorithms achieving the derived capacity lower bounds.

Capacity Lower Bounds for Two-Dimensional M-ary (0, k) and (d, ∞) Runlength-limited Channels

We propose a neural network equalizer with a fuzzy decision learning rule based on the generalized probabilistic descent algorithm with the minimum decision error formulation. The neural network used is the multi-layer perceptron. It is shown that the decision region overlapped by noise can be overcome by the use of a fuzzy decision learning rule based on the generalized probabilistic descent algorithm. We apply this algorithm to a neural network equalizer with binary sequences in a nonlinear distortion channel. Simulation results confirm that the fuzzy decision learning algorithm works more effectively than hard decision learning algorithms when the learning patterns are not separable by high additive noise.

A neural network equalizer with the fuzzy decision learning rule

This paper presents a family of approximate Bayesian methods for joint anomaly detection and linear regression in the presence of non-Gaussian noise Robust anomaly detection using non-convex sparsity-promoting regularization terms is generally challenging, in particular when additional uncertainty measures about the estimation process are needed, eg, posterior probabilities of anomaly presence The problem becomes even more challenging in the presence of non-Gaussian, (eg, Poisson distributed), additional constraints on the regression coefficients (eg, positivity) and when the anomalies present complex structures (eg, structured sparsity) Uncertainty quantification is classically addressed using Bayesian methods Specifically, Monte Carlo methods are the preferred tools to handle complex models Unfortunately, such simulation methods suffer from a significant computational cost and are thus not scalable for fast inference in high dimensional problems In this paper, we thus propose fast alternatives based on Expectation-Propagation (EP) methods, which aim at approximating complex distributions by more tractable models to simplify the inference process The main problem addressed in this paper is linear regression and (sparse) anomaly detection in the presence of noisy measurements The aim of this paper is to demonstrate the potential benefits and assess the performance of such EP-based methods The results obtained illustrate that approximate methods can provide satisfactory results with a reasonable computational cost It is important to note that the proposed methods are sufficiently generic to be used in other applications involving condition monitoring

Robust Linear Regression and Anomaly Detection in the Presence of Poisson Noise Using Expectation-Propagation

This paper describes a nonlinear model that is able to generate vowel sounds of any required duration which also contain jitter and shimmer, and hence are more natural-sounding than the equivalent sounds generated by linear prediction techniques. The model is based on a radial basis function (RBF) neural network, with a global feedback loop. The network is trained by first placing the radial basis centres onto either a subset of the input data or a fixed hyper-lattice structure. The network weights are then found so as to minimise the mean square error between the input data (which will be a stationary vowel sound segment) and the network output. Regularisation is used when calculating the weight values, as this ensures stability when the global feedback loop is connected for synthesis. (6 pages)

Stephen McLaughlin

Papers

Robust Frequency-Domain Bispectrum Estimation

Capacity Lower Bounds for Two-Dimensional M-ary (0, k) and (d, ∞) Runlength-limited Channels

A neural network equalizer with the fuzzy decision learning rule

Robust Linear Regression and Anomaly Detection in the Presence of Poisson Noise Using Expectation-Propagation

Using a nonlinear model to synthesise natural-sounding vowels