Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

On robust estimation of the location parameter

It is shown that, particularly for terrestrial cellular telephony, the interference-suppression feature of CDMA (code division multiple access) can result in a many-fold increase in capacity over analog and even over competing digital techniques. A single-cell system, such as a hubbed satellite network, is addressed, and the basic expression for capacity is developed. The corresponding expressions for a multiple-cell system are derived. and the distribution on the number of users supportable per cell is determined. It is concluded that properly augmented and power-controlled multiple-cell CDMA promises a quantum increase in current cellular capacity. >

On the capacity of a cellular CDMA system

Imaging spectrometers measure electromagnetic energy scattered in their instantaneous field view in hundreds or thousands of spectral channels with higher spectral resolution than multispectral cameras. Imaging spectrometers are therefore often referred to as hyperspectral cameras (HSCs). Higher spectral resolution enables material identification via spectroscopic analysis, which facilitates countless applications that require identifying materials in scenarios unsuitable for classical spectroscopic analysis. Due to low spatial resolution of HSCs, microscopic material mixing, and multiple scattering, spectra measured by HSCs are mixtures of spectra of materials in a scene. Thus, accurate estimation requires unmixing. Pixels are assumed to be mixtures of a few materials, called endmembers. Unmixing involves estimating all or some of: the number of endmembers, their spectral signatures, and their abundances at each pixel. Unmixing is a challenging, ill-posed inverse problem because of model inaccuracies, observation noise, environmental conditions, endmember variability, and data set size. Researchers have devised and investigated many models searching for robust, stable, tractable, and accurate unmixing algorithms. This paper presents an overview of unmixing methods from the time of Keshava and Mustard's unmixing tutorial to the present. Mixing models are first discussed. Signal-subspace, geometrical, statistical, sparsity-based, and spatial-contextual unmixing algorithms are described. Mathematical problems and potential solutions are described. Algorithm characteristics are illustrated experimentally.

/pdf/hyperspectral-unmixing-overview-geometrical-statistical-and-2lb11fr9l4.pdf

Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches

Stable non-Gaussian random processes , by G. Samorodnitsky and M. S. Taqqu. Pp. 632. £49.50. 1994. ISBN 0-412-05171-0 (Chapman and Hall).

Advances in automated data acquisition in electron tomography have led to an explosion in the amount of data that can be obtained about the spatial architecture of a variety of biologically and medically relevant objects with resolutions in the "nano" range of 10-1000 nm. The development of methods to automatically analyze the vast amounts of information contained in these tomograms is a major challenge since the electron tomograms are intrinsically very noisy. A fundamental step in the automatic analysis of large amounts of data for statistical inference is to segment relevant 3D features in cellular tomograms. Procedures for segmentation must work robustly and rapidly in spite of the low signal to noise ratios inherent to biological electron microscopy. This work first evaluates various non-linear denoising techniques on tomograms recorded at cryogenic temperatures. Using datasets of bacterial tomograms as an example, we demonstrate that non-linear diffusion techniques significantly improve the fidelity of automated feature extraction. Our approach represents an important step in automating the efficient extraction of useful information from large datasets in biological tomography, and facilitates the overall goal of speeding up the process of reducing gigabyte-sized tomograms to relevant byte-sized data.

/pdf/from-gigabytes-to-bytes-automated-denoising-and-feature-1wix17wwez.pdf

From gigabytes to bytes: automated denoising and feature identification in electron tomograms of intact bacterial cells

A Wiener estimator structure is defined for the nonlinear prediction of scalar time series. This leads to a three stage strategy for constructing the predictor which involves: (1) estimation of the probability density function (pdf) associated with the embedding vector; (2) construction of a set of orthonormal polynomials defined on that pdf; (3) calculation of a set of coefficients to define a linear combiner which forms the prediction. The practical difficulties associated with (1) and the theoretical problems inherent in (2) lead to an approach based on a Volterra-type expansion. A set of orthonormal polynomials is constructed through eigenvalue analysis of the `correlation' matrix associated with the Volterra expansion. The orthonormal polynomials are defined by the eigenvalues and eigenvectors of this matrix -- the rank indicating the degree of the orthonormal approximation. Unlike the Wiener approach, the orthonormal polynomials are defined by the higher order moments present in the `correlation' matrix rather than the pdf. The radial basis function (RBF) network is then reexamined in light of this interpretation. The pseudoinverse, which is often used to calculate the coefficients of the RBF network, is in fact a method for deriving a set of orthonormal functions. Since each RBF can itself be expanded as power series, the eigenvalue analysis inherent in the formation of the pseudo-inverse defines a set of infinite orthonormal functions. In theory at least, the RBF network exploits the entire pdf as demanded by the Wiener approach. Some simulation results are presented concerning the Volterra analysis, albeit using least squares rather than this revised approach.© (1991) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Nonlinear prediction and the Wiener process

Target signature estimation is important in passive sonar signal processing. Higher order spectral methods such as bispectral estimation are often used as a complementary tool to conventional analysis. However, these methods require larger data sets to achieve consistent estimates. The sensitivity for non-stationarities are also more pronounced for bispectral methods. Here, we use a stepwise outlier rejection algorithm as both a stationarity test and a data conditioner for experimental sonar data. We demonstrate the importance of testing for significant quadratic phase coupling and using outlier rejection as a first step in bicoherence analysis of sonar data.

Sonar data conditioning by means of stepwise outlier rejection

The Use of Ray Tracing in Characterising The Indoor Mobile Radio Channel

Single-photon lidar has emerged as a prime candidate technology for depth imaging through challenging environments. Until now, a major limitation has been the significant amount of time required for the analysis of the recorded data. Here we show a new computational framework for real-time three-dimensional (3D) scene reconstruction from single-photon data. By combining statistical models with highly scalable computational tools from the computer graphics community, we demonstrate 3D reconstruction of complex outdoor scenes with processing times of the order of 20 ms, where the lidar data was acquired in broad daylight from distances up to 320 metres. The proposed method can handle an unknown number of surfaces in each pixel, allowing for target detection and imaging through cluttered scenes. This enables robust, real-time target reconstruction of complex moving scenes, paving the way for single-photon lidar at video rates for practical 3D imaging applications.

Stephen McLaughlin

Papers

From gigabytes to bytes: automated denoising and feature identification in electron tomograms of intact bacterial cells

Nonlinear prediction and the Wiener process

Sonar data conditioning by means of stepwise outlier rejection

The Use of Ray Tracing in Characterising The Indoor Mobile Radio Channel

Real-time 3D reconstruction of complex scenes using single-photon lidar: when image processing meets computer graphics