Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

On robust estimation of the location parameter

It is shown that, particularly for terrestrial cellular telephony, the interference-suppression feature of CDMA (code division multiple access) can result in a many-fold increase in capacity over analog and even over competing digital techniques. A single-cell system, such as a hubbed satellite network, is addressed, and the basic expression for capacity is developed. The corresponding expressions for a multiple-cell system are derived. and the distribution on the number of users supportable per cell is determined. It is concluded that properly augmented and power-controlled multiple-cell CDMA promises a quantum increase in current cellular capacity. >

On the capacity of a cellular CDMA system

Imaging spectrometers measure electromagnetic energy scattered in their instantaneous field view in hundreds or thousands of spectral channels with higher spectral resolution than multispectral cameras. Imaging spectrometers are therefore often referred to as hyperspectral cameras (HSCs). Higher spectral resolution enables material identification via spectroscopic analysis, which facilitates countless applications that require identifying materials in scenarios unsuitable for classical spectroscopic analysis. Due to low spatial resolution of HSCs, microscopic material mixing, and multiple scattering, spectra measured by HSCs are mixtures of spectra of materials in a scene. Thus, accurate estimation requires unmixing. Pixels are assumed to be mixtures of a few materials, called endmembers. Unmixing involves estimating all or some of: the number of endmembers, their spectral signatures, and their abundances at each pixel. Unmixing is a challenging, ill-posed inverse problem because of model inaccuracies, observation noise, environmental conditions, endmember variability, and data set size. Researchers have devised and investigated many models searching for robust, stable, tractable, and accurate unmixing algorithms. This paper presents an overview of unmixing methods from the time of Keshava and Mustard's unmixing tutorial to the present. Mixing models are first discussed. Signal-subspace, geometrical, statistical, sparsity-based, and spatial-contextual unmixing algorithms are described. Mathematical problems and potential solutions are described. Algorithm characteristics are illustrated experimentally.

/pdf/hyperspectral-unmixing-overview-geometrical-statistical-and-2lb11fr9l4.pdf

Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches

Stable non-Gaussian random processes , by G. Samorodnitsky and M. S. Taqqu. Pp. 632. £49.50. 1994. ISBN 0-412-05171-0 (Chapman and Hall).

Time-correlated single-photon technology is emerging as an important approach to 3D Imaging. This paper presents a reconstruction algorithm that exploits data statistics and multi-scale information to deliver clean depth and reflectivity images together with associated uncertainty maps. The statistical method has been implemented to run on graphics processing units (GPUs) that enable real-time reconstruction of moving scenes at more than 1000 depth frames per second on the 32 × 64 pixels real Quantic4x4 SPAD sensor array data. Comparisons with state-of-the-art algorithms on simulated and real data demonstrate the robust and efficient performance of the proposed method. 

Fast Multiscale 3D Reconstruction Using Single-Photon Lidar Data

In this paper, a new Expectation Propagation (EP) algorithm using ℓ1-norm total variation (ℓ1-TV) prior is proposed for color image restoration in the low photon-count regime. Different from most color image restoration methods proposed for the restoration of color images from observations that are already color images with some missing pixels and/or are usually corrupted by Gaussian noise, the observations considered in this paper are only a single channel grayscale image without color information and are corrupted by Poisson noise, making the color image restoration problem more difficult. To address the problem, a new efficient EP algorithm is proposed to estimate the RGB values of each pixel from such observations and simultaneously provide uncertainty quantification of the estimates. Moreover, by coupling the EP algorithm with a variational Expectation Maximization (EM) approach, the ℓ1-TV prior hyperparameter can be adjusted automatically with-out user supervision. Experiments on color image inpainting and compressive sensing (CS) reconstruction are conducted to illustrate the potential benefits of the proposed EP algorithm for color image restoration in the low photon-count regime.

Color Image Restoration in the Low Photon-Count Regime Using Expectation Propagation

This paper addresses the problem of estimating the parameters of a moving average (MA) model from either only third- or fourth-order cumulants of the noisy observations of the system output. The system is driven by an independent and identically distributed non-Gaussian sequence that is not observed. The unknown model parameters are obtained using a batch least squares method. Recursive methods are also developed and used to claim the uniqueness of the batch least squares solutions. A novel technique for the enhancement of third-order cumulants of MA processes is introduced. This new technique is based on the concept of composite property mappings and helps reduce the variance of the estimates of third- (or fourth)-order cumulants of MA processes. Simulation results are presented that demonstrate the performance of the new methods and compare them with a range of existing techniques.

http://ieeexplore.ieee.org/iel4/78/11014/00510618.pdf

MA Parameter Estimation

A new chip-level MMSE frequency domain equalizer (FDE) is investigated for a universal mobile telephone system (UMTS) downlink. Multiple access interference (MAI) and inter chip interference (ICI) can be reduced at the chip level before despreading. By exploiting the frame and slot structures of the UMTS downlink, the pilots within one slot (for FDD mode) can be used for the required cyclic reconstruction in an FDE. No cyclic prefix (CP) or zero-padding is required at the transmitter, which would change the signal format of the current UMTS system. Furthermore, one slot signal is split into multiple segments for the sake of combating channel variance within one slot. The performance of the proposed MMSE-FDE is studied through simulation. A multimode receiver that can equalize both UMTS and OFDM systems in one structure is then feasible.

UMTS FDD frequency domain equalization based on slot segmentation

This paper presents a new Expectation Propagation (EP) framework for image restoration using patch-based prior distributions. While Monte Carlo techniques are classically used to sample from intractable posterior distributions, they can suffer from scalability issues in high-dimensional inference problems such as image restoration. To address this issue, EP is used here to approximate the posterior distributions using products of multivariate Gaussian densities. Moreover, imposing structural constraints on the covariance matrices of these densities allows for greater scalability and distributed computation. While the method is naturally suited to handle additive Gaussian observation noise, it can also be extended to non-Gaussian noise. Experiments conducted for denoising, inpainting and deconvolution problems with Gaussian and Poisson noise illustrate the potential benefits of such flexible approximate Bayesian method for uncertainty quantification in imaging problems, at a reduced computational cost compared to sampling techniques.

Stephen McLaughlin

Papers

Fast Multiscale 3D Reconstruction Using Single-Photon Lidar Data

Color Image Restoration in the Low Photon-Count Regime Using Expectation Propagation

MA Parameter Estimation

UMTS FDD frequency domain equalization based on slot segmentation

Patch-Based Image Restoration using Expectation Propagation.