Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

/pdf/convex-analysisnoer-san-nojin-zhan-nituite-5dl2rbmoyr.pdf

Convex Analysisの二,三の進展について

On robust estimation of the location parameter

A random forest (RF) classifier is an ensemble classifier that produces multiple decision trees, using a randomly selected subset of training samples and variables. This classifier has become popular within the remote sensing community due to the accuracy of its classifications. The overall objective of this work was to review the utilization of RF classifier in remote sensing. This review has revealed that RF classifier can successfully handle high data dimensionality and multicolinearity, being both fast and insensitive to overfitting. It is, however, sensitive to the sampling design. The variable importance (VI) measurement provided by the RF classifier has been extensively exploited in different scenarios, for example to reduce the number of dimensions of hyperspectral data, to identify the most relevant multisource remote sensing and geographic data, and to select the most suitable season to classify particular target classes. Further investigations are required into less commonly exploited uses of this classifier, such as for sample proximity analysis to detect and remove outliers in the training samples.

Random forest in remote sensing: A review of applications and future directions

Context. The direct detection of faint exoplanets with high-contrast instruments can be boosted by combining it with high spectral resolution. For integral field spectrographs yielding hyperspectral data, this means that the majority of the field of view consists of diffracted starlight spectra and a spatially localized planet. Observation analysis usually relies on classic cross-correlation with theoretical spectra, maximized at the position and with the properties of the planet. In a purely blind-search context, this supervised strategy can be biased with model mismatch and/or be computationally inefficient.Aims. Using an approach that is inspired by the analysis of hyperspectral data within the remote-sensing community, we aim to propose an alternative to cross-correlation that is fully data-driven, which decomposes the data into a set of individual spectra and their corresponding spatial distributions. This strategy is called spectral unmixing.Methods. We used an orthogonal subspace projection to identify the most distinct spectra in the field of view. Their spatial distribution maps were then obtained by inverting the data. These spectra were then used to break the original hyperspectral images into their corresponding spatial distribution maps via non-negative least squares. A matched filter with the instrument point-spread function (or visual inspection) was then used to detect the planet on one of the maps. The performance of our method was evaluated and compared with a cross-correlation using simulated hyperspectral data with medium resolution from the ELT/HARMONI integral field spectrograph.Results. We show that spectral unmixing effectively leads to a planet detection solely based on spectral dissimilarities at significantly reduced computational cost. The extracted spectrum holds significant signatures of the planet while being not perfectly separated from residual starlight. The sensitivity of the supervised cross-correlation is three to four times higher than with unsupervised spectral unmixing, the gap is biased toward the former because the injected and correlated spectrum match perfectly. The algorithm was furthermore vetted on real data obtained with VLT/SINFONI of the β Pictoris system. This led to the detection of β Pictoris b with a signal-to-noise ratio of 28.5.Conclusions. Spectral unmixing is a viable alternative strategy to a cross-correlation to search for and characterize exoplanets in hyperspectral data in a purely data-driven approach. The advent of large data from the forthcoming IFS on board JWST and future ELTs motivates further algorithm development along this path.

/pdf/spectral-unmixing-for-exoplanet-direct-detection-in-4lu0n6sqwp.pdf

Spectral unmixing for exoplanet direct detection in hyperspectral data

The classification of hyperspectral imagery, using multiple classifier systems is discussed and an SVM-based ensemble is introduced. The data set is separated into separate feature subsets using the correlation between the different spectral bands as a criterion. Afterwards, each source is classified separately by an SVM classifier. Finally, the different outputs are used as inputs for final decision fusion that is based on an additional SVM classifier. The results using the proposed strategy are compared to classification results achieved by a single SVM and other well known classifier ensembles, such as random forests, boosting and bagging.

Ensemble Strategies for Classifying Hyperspectral Remote Sensing Data

Synthetic Aperture Sonar (SAS) imagery is largely used in detection, location and classification of underwater mines laying or buried in the sea bed. This paper proposes a detection method using Higher Order Statistics (HOS) on SAS images. The proposed method can be divided into two steps. Firstly, the HOS (Skewness and Kurtosis) are locally estimated using a square sliding computation window. In a second step, the results are focused by a correlation process. This enables the precise location of the objects. This method is tested on real SAS data containing both underwater mines laying on the sea bed and buried objects.

/pdf/on-the-use-of-higher-order-statistics-in-sas-imagery-li7ks2f52l.pdf

On the Use of Higher Order Statistics in SAS Imagery

Accurate representations of the Earth surface in both spatial and spectral domains are highly desirable in many applications using remotely sensed data. An effective solution is achieved by combining hyperspectral data, which are characterized by a high spectral diversity, with high spatial resolution images, collected by multispectral or panchromatic sensors. In this work, we compare the outcomes provided by fusing single-platform or multi-platform data. We demonstrate that the optimal choice depends on the target spatial resolution to be achieved. To this aim, real images collected by the Hyperion sensor are combined with data acquired by the ALI sensor or the QuickBird sensor assessing the fused outcomes at reduced resolution.

Pansharpening of hyperspectral images: Exploiting data acquired by multiple platforms

Classification of hyperspectral data with high spatial resolution using both spatial and spectral approaches is discussed. The spatial approach is based on mathematical morphology. In the method, several principal components (PCs) from the hyperspectral data are used. From each of the PCs, a morphological profile is built. These profiles are used together in one extended morphological profile, which is then classified with a neural network. The spectral classification approach is based on maximum likelihood classification and nonparametric weighted feature extraction (NWFE). The results from the spectral and spatial modeling are finally fused together using several different fusion rules. Experimental results are given on a hyperspectral data from an urban area.

/pdf/decision-level-fusion-in-classification-of-hyperspectral-58f1f72fn2.pdf

Jocelyn Chanussot

Papers

Spectral unmixing for exoplanet direct detection in hyperspectral data

Ensemble Strategies for Classifying Hyperspectral Remote Sensing Data

On the Use of Higher Order Statistics in SAS Imagery

Pansharpening of hyperspectral images: Exploiting data acquired by multiple platforms

Decision level fusion in classification of hyperspectral data from urban areas