Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

/pdf/convex-analysisnoer-san-nojin-zhan-nituite-5dl2rbmoyr.pdf

Convex Analysisの二,三の進展について

On robust estimation of the location parameter

A random forest (RF) classifier is an ensemble classifier that produces multiple decision trees, using a randomly selected subset of training samples and variables. This classifier has become popular within the remote sensing community due to the accuracy of its classifications. The overall objective of this work was to review the utilization of RF classifier in remote sensing. This review has revealed that RF classifier can successfully handle high data dimensionality and multicolinearity, being both fast and insensitive to overfitting. It is, however, sensitive to the sampling design. The variable importance (VI) measurement provided by the RF classifier has been extensively exploited in different scenarios, for example to reduce the number of dimensions of hyperspectral data, to identify the most relevant multisource remote sensing and geographic data, and to select the most suitable season to classify particular target classes. Further investigations are required into less commonly exploited uses of this classifier, such as for sample proximity analysis to detect and remove outliers in the training samples.

Random forest in remote sensing: A review of applications and future directions

Ecosystem management and monitoring are essential to preserve natural resources. Hyperspectral imagery (HSI) is a useful tool to obtain accurate classification maps, providing significant level of detail. Thus, traditional and novel methodologies based on pixel and object classification approaches are compared and evaluated in a homogeneous and mixed vulnerable ecosystem. Considering the challenging ecosystem, all classifications successfully resulted in high OA (higher than 82%), showing that HSI is very useful providing accurate vegetation maps to evaluate and monitor the ecosystems in a faster and economic way.

Evaluation of Hyperspectral Classification Maps in Heterogeneous Ecosystem

Machine learning is devoted to the design of methods and algorithms able to learn from empirical data. This approach is especially important in signal and image processing, where sets of sensors, usually large and heterogeneous, provide large amounts of data, usually noisy and corrupted with various sources of interference. From a methodological point of view, machine learning is concerned with multi-dimensional and statistical signal processing, especially with problems such as detection, estimation, and optimization. In addition to classical supervised or unsupervised learning, reinforcement learning and semisupervised learning, machine learning methods include Bayesian modeling, Markov models, support vector machines, and kernel methods. It spans a broad area of applications, such as adaptive filtering, pattern recognition, scene analysis in computer vision, data mining, robot control, data fusion, blind and semi-blind source separation, sparse component analysis, brain-computer interfaces, hyperspectral images, and cognitive radio. This special issue has been designed following the IEEE international workshop Machine Learning for Signal Processing which was held in Grenoble (France) in September 2009. The papers have been extended and rereviewed. In addition to theoretical contributions in signal detection, pattern recognition and classification, blind source separation, learning theory, Bayesian learning and modeling, and to applied contributions in speech and audio processing, biomedical application and communications, we have featured three Special Sessions, one on Braincomputer Interfaces, the second on Machine Learning in Remote Sensing Data Processing, and the third one on Learning in Markov Models. The 19 papers of this special issue reflect the current trends in Machine Learning.

/pdf/special-issue-on-machine-learning-for-signal-processing-2d1w445rum.pdf

Special Issue on Machine Learning for Signal Processing

On the transient signal characterization using phase space trajectory dynamics

Multispectral and hyperspectral imagery are often used for estimating crop yield. This paper describes an unsupervised unmixing scheme of hyperspectral images on field in order to estimate the crop yield. From the hyperspectral images, the endmembers and their abundance maps are computed by unsupervised unmixing. The abundance maps are then compared with the crop yield data. The results show the capability for estimating crop yield of the unmixing scheme, thanks to the high correlations between the crop yield data and the abundance maps of the endmembers corresponding to crop, even though the scheme is totally unsupervised.

/pdf/unsupervised-linear-unmixing-of-hyperspectral-image-for-crop-3re8gn0rt8.pdf

Unsupervised linear unmixing of hyperspectral image for crop yield estimation

Hyperspectral anomaly detection (AD) aims to detect objects significantly different from their surrounding background. Recently, many detectors based on autoencoder (AE) exhibited promising performances in hyperspectral AD tasks. However, the fundamental hypothesis of the AE-based detector that anomaly is more challenging to be reconstructed than background may not always be true in practice. We demonstrate that an AE could well reconstruct anomalies even without anomalies for training, because AE models mainly focus on the quality of sample reconstruction and do not care if the encoded features solely represent the background rather than anomalies. If more information is preserved than needed to reconstruct the background, the anomalies will be well reconstructed. This article proposes a cluster-memory augmented deep autoencoder via optimal transportation for hyperspectral anomaly detection (OTCMA) clustering for hyperspectral AD to solve this problem. The deep clustering method based on optimal transportation (OT) is proposed to enhance the features consistency of samples within the same categories and features discrimination of samples in different categories. The memory module stores the background’s consistent features, which are the cluster centers for each category background. We retrieve more consistent features from the memory module instead of reconstructing a sample utilizing its own encoded features. The network focuses more on consistent feature reconstruction by training AE with a memory module. This effectively restricts the reconstruction ability of AE and prevents reconstructing anomalies. Extensive experiments on the benchmark datasets demonstrate that our proposed OTCMA achieves state-of-the-art results. Besides, this article presents further discussions about the effectiveness of our proposed memory module and different criteria for better AD.

Jocelyn Chanussot

Papers

Evaluation of Hyperspectral Classification Maps in Heterogeneous Ecosystem

Special Issue on Machine Learning for Signal Processing

On the transient signal characterization using phase space trajectory dynamics

Unsupervised linear unmixing of hyperspectral image for crop yield estimation

Cluster-Memory Augmented Deep Autoencoder via Optimal Transportation for Hyperspectral Anomaly Detection