Machine Learning is the study of methods for programming computers to learn. Computers are applied to a wide range of tasks, and for most of these it is relatively easy for programmers to design and implement the necessary software. However, there are many tasks for which this is difficult or impossible. These can be divided into four general categories. First, there are problems for which there exist no human experts. For example, in modern automated manufacturing facilities, there is a need to predict machine failures before they occur by analyzing sensor readings. Because the machines are new, there are no human experts who can be interviewed by a programmer to provide the knowledge necessary to build a computer system. A machine learning system can study recorded data and subsequent machine failures and learn prediction rules. Second, there are problems where human experts exist, but where they are unable to explain their expertise. This is the case in many perceptual tasks, such as speech recognition, hand-writing recognition, and natural language understanding. Virtually all humans exhibit expert-level abilities on these tasks, but none of them can describe the detailed steps that they follow as they perform them. Fortunately, humans can provide machines with examples of the inputs and correct outputs for these tasks, so machine learning algorithms can learn to map the inputs to the outputs. Third, there are problems where phenomena are changing rapidly. In finance, for example, people would like to predict the future behavior of the stock market, of consumer purchases, or of exchange rates. These behaviors change frequently, so that even if a programmer could construct a good predictive computer program, it would need to be rewritten frequently. A learning program can relieve the programmer of this burden by constantly modifying and tuning a set of learned prediction rules. Fourth, there are applications that need to be customized for each computer user separately. Consider, for example, a program to filter unwanted electronic mail messages. Different users will need different filters. It is unreasonable to expect each user to program his or her own rules, and it is infeasible to provide every user with a software engineer to keep the rules up-to-date. A machine learning system can learn which mail messages the user rejects and maintain the filtering rules automatically. Machine learning addresses many of the same research questions as the fields of statistics, data mining, and psychology, but with differences of emphasis. Statistics focuses on understanding the phenomena that have generated the data, often with the goal of testing different hypotheses about those phenomena. Data mining seeks to find patterns in the data that are understandable by people. Psychological studies of human learning aspire to understand the mechanisms underlying the various learning behaviors exhibited by people (concept learning, skill acquisition, strategy change, etc.).

Machine learning

Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

/pdf/convex-analysisnoer-san-nojin-zhan-nituite-5dl2rbmoyr.pdf

Convex Analysisの二,三の進展について

On robust estimation of the location parameter

A random forest (RF) classifier is an ensemble classifier that produces multiple decision trees, using a randomly selected subset of training samples and variables. This classifier has become popular within the remote sensing community due to the accuracy of its classifications. The overall objective of this work was to review the utilization of RF classifier in remote sensing. This review has revealed that RF classifier can successfully handle high data dimensionality and multicolinearity, being both fast and insensitive to overfitting. It is, however, sensitive to the sampling design. The variable importance (VI) measurement provided by the RF classifier has been extensively exploited in different scenarios, for example to reduce the number of dimensions of hyperspectral data, to identify the most relevant multisource remote sensing and geographic data, and to select the most suitable season to classify particular target classes. Further investigations are required into less commonly exploited uses of this classifier, such as for sample proximity analysis to detect and remove outliers in the training samples.

Random forest in remote sensing: A review of applications and future directions

Although hyperspectral data, especially spaceborne images, are rich in spectral information, their spatial resolution is usually low due to the limitation of sensor design and other factors. Therefore, for the application of hyperspectral images, unmixing technology is a key processing technology, such as linear mixing model and its derived algorithms have made a certain progress. However, a real scene often contains both pure and mixed pixels. The existing methods usually ignore the consideration and analysis of this situation in the process of model design and simulation experiment. In this context, this paper proposes a robust linear unmixing model with the enhanced constraint of classification for hyperspectral image. In general, it designs a framework combining unmixing and classification. In the task for real scene data, endmembers are extracted first, and then the hard classification term constructed after the expansion of endmembers (training samples) based on similarity is introduced to provide the sparsity constraint of the overall model, so as to realize relatively complete adjustment and effective image unmixing under complex conditions. Considering the scene with different distributions, the simulation experiment designs several groups of data tests, including different proportions of pure and mixing pixels. The unmixing results of three simulated datasets and two real datasets show that the unmixing results of this method are better than those of the other six comparison methods. This model improves the accuracy of unmixing and realizes effective unmixing. 

https://onlinelibrary.wiley.com/doi/pdfdirect/10.1049/ipr2.12568

Robust linear unmixing with enhanced constraint of classification for hyperspectral remote sensing imagery

Hyperspectral imaging offers new perspectives for diverse applications, ranging from the monitoring of the environment using airborne or satellite remote sensing, precision farming, food safety, planetary exploration, or astrophysics. Unfortunately, the spectral diversity of information comes at the expense of various sources of degradation, and the lack of accurate ground-truth "clean" hyperspectral signals acquired on the spot makes restoration tasks challenging. In particular, training deep neural networks for restoration is difficult, in contrast to traditional RGB imaging problems where deep models tend to shine. In this paper, we advocate instead for a hybrid approach based on sparse coding principles that retains the interpretability of classical techniques encoding domain knowledge with handcrafted image priors, while allowing to train model parameters end-to-end without massive amounts of data. We show on various denoising benchmarks that our method is computationally efficient and significantly outperforms the state of the art.

A Trainable Spectral-Spatial Sparse Coding Model for Hyperspectral Image Restoration.

Multimodal data provide complementary information of a natural phenomenon by integrating data from various domains with very different statistical properties. Capturing the intra-modality and cross-modality information of multimodal data is the essential capability of multimodal learning methods. The geometry-aware data analysis approaches provide these capabilities by implicitly representing data in various modalities based on their geometric underlying structures. Also, in many applications, data are explicitly defined on an intrinsic geometric structure. Generalizing deep learning methods to the non-Euclidean domains is an emerging research field, which has recently been investigated in many studies. Most of those popular methods are developed for unimodal data. In this paper, a multimodal multi-scaled graph wavelet convolutional network (M-GWCN) is proposed as an end-to-end network. M-GWCN simultaneously finds intra-modality representation by applying the multiscale graph wavelet transform to provide helpful localization properties in the graph domain of each modality, and cross-modality representation by learning permutations that encode correlations among various modalities. M-GWCN is not limited to either the homogeneous modalities with the same number of data, or any prior knowledge indicating correspondences between modalities. Several semi-supervised node classification experiments have been conducted on three popular unimodal explicit graph-based datasets and five multimodal implicit ones. The experimental results indicate the superiority and effectiveness of the proposed methods compared with both spectral graph domain convolutional neural networks and state-of-the-art multimodal methods.

Geometric Multimodal Deep Learning with Multi-Scaled Graph Wavelet Convolutional Network

The accurate and fast assessment of damaged buildings following a disaster is critical for planning rescue and reconstruction efforts. The damage assessment by the traditional methods is time-consuming and with limited performance. In this article, we propose an end-to-end deep-learning network named building damage detection network-plus (BDD-Net+). The BDD-Net+ is based on a combination of convolution layers and transformer blocks. The proposed framework takes the advantage of the multiscale residual convolution blocks and self-attention layers. The proposed framework consists of four main steps: data preparation, model training, damage map generation and evaluation, and the use of an explainable artificial intelligence (XAI) framework for understanding and interpretation of the operation model. The experimental results include two representative real-world benchmark datasets (i.e., the Haiti earthquake and the Bata explosion). The obtained results illustrate that BDD-Net+ achieves excellent efficacy in comparison with other state-of-the-art methods. Furthermore, the visualization of the results by XAI shows that BDD-Net+ provides more interpretable and explainable results for damage detection than the other studied methods.

https://ieeexplore.ieee.org/ielx7/4609443/4609444/10103581.pdf

BDD-Net+: A Building Damage Detection Framework Based on Modified Coat-Net

The classical definition of vector order filters consists in selecting the pixel that minimizes the cumulated distance to the other pixels of the filtering window. The "most representative" pixel of the filtering window is then selected and the noise is thus reduced. But, when the filtering window reaches a transition, the result is biased and the smoothing is not optimal any more. The proposed technique consists in progressively decimating the filtering window by suppressing the pixels that maximize the cumulated distance. The process is then iterated (computation of the cumulated distances among the remaining pixels and suppression of the less typical pixels) until one single pixel remains. It is then selected for the filter output. In practical cases, the number of iterations is limited. Applied on colour images, the filter results in an enhancement of the edges and in an automatic registration of the different components. The counterpart is a slight degradation of the smoothing performance in homogeneous regions. The paper details the proposed method and compares it with other enhancement filters.

Jocelyn Chanussot

Papers

Robust linear unmixing with enhanced constraint of classification for hyperspectral remote sensing imagery

A Trainable Spectral-Spatial Sparse Coding Model for Hyperspectral Image Restoration.

Geometric Multimodal Deep Learning with Multi-Scaled Graph Wavelet Convolutional Network

BDD-Net+: A Building Damage Detection Framework Based on Modified Coat-Net

Order filter with progressively decimated filtering window: application to colour image enhancement