Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

A random forest (RF) classifier is an ensemble classifier that produces multiple decision trees, using a randomly selected subset of training samples and variables. This classifier has become popular within the remote sensing community due to the accuracy of its classifications. The overall objective of this work was to review the utilization of RF classifier in remote sensing. This review has revealed that RF classifier can successfully handle high data dimensionality and multicolinearity, being both fast and insensitive to overfitting. It is, however, sensitive to the sampling design. The variable importance (VI) measurement provided by the RF classifier has been extensively exploited in different scenarios, for example to reduce the number of dimensions of hyperspectral data, to identify the most relevant multisource remote sensing and geographic data, and to select the most suitable season to classify particular target classes. Further investigations are required into less commonly exploited uses of this classifier, such as for sample proximity analysis to detect and remove outliers in the training samples.

Random forest in remote sensing: A review of applications and future directions

A wide range of methods for analysis of airborne- and satellite-derived imagery continues to be proposed and assessed. In this paper, we review remote sensing implementations of support vector machines (SVMs), a promising machine learning methodology. This review is timely due to the exponentially increasing number of works published in recent years. SVMs are particularly appealing in the remote sensing field due to their ability to generalize well even with limited training samples, a common limitation for remote sensing applications. However, they also suffer from parameter assignment issues that can significantly affect obtained results. A summary of empirical results is provided for various applications of over one hundred published works (as of April, 2010). It is our hope that this survey will provide guidelines for future applications of SVMs and possible areas of algorithm enhancement.

https://www.aboutgis.com/Publications/Mountrakis_SVM_review_in_remote_sensing_ISPRS2010.pdf

Support vector machines in remote sensing: A review

Land cover monitoring using remotely sensed data requires robust classification methods which allow for the accurate mapping of complex land cover and land use categories. Random forest (RF) is a powerful machine learning classifier that is relatively unknown in land remote sensing and has not been evaluated thoroughly by the remote sensing community compared to more conventional pattern recognition techniques. Key advantages of RF include: their non-parametric nature; high classification accuracy; and capability to determine variable importance. However, the split rules for classification are unknown, therefore RF can be considered to be black box type classifier. RF provides an algorithm for estimating missing values; and flexibility to perform several types of data analysis, including regression, classification, survival analysis, and unsupervised learning.

In this paper, the performance of the RF classifier for land cover classification of a complex area is explored. Evaluation was based on several criteria: mapping accuracy, sensitivity to data set size and noise. Landsat-5 Thematic Mapper data captured in European spring and summer were used with auxiliary variables derived from a digital terrain model to classify 14 different land categories in the south of Spain. Results show that the RF algorithm yields accurate land cover classifications, with 92% overall accuracy and a Kappa index of 0.92. RF is robust to training data reduction and noise because significant differences in kappa values were only observed for data reduction and noise addition values greater than 50 and 20%, respectively. Additionally, variables that RF identified as most important for classifying land cover coincided with expectations. A McNemar test indicates an overall better performance of the random forest model over a single decision tree at the 0.00001 significance level.

An assessment of the effectiveness of a random forest classifier for land-cover classification

The development of the Internet in recent years has made it possible and useful to access many different information systems anywhere in the world to obtain information. While there is much research on the integration of heterogeneous information systems, most commercial systems stop short of the actual integration of available data. Data fusion is the process of fusing multiple records representing the same real-world object into a single, consistent, and clean representation.This article places data fusion into the greater context of data integration, precisely defines the goals of data fusion, namely, complete, concise, and consistent data, and highlights the challenges of data fusion, namely, uncertain and conflicting data values. We give an overview and classification of different ways of fusing data and present several techniques based on standard and advanced operators of the relational algebra and SQL. Finally, the article features a comprehensive survey of data integration systems from academia and industry, showing if and how data fusion is performed in each.

Data fusion

The accuracy of supervised land cover classifications depends on factors such as the chosen classification algorithm, adequate training data, the input data characteristics, and the selection of features. Hyperspectral imaging provides more detailed spectral and spatial information on the land cover than other remote sensing resources. Over the past ten years, traditional and formerly widely accepted statistical classification methods have been superseded by more recent machine learning algorithms, e.g., support vector machines (SVMs), or by multiple classifier systems (MCS). This can be explained by limitations of statistical approaches with regard to high-dimensional data, multimodal classes, and often limited availability of training data. In the presented study, MCSs based on SVM and random feature selection (RFS) are applied to explore the potential of a synergetic use of the two concepts. We investigated how the number of selected features and the size of the MCS influence classification accuracy using two hyperspectral data sets, from different environmental settings. In addition, experiments were conducted with a varying number of training samples. Accuracies are compared with regular SVM and random forests. Experimental results clearly demonstrate that the generation of an SVM-based classifier system with RFS significantly improves overall classification accuracy as well as producer's and user's accuracies. In addition, the ensemble strategy results in smoother, i.e., more realistic, classification maps than those from stand-alone SVM. Findings from the experiments were successfully transferred onto an additional hyperspectral data set.

/pdf/sensitivity-of-support-vector-machines-to-random-feature-18rtr1yhj5.pdf

Sensitivity of Support Vector Machines to Random Feature Selection in Classification of Hyperspectral Data

SAR data are almost independent from weather conditions, and thus are well suited for mapping of seasonally changing variables such as land cover In regard to recent and upcoming missions, multitemporal and multi-frequency approaches become even more attractive In the present study, classifier ensembles (ie, boosted decision tree and random forests) are applied to multi-temporal C-band SAR data, from different study sites and years A detailed accuracy assessment shows that classifier ensembles, in particularly random forests, outperform standard approaches like a single decision tree and a conventional maximum likelihood classifier by more than 10% independently from the site and year They reach up to almost 84% of overall accuracy in rural areas with large plots Visual interpretation confirms the statistical accuracy assessment and reveals that also typical random noise is considerably reduced In addition the results demonstrate that random forests are less sensitive to the number of training samples and perform well even with only a small number Random forests are computationally highly efficient and are hence considered very well suited for land cover classifications of future multifrequency and multitemporal stacks of SAR imagery

Classifier ensembles for land cover mapping using multitemporal SAR imagery

A strategy for the joint classification of multiple segmentation levels from multisensor imagery is introduced by using synthetic aperture radar and optical data. At first, the two data sets are separately segmented, creating independent aggregation levels at different scales. Each individual level from the two sensors is then preclassified by a support vector machine (SVM). The original outputs of each SVM, i.e., images showing the distances of the pixels to the hyperplane fitted by the SVM, are used in a decision fusion to determine the final classes. The fusion strategy is based on the application of an additional classifier, which is applied on the preclassification results. Both a second SVM and random forests (RF) were tested for the decision fusion. The results are compared with SVM and RF applied to the full data set without preclassification. Both the integration of multilevel information and the use of multisensor imagery increase the overall accuracy. It is shown that the classification of multilevel-multisource data sets with SVM and RF is feasible and does not require a definition of ideal aggregation levels. The proposed decision fusion approach that applies RF to the preclassification outperforms all other approaches.

Classifying Multilevel Imagery From SAR and Optical Sensors by Decision Fusion

Very high spatial resolution (VHR) images allow to feature man-made structures such as roads and thus enable their accurate analysis. Geometrical characteristics can be extracted using mathematical morphology. However, the prior choice of a reference shape (structuring element) introduces a shape-bias. This paper presents a new method for extracting roads in Very High Resolution remotely sensed images based on advanced directional morphological operators. The proposed approach introduces the use of Path Openings and Path Closings in order to extract structural pixel information. These morphological operators remain flexible enough to fit rectilinear and slightly curved structures since they do not depend on the choice of a structural element shape. As a consequence, they outperform standard approaches using rotating rectangular structuring elements. The method consists in building a granulometry chain using Path Openings and Path Closing to construct Morphological Profiles. For each pixel, the Morphological Profile constitutes the feature vector on which our road extraction is based.

https://hal.archives-ouvertes.fr/hal-00578852/document

Advanced directional mathematical morphology for the detection of the road network in very high resolution remote sensing images

Hyperspectral imaging provides detailed spectral and spatial information from the land cover that enables a precise differentiation between various surface materials. On the other hand, the performance of traditional and widely used statistical classification methods is often limited in this context, and thus alternative methods are required. In the study presented here, the performance of two machine-learning techniques, namely support vector machines (SVMs) and random forests (RFs), is investigated and the classification results are compared with those from well-known methods (i.e., maximum likelihood classifier and spectral angle mapper). The classifiers are applied to an Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) dataset that was acquired near the Hekla volcano in Iceland. The results clearly show the advantages of the two proposed classifier algorithms in terms of accuracy. They significantly outperform the other methods and achieve overall accuracies of approximately 90%. Although SVM a...

Björn Waske

Papers

Sensitivity of Support Vector Machines to Random Feature Selection in Classification of Hyperspectral Data

Classifier ensembles for land cover mapping using multitemporal SAR imagery

Classifying Multilevel Imagery From SAR and Optical Sensors by Decision Fusion

Advanced directional mathematical morphology for the detection of the road network in very high resolution remote sensing images

Mapping of hyperspectral AVIRIS data using machine-learning algorithms