Combining MLC and SVM classifiers for learning based decision making: analysis and evaluations
01 Jan 2015-Computational Intelligence and Neuroscience (Hindawi Publishing Corporation)-Vol. 2015, pp 423581-423581
TL;DR: MLC and SVM are combined in learning and classification, which helps to yield probabilistic output for SVM and facilitate soft decision making and interesting results are reported to indicate how the combined classifier may work under various conditions.
Abstract: Maximum likelihood classifier (MLC) and support vector machines (SVM) are two commonly used approaches in machine learning. MLC is based on Bayesian theory in estimating parameters of a probabilistic model, whilst SVM is an optimization based nonparametric method in this context. Recently, it is found that SVM in some cases is equivalent to MLC in probabilistically modeling the learning process. In this paper, MLC and SVM are combined in learning and classification, which helps to yield probabilistic output for SVM and facilitate soft decision making. In total four groups of data are used for evaluations, covering sonar, vehicle, breast cancer, and DNA sequences.The data samples are characterized in terms of Gaussian/non-Gaussian distributed and balanced/unbalanced samples which are then further used for performance assessment in comparing the SVM and the combined SVM-MLC classifier. Interesting results are reported to indicate how the combined classifier may work under various conditions.
Citations
More filters
18 Nov 2016
TL;DR: A novel generalized deep transfer networks (DTNs) capable of transferring label information across heterogeneous domains, textual domain to visual domain, and to share the labels between two domains are proposed, able to generate domain-specific and shared interdomain features.
Abstract: In recent years, deep neural networks have been successfully applied to model visual concepts and have achieved competitive performance on many tasks. Despite their impressive performance, traditional deep networks are subjected to the decayed performance under the condition of lacking sufficient training data. This problem becomes extremely severe for deep networks trained on a very small dataset, making them overfitting by capturing nonessential or noisy information in the training set. Toward this end, we propose a novel generalized deep transfer networks (DTNs), capable of transferring label information across heterogeneous domains, textual domain to visual domain. The proposed framework has the ability to adequately mitigate the problem of insufficient training images by bringing in rich labels from the textual domain. Specifically, to share the labels between two domains, we build parameter- and representation-shared layers. They are able to generate domain-specific and shared interdomain features, making this architecture flexible and powerful in capturing complex information from different domains jointly. To evaluate the proposed method, we release a new dataset extended from NUS-WIDE at http://imag.njust.edu.cn/NUS-WIDE-128.html. Experimental results on this dataset show the superior performance of the proposed DTNs compared to existing state-of-the-art methods.
137 citations
Cites methods from "Combining MLC and SVM classifiers f..."
...(1) SVM: SVM is the conventional shallow structured classifier [Zhang et al. 2015a] and is set as the baseline for comparisons....
[...]
...We compare the proposed generalized DTNs (sig-tDTNs and duft-tDTNs) to the following methods: (1) SVM: SVM is the conventional shallow structured classifier [Zhang et al. 2015a] and is set as the baseline for comparisons....
[...]
TL;DR: Experimental results show that the proposed model outperforms five other state-of-the-art video saliency detection approaches and the proposed framework is found useful for other video content based applications such as video highlights.
130 citations
Cites methods from "Combining MLC and SVM classifiers f..."
...For regression purpose, a linear SVM is adopted for its simplicity and effectiveness[23][102]....
[...]
TL;DR: The validation test on UCI data sets demonstrates that for imbalanced medical data, the proposed method enhanced the overall performance of the classifier while producing high accuracy in identifying both majority and minority class.
Abstract: The classification in class imbalanced data has drawn significant interest in medical application. Most existing methods are prone to categorize the samples into the majority class, resulting in bias, in particular the insufficient identification of minority class. A kind of novel approach, class weights random forest is introduced to address the problem, by assigning individual weights for each class instead of a single weight. The validation test on UCI data sets demonstrates that for imbalanced medical data, the proposed method enhanced the overall performance of the classifier while producing high accuracy in identifying both majority and minority class.
128 citations
Cites methods from "Combining MLC and SVM classifiers f..."
...There were many popular algorithms concerning about Classifier Combination; such as Bayesian [41], [42], Dempster–Shafer [43]–[47], Fuzzy Integral [48], [49], and Voting Methods [50]–[57]....
[...]
TL;DR: The conclusion is that both decision-level and pixel-level fusion approaches produced comparable classification results, and either of the procedures can be adopted in areas with inescapable cloud problems for updating crop inventories and acreage estimation at regional scales.
Abstract: Crops mapping unequivocally becomes a daunting task in humid, tropical, or subtropical regions due to unattainability of adequate cloud-free optical imagery. Objective of this study is to evaluate the comparative performance between decision- and pixel-levels data fusion ensemble classified maps using Landsat 8, Landsat 7, and Sentinel-2 data. This research implements parallel and concatenation approach to ensemble classify the images. The multiclassifier system comprises of Maximum Likelihood, Support Vector Machines, and Spectral Information Divergence as base classifiers. Decision-level fusion is achieved by implementing plurality voting method. Pixel-level fusion is achieved by implementing fusion by mosaicking approach, thus appending cloud-free pixels from either Sentinel-2 or Landsat 7. The comparison is based on the assessment of classification accuracy. Overall accuracy results show that decision-level fusion achieved an accuracy of 85.4%, whereas pixel-level fusion classification attained 82.5%, but their respective kappa coefficients of 0.84 and 0.80 but are not significantly different according to Z-test at $\alpha = {\text{0.05}}$ . F1-score values reveal that decision-level performed better on most individual classes than pixel-level. Regression coefficient between planted areas from both approaches is 0.99. However, Support Vector Machines performed the best of the three classifiers. The conclusion is that both decision-level and pixel-level fusion approaches produced comparable classification results. Therefore, either of the procedures can be adopted in areas with inescapable cloud problems for updating crop inventories and acreage estimation at regional scales. Future work can focus on performing more comparison tests on different areas, run tests using different multiclassifier systems, and use different imagery.
23 citations
Additional excerpts
...[56] and Szuster et al....
[...]
TL;DR: The results revealed that the supervised object-based NN approach using the visible and near-infrared bands of both satellite imagery produced the most homogenous and accurate map among the other methods.
Abstract: Impervious surface is mainly defined as any surface which water cannot infiltrate the soil. Due to the impact of urban impervious surfaces (UIS) on environmental issues, the amount of impervious surfaces has been recognized as the most significant index of environmental quality. Detection and analysis of impervious surfaces within a watershed is one of the developing areas of scientific interest. This study evaluates and compares the accuracy and performance of five classification algorithms—supervised object-based nearest neighbour (NN) classifier, supervised pixel-based maximum likelihood classifier (MLC), supervised pixel-based spectral angle mapper (SAM), band ratioing normalized difference built-up index (NDBI), and normalized difference impervious index (NDII)—in extracting urban impervious surfaces. Our first aim was to identify the most effective method for mapping UIS using Sentinel-2A and Landsat-8 satellite data. The second aim was to compare and reveal the efficiency of the spatial and spectral resolution of Sentinel-2A and Landsat-8 data in extracting UIS. The results revealed that the supervised object-based NN approach using the visible and near-infrared bands of both satellite imagery produced the most homogenous and accurate map among the other methods. The object-based NN algorithm achieved an overall classification accuracy of 90.91% and 88.64%, and Kappa coefficient of 0.82 and 0.77 for Sentinel-2 and Landsat-8 images, respectively. The study also showed that the Sentinel-2 image yielded better results than the Landsat-8 pan-sharpened image in extracting detail and classification accuracy. Comparing these methods in the selected challenging study area can provide insight into the selection of the classification method for rapid and reliable extraction of UIS.
19 citations
Cites background or methods from "Combining MLC and SVM classifiers f..."
...…impact of urban impervious surfaces on environmental issues such as water and air pollution, flooding, and urban climate, the amount of impervious surfaces (IS) has been recognized as the most significant index of environmental quality (Arnold Jr and Gibbons 1996; Weng 2012; Zhang et al. 2015a)....
[...]
...…it is also reported that the distribution of IS plays a crucial role in estimating numerous socioeconomic factors such as urban development, population distribution and density, social conditions, and fluctuation of housing prices (Wu and Murray 2003; Yuan and Bauer 2007; Zhang et al. 2015a)....
[...]
...This algorithm is based on Bayesian theory in estimating parameters of a probabilistic model (Zhang et al. 2015b)....
[...]
...Nevertheless, accuratemapping of impervious surfaces using satellite passive sensor data has been a challenging task due to the diversity of urban land cover classes, where confusion often occurs between pervious and impervious surfaces (Weng 2012; Zhang et al. 2015a, 2016; Ma et al. 2017b)....
[...]
...A number of studies on the extraction of IS, including Slonecker et al. (2001), Bauer et al. (2005), Yuan and Bauer (2007), Weng (2012), Wang et al. (2015), Zhang et al. (2015a), and Wei and Blaschke (2018), have shown the effectiveness and reliability of remote sensing in the monitoring of UIS....
[...]
References
More filters
TL;DR: An introduction to the theoretical development of the SVM and an experimental evaluation of its accuracy, stability and training speed in deriving land cover classifications from satellite images are given.
Abstract: The support vector machine (SVM) is a group of theoretically superior machine learning algorithms. It was found competitive with the best available machine learning algorithms in classifying high-dimensional data sets. This paper gives an introduction to the theoretical development of the SVM and an experimental evaluation of its accuracy, stability and training speed in deriving land cover classifications from satellite images. The SVM was compared to three other popular classifiers, including the maximum likelihood classifier (MLC), neural network classifiers (NNC) and decision tree classifiers (DTC). The impacts of kernel configuration on the performance of the SVM and of the selection of training data and input variables on the four classifiers were also evaluated in this experiment.
1,580 citations
"Combining MLC and SVM classifiers f..." refers methods in this paper
...Taking the application in remote sensing for example, in Pal and Mather [12] and Huang et al [13], it is found SVM outperforms MLC and several other classifiers....
[...]
01 Jan 2010
TL;DR: In this paper, the augmented Lagrangian method is applied to solve the full-space integration problem which takes a block angular structure to resolve the non-separability issue in the augmented lagrangian relaxation, and a new decomposition strategy based on two-level optimization is proposed.
Abstract: To improve the quality of decision making in the process operations, it is essential to implement integrated planning and scheduling optimization Major challenge for the integration lies in that the corresponding optimization problem is generally hard to solve because of the intractable model size In this paper, augmented Lagrangian method is applied to solve the full-space integration problem which takes a block angular structure To resolve the non-separability issue in the augmented Lagrangian relaxation, we study the traditional method which approximates the cross-product term through linearization and also propose a new decomposition strategy based on two-level optimization The results from case study show that the augmented Lagrangian method is effective in solving the large integration problem and generating a feasible solution Furthermore, the proposed decomposition strategy based on two-level optimization can get better feasible solution than the traditional linearization method
1,032 citations
TL;DR: In this work, a versatile signal processing and analysis framework for Electroencephalogram (EEG) was proposed and a set of statistical features was extracted from the sub-bands to represent the distribution of wavelet coefficients.
Abstract: In this work, we proposed a versatile signal processing and analysis framework for Electroencephalogram (EEG). Within this framework the signals were decomposed into the frequency sub-bands using DWT and a set of statistical features was extracted from the sub-bands to represent the distribution of wavelet coefficients. Principal components analysis (PCA), independent components analysis (ICA) and linear discriminant analysis (LDA) is used to reduce the dimension of data. Then these features were used as an input to a support vector machine (SVM) with two discrete outputs: epileptic seizure or not. The performance of classification process due to different methods is presented and compared to show the excellent of classification process. These findings are presented as an example of a method for training, and testing a seizure prediction method on data from individual petit mal epileptic patients. Given the heterogeneity of epilepsy, it is likely that methods of this type will be required to configure intelligent devices for treating epilepsy to each individual's neurophysiology prior to clinical operation.
1,010 citations
"Combining MLC and SVM classifiers f..." refers background in this paper
...On the other hand, support vector machines (SVM) have attracted much increasing attention, which can be found in almost all areas when prediction and classification of signal are required, such as scour prediction on grade-control structure [7], fault diagnosis [8], EEG signal classification [9], and fire detection [10] as well as road sign detection and recognition [11]....
[...]
TL;DR: An improved algorithm that theoretically converges and avoids numerical difficulties is proposed for Platt’s probabilistic outputs for Support Vector Machines.
Abstract: Platt's probabilistic outputs for Support Vector Machines (Platt, J. in Smola, A., et al. (eds.) Advances in large margin classifiers. Cambridge, 2000) has been popular for applications that require posterior class probabilities. In this note, we propose an improved algorithm that theoretically converges and avoids numerical difficulties. A simple and ready-to-use pseudo code is included.
926 citations
"Combining MLC and SVM classifiers f..." refers background in this paper
...In addition, in Lin et al [27] Platt’s approach is further improved to avoid any numerical difficulty, i.e. overflow or underflow, in determining ip in cases BAgE iSVMi )(x is either too large or too small. otherwiseee Eife p ii i EE i E i 1 1 )1( 0)1( (24) Although there are significant differences between SVM and MLC, the probabilistic model above has uncovered the connection between these two classifiers....
[...]
...In addition, in Lin et al [27] Platt’s approach is further improved to avoid any numerical difficulty, i....
[...]
TL;DR: The MSVM is proposed, which extends the binary SVM to the multicategory case and has good theoretical properties, and an approximate leave-one-out cross-validation function is derived, analogous to the binary case.
Abstract: Two-category support vector machines (SVM) have been very popular in the machine learning community for classification problems. Solving multicategory problems by a series of binary classifiers is quite common in the SVM paradigm; however, this approach may fail under various circumstances. We propose the multicategory support vector machine (MSVM), which extends the binary SVM to the multicategory case and has good theoretical properties. The proposed method provides a unifying framework when there are either equal or unequal misclassification costs. As a tuning criterion for the MSVM, an approximate leave-one-out cross-validation function, called Generalized Approximate Cross Validation, is derived, analogous to the binary case. The effectiveness of the MSVM is demonstrated through the applications to cancer classification using microarray data and cloud classification with satellite radiance profiles.
767 citations