Combining MLC and SVM classifiers for learning based decision making: analysis and evaluations
01 Jan 2015-Computational Intelligence and Neuroscience (Hindawi Publishing Corporation)-Vol. 2015, pp 423581-423581
TL;DR: MLC and SVM are combined in learning and classification, which helps to yield probabilistic output for SVM and facilitate soft decision making and interesting results are reported to indicate how the combined classifier may work under various conditions.
Abstract: Maximum likelihood classifier (MLC) and support vector machines (SVM) are two commonly used approaches in machine learning. MLC is based on Bayesian theory in estimating parameters of a probabilistic model, whilst SVM is an optimization based nonparametric method in this context. Recently, it is found that SVM in some cases is equivalent to MLC in probabilistically modeling the learning process. In this paper, MLC and SVM are combined in learning and classification, which helps to yield probabilistic output for SVM and facilitate soft decision making. In total four groups of data are used for evaluations, covering sonar, vehicle, breast cancer, and DNA sequences.The data samples are characterized in terms of Gaussian/non-Gaussian distributed and balanced/unbalanced samples which are then further used for performance assessment in comparing the SVM and the combined SVM-MLC classifier. Interesting results are reported to indicate how the combined classifier may work under various conditions.
Citations
More filters
18 Nov 2016
TL;DR: A novel generalized deep transfer networks (DTNs) capable of transferring label information across heterogeneous domains, textual domain to visual domain, and to share the labels between two domains are proposed, able to generate domain-specific and shared interdomain features.
Abstract: In recent years, deep neural networks have been successfully applied to model visual concepts and have achieved competitive performance on many tasks. Despite their impressive performance, traditional deep networks are subjected to the decayed performance under the condition of lacking sufficient training data. This problem becomes extremely severe for deep networks trained on a very small dataset, making them overfitting by capturing nonessential or noisy information in the training set. Toward this end, we propose a novel generalized deep transfer networks (DTNs), capable of transferring label information across heterogeneous domains, textual domain to visual domain. The proposed framework has the ability to adequately mitigate the problem of insufficient training images by bringing in rich labels from the textual domain. Specifically, to share the labels between two domains, we build parameter- and representation-shared layers. They are able to generate domain-specific and shared interdomain features, making this architecture flexible and powerful in capturing complex information from different domains jointly. To evaluate the proposed method, we release a new dataset extended from NUS-WIDE at http://imag.njust.edu.cn/NUS-WIDE-128.html. Experimental results on this dataset show the superior performance of the proposed DTNs compared to existing state-of-the-art methods.
137 citations
Cites methods from "Combining MLC and SVM classifiers f..."
...(1) SVM: SVM is the conventional shallow structured classifier [Zhang et al. 2015a] and is set as the baseline for comparisons....
[...]
...We compare the proposed generalized DTNs (sig-tDTNs and duft-tDTNs) to the following methods: (1) SVM: SVM is the conventional shallow structured classifier [Zhang et al. 2015a] and is set as the baseline for comparisons....
[...]
TL;DR: Experimental results show that the proposed model outperforms five other state-of-the-art video saliency detection approaches and the proposed framework is found useful for other video content based applications such as video highlights.
130 citations
Cites methods from "Combining MLC and SVM classifiers f..."
...For regression purpose, a linear SVM is adopted for its simplicity and effectiveness[23][102]....
[...]
TL;DR: The validation test on UCI data sets demonstrates that for imbalanced medical data, the proposed method enhanced the overall performance of the classifier while producing high accuracy in identifying both majority and minority class.
Abstract: The classification in class imbalanced data has drawn significant interest in medical application. Most existing methods are prone to categorize the samples into the majority class, resulting in bias, in particular the insufficient identification of minority class. A kind of novel approach, class weights random forest is introduced to address the problem, by assigning individual weights for each class instead of a single weight. The validation test on UCI data sets demonstrates that for imbalanced medical data, the proposed method enhanced the overall performance of the classifier while producing high accuracy in identifying both majority and minority class.
128 citations
Cites methods from "Combining MLC and SVM classifiers f..."
...There were many popular algorithms concerning about Classifier Combination; such as Bayesian [41], [42], Dempster–Shafer [43]–[47], Fuzzy Integral [48], [49], and Voting Methods [50]–[57]....
[...]
TL;DR: The conclusion is that both decision-level and pixel-level fusion approaches produced comparable classification results, and either of the procedures can be adopted in areas with inescapable cloud problems for updating crop inventories and acreage estimation at regional scales.
Abstract: Crops mapping unequivocally becomes a daunting task in humid, tropical, or subtropical regions due to unattainability of adequate cloud-free optical imagery. Objective of this study is to evaluate the comparative performance between decision- and pixel-levels data fusion ensemble classified maps using Landsat 8, Landsat 7, and Sentinel-2 data. This research implements parallel and concatenation approach to ensemble classify the images. The multiclassifier system comprises of Maximum Likelihood, Support Vector Machines, and Spectral Information Divergence as base classifiers. Decision-level fusion is achieved by implementing plurality voting method. Pixel-level fusion is achieved by implementing fusion by mosaicking approach, thus appending cloud-free pixels from either Sentinel-2 or Landsat 7. The comparison is based on the assessment of classification accuracy. Overall accuracy results show that decision-level fusion achieved an accuracy of 85.4%, whereas pixel-level fusion classification attained 82.5%, but their respective kappa coefficients of 0.84 and 0.80 but are not significantly different according to Z-test at $\alpha = {\text{0.05}}$ . F1-score values reveal that decision-level performed better on most individual classes than pixel-level. Regression coefficient between planted areas from both approaches is 0.99. However, Support Vector Machines performed the best of the three classifiers. The conclusion is that both decision-level and pixel-level fusion approaches produced comparable classification results. Therefore, either of the procedures can be adopted in areas with inescapable cloud problems for updating crop inventories and acreage estimation at regional scales. Future work can focus on performing more comparison tests on different areas, run tests using different multiclassifier systems, and use different imagery.
23 citations
Additional excerpts
...[56] and Szuster et al....
[...]
TL;DR: The results revealed that the supervised object-based NN approach using the visible and near-infrared bands of both satellite imagery produced the most homogenous and accurate map among the other methods.
Abstract: Impervious surface is mainly defined as any surface which water cannot infiltrate the soil. Due to the impact of urban impervious surfaces (UIS) on environmental issues, the amount of impervious surfaces has been recognized as the most significant index of environmental quality. Detection and analysis of impervious surfaces within a watershed is one of the developing areas of scientific interest. This study evaluates and compares the accuracy and performance of five classification algorithms—supervised object-based nearest neighbour (NN) classifier, supervised pixel-based maximum likelihood classifier (MLC), supervised pixel-based spectral angle mapper (SAM), band ratioing normalized difference built-up index (NDBI), and normalized difference impervious index (NDII)—in extracting urban impervious surfaces. Our first aim was to identify the most effective method for mapping UIS using Sentinel-2A and Landsat-8 satellite data. The second aim was to compare and reveal the efficiency of the spatial and spectral resolution of Sentinel-2A and Landsat-8 data in extracting UIS. The results revealed that the supervised object-based NN approach using the visible and near-infrared bands of both satellite imagery produced the most homogenous and accurate map among the other methods. The object-based NN algorithm achieved an overall classification accuracy of 90.91% and 88.64%, and Kappa coefficient of 0.82 and 0.77 for Sentinel-2 and Landsat-8 images, respectively. The study also showed that the Sentinel-2 image yielded better results than the Landsat-8 pan-sharpened image in extracting detail and classification accuracy. Comparing these methods in the selected challenging study area can provide insight into the selection of the classification method for rapid and reliable extraction of UIS.
19 citations
Cites background or methods from "Combining MLC and SVM classifiers f..."
...…impact of urban impervious surfaces on environmental issues such as water and air pollution, flooding, and urban climate, the amount of impervious surfaces (IS) has been recognized as the most significant index of environmental quality (Arnold Jr and Gibbons 1996; Weng 2012; Zhang et al. 2015a)....
[...]
...…it is also reported that the distribution of IS plays a crucial role in estimating numerous socioeconomic factors such as urban development, population distribution and density, social conditions, and fluctuation of housing prices (Wu and Murray 2003; Yuan and Bauer 2007; Zhang et al. 2015a)....
[...]
...This algorithm is based on Bayesian theory in estimating parameters of a probabilistic model (Zhang et al. 2015b)....
[...]
...Nevertheless, accuratemapping of impervious surfaces using satellite passive sensor data has been a challenging task due to the diversity of urban land cover classes, where confusion often occurs between pervious and impervious surfaces (Weng 2012; Zhang et al. 2015a, 2016; Ma et al. 2017b)....
[...]
...A number of studies on the extraction of IS, including Slonecker et al. (2001), Bauer et al. (2005), Yuan and Bauer (2007), Weng (2012), Wang et al. (2015), Zhang et al. (2015a), and Wei and Blaschke (2018), have shown the effectiveness and reliability of remote sensing in the monitoring of UIS....
[...]
References
More filters
TL;DR: A pattern recognition method for off-line estimation of both L/H and H/L transition times in JET, based on a combined classifier to identify the confinement regime (L or H) at any time instant during a discharge.
Abstract: This paper describes a pattern recognition method for off-line estimation of both L/H and H/L transition times in JET. The technique is based on a combined classifier to identify the confinement regime (L or H) at any time instant during a discharge. The classifier is a combination of two different classification systems: a Bayesian classifier whose likelihood is computed by means of a non-parametric statistical classifier (Parzen window) and a support vector machine classifier. They are combined through a fuzzy aggregation operator, in particular the Einstein sum. The success rate achieved exceeds 99% for the L to H transition and 96% for the H to L transition. The estimation of transition times is accomplished by following the temporal evolution of the confinement regimes.
30 citations
"Combining MLC and SVM classifiers f..." refers methods in this paper
...[18], Bayesian statistics are combined with SVM for parameter optimization....
[...]
TL;DR: It is found that the neural classifier outperforms the maximum likelihood classifier in this context, and is applied for pixel-level classification, followed by modal-filter based post-processing for robustness.
Abstract: Maximum likelihood and neural classifiers are two typical techniques in image classification. This paper investigates how to adapt these approaches to hyperspectral imaging for the classification of five kinds of Chinese tea samples, using visible light hyperspectral spectroscopy rather than near-infrared. After removal of unnecessary parts from each imaged tea sample using a morphological cropper, principal component analysis is employed for feature extraction. The two classifiers are then respectively applied for pixel-level classification, followed by modal-filter based post-processing for robustness. Although the samples look similar to the naked eye, promising results are reported and analysed in these comprehensive experiments. In addition, it is found that the neural classifier outperforms the maximum likelihood classifier in this context.
28 citations
"Combining MLC and SVM classifiers f..." refers background in this paper
...Future work will focus on combining other classifiers such as neural network for applications in medical imaging [31-33] and recognition and classification tasks [34-35]....
[...]
TL;DR: A method with a model assessment of the interplay between various classification decisions using probability, corresponding decision costs, and quadratic program of optimal margin classifier called: Bayesian Support Vector Machines (BSVMs) learning strategy is proposed.
Abstract: Research highlights? In imbalanced problems, we propose a method based on assessing the interplay between various classification decisions using probability, corresponding decision costs, and optimal margin. Such a more structured model specially designs to overcome the imbalanced problems. Classification approaches usually present the poor generalization performance with an apparent class imbalance problem. Surely, a measures of the quality of the possible models reflected the remaining uncertainty in the class imbalance on learning. The purpose of our learning method is to lead an attractive pragmatic expansion scheme of the Bayesian approach to assess how well it is aligned with the class imbalance problem. Thus, we propose a method with a model assessment of the interplay between various classification decisions using probability, corresponding decision costs, and quadratic program of optimal margin classifier called: Bayesian Support Vector Machines (BSVMs) learning strategy. In the framework, we did modify in the objects and conditions of primal problem to reproduce an appropriate learning rule for an observation sample. The experiments on several existing data sets showed that BSVMs may appropriately capture the true relationship between the inputs and outputs by experimental evidence.
22 citations
TL;DR: In this article, a model-based clustering using the classification maximum-likelihood approach was used to find clusters in the principal component (PC) scores, which indicated that there are three spherical clusters of variable volume in the chemical data.
20 citations
"Combining MLC and SVM classifiers f..." refers background in this paper
...Maximum likelihood classification (MLC) is one of the most commonly used approaches in signal classification and identification, which has been successfully applied in a wide range of engineering applications including classification for digital amplitude-phase modulations [1], remote sensing [2], genes selection for tissue classification [3], nonnative speech recognition [4], chemical analysis in archaeological applications [5], and speaker recognition [6]....
[...]
TL;DR: An improved method of model complexity selection for nonnative speech recognition is proposed by using maximum a posteriori (MAP) estimation of bias distributions, and an automatic accent classification algorithm is also proposed for integration with dynamic model selection and adaptation.
3 citations