Combining MLC and SVM classifiers for learning based decision making: analysis and evaluations
01 Jan 2015-Computational Intelligence and Neuroscience (Hindawi Publishing Corporation)-Vol. 2015, pp 423581-423581
TL;DR: MLC and SVM are combined in learning and classification, which helps to yield probabilistic output for SVM and facilitate soft decision making and interesting results are reported to indicate how the combined classifier may work under various conditions.
Abstract: Maximum likelihood classifier (MLC) and support vector machines (SVM) are two commonly used approaches in machine learning. MLC is based on Bayesian theory in estimating parameters of a probabilistic model, whilst SVM is an optimization based nonparametric method in this context. Recently, it is found that SVM in some cases is equivalent to MLC in probabilistically modeling the learning process. In this paper, MLC and SVM are combined in learning and classification, which helps to yield probabilistic output for SVM and facilitate soft decision making. In total four groups of data are used for evaluations, covering sonar, vehicle, breast cancer, and DNA sequences.The data samples are characterized in terms of Gaussian/non-Gaussian distributed and balanced/unbalanced samples which are then further used for performance assessment in comparing the SVM and the combined SVM-MLC classifier. Interesting results are reported to indicate how the combined classifier may work under various conditions.
Citations
More filters
18 Nov 2016
TL;DR: A novel generalized deep transfer networks (DTNs) capable of transferring label information across heterogeneous domains, textual domain to visual domain, and to share the labels between two domains are proposed, able to generate domain-specific and shared interdomain features.
Abstract: In recent years, deep neural networks have been successfully applied to model visual concepts and have achieved competitive performance on many tasks. Despite their impressive performance, traditional deep networks are subjected to the decayed performance under the condition of lacking sufficient training data. This problem becomes extremely severe for deep networks trained on a very small dataset, making them overfitting by capturing nonessential or noisy information in the training set. Toward this end, we propose a novel generalized deep transfer networks (DTNs), capable of transferring label information across heterogeneous domains, textual domain to visual domain. The proposed framework has the ability to adequately mitigate the problem of insufficient training images by bringing in rich labels from the textual domain. Specifically, to share the labels between two domains, we build parameter- and representation-shared layers. They are able to generate domain-specific and shared interdomain features, making this architecture flexible and powerful in capturing complex information from different domains jointly. To evaluate the proposed method, we release a new dataset extended from NUS-WIDE at http://imag.njust.edu.cn/NUS-WIDE-128.html. Experimental results on this dataset show the superior performance of the proposed DTNs compared to existing state-of-the-art methods.
137 citations
Cites methods from "Combining MLC and SVM classifiers f..."
...(1) SVM: SVM is the conventional shallow structured classifier [Zhang et al. 2015a] and is set as the baseline for comparisons....
[...]
...We compare the proposed generalized DTNs (sig-tDTNs and duft-tDTNs) to the following methods: (1) SVM: SVM is the conventional shallow structured classifier [Zhang et al. 2015a] and is set as the baseline for comparisons....
[...]
TL;DR: Experimental results show that the proposed model outperforms five other state-of-the-art video saliency detection approaches and the proposed framework is found useful for other video content based applications such as video highlights.
130 citations
Cites methods from "Combining MLC and SVM classifiers f..."
...For regression purpose, a linear SVM is adopted for its simplicity and effectiveness[23][102]....
[...]
TL;DR: The validation test on UCI data sets demonstrates that for imbalanced medical data, the proposed method enhanced the overall performance of the classifier while producing high accuracy in identifying both majority and minority class.
Abstract: The classification in class imbalanced data has drawn significant interest in medical application. Most existing methods are prone to categorize the samples into the majority class, resulting in bias, in particular the insufficient identification of minority class. A kind of novel approach, class weights random forest is introduced to address the problem, by assigning individual weights for each class instead of a single weight. The validation test on UCI data sets demonstrates that for imbalanced medical data, the proposed method enhanced the overall performance of the classifier while producing high accuracy in identifying both majority and minority class.
128 citations
Cites methods from "Combining MLC and SVM classifiers f..."
...There were many popular algorithms concerning about Classifier Combination; such as Bayesian [41], [42], Dempster–Shafer [43]–[47], Fuzzy Integral [48], [49], and Voting Methods [50]–[57]....
[...]
TL;DR: The conclusion is that both decision-level and pixel-level fusion approaches produced comparable classification results, and either of the procedures can be adopted in areas with inescapable cloud problems for updating crop inventories and acreage estimation at regional scales.
Abstract: Crops mapping unequivocally becomes a daunting task in humid, tropical, or subtropical regions due to unattainability of adequate cloud-free optical imagery. Objective of this study is to evaluate the comparative performance between decision- and pixel-levels data fusion ensemble classified maps using Landsat 8, Landsat 7, and Sentinel-2 data. This research implements parallel and concatenation approach to ensemble classify the images. The multiclassifier system comprises of Maximum Likelihood, Support Vector Machines, and Spectral Information Divergence as base classifiers. Decision-level fusion is achieved by implementing plurality voting method. Pixel-level fusion is achieved by implementing fusion by mosaicking approach, thus appending cloud-free pixels from either Sentinel-2 or Landsat 7. The comparison is based on the assessment of classification accuracy. Overall accuracy results show that decision-level fusion achieved an accuracy of 85.4%, whereas pixel-level fusion classification attained 82.5%, but their respective kappa coefficients of 0.84 and 0.80 but are not significantly different according to Z-test at $\alpha = {\text{0.05}}$ . F1-score values reveal that decision-level performed better on most individual classes than pixel-level. Regression coefficient between planted areas from both approaches is 0.99. However, Support Vector Machines performed the best of the three classifiers. The conclusion is that both decision-level and pixel-level fusion approaches produced comparable classification results. Therefore, either of the procedures can be adopted in areas with inescapable cloud problems for updating crop inventories and acreage estimation at regional scales. Future work can focus on performing more comparison tests on different areas, run tests using different multiclassifier systems, and use different imagery.
23 citations
Additional excerpts
...[56] and Szuster et al....
[...]
TL;DR: The results revealed that the supervised object-based NN approach using the visible and near-infrared bands of both satellite imagery produced the most homogenous and accurate map among the other methods.
Abstract: Impervious surface is mainly defined as any surface which water cannot infiltrate the soil. Due to the impact of urban impervious surfaces (UIS) on environmental issues, the amount of impervious surfaces has been recognized as the most significant index of environmental quality. Detection and analysis of impervious surfaces within a watershed is one of the developing areas of scientific interest. This study evaluates and compares the accuracy and performance of five classification algorithms—supervised object-based nearest neighbour (NN) classifier, supervised pixel-based maximum likelihood classifier (MLC), supervised pixel-based spectral angle mapper (SAM), band ratioing normalized difference built-up index (NDBI), and normalized difference impervious index (NDII)—in extracting urban impervious surfaces. Our first aim was to identify the most effective method for mapping UIS using Sentinel-2A and Landsat-8 satellite data. The second aim was to compare and reveal the efficiency of the spatial and spectral resolution of Sentinel-2A and Landsat-8 data in extracting UIS. The results revealed that the supervised object-based NN approach using the visible and near-infrared bands of both satellite imagery produced the most homogenous and accurate map among the other methods. The object-based NN algorithm achieved an overall classification accuracy of 90.91% and 88.64%, and Kappa coefficient of 0.82 and 0.77 for Sentinel-2 and Landsat-8 images, respectively. The study also showed that the Sentinel-2 image yielded better results than the Landsat-8 pan-sharpened image in extracting detail and classification accuracy. Comparing these methods in the selected challenging study area can provide insight into the selection of the classification method for rapid and reliable extraction of UIS.
19 citations
Cites background or methods from "Combining MLC and SVM classifiers f..."
...…impact of urban impervious surfaces on environmental issues such as water and air pollution, flooding, and urban climate, the amount of impervious surfaces (IS) has been recognized as the most significant index of environmental quality (Arnold Jr and Gibbons 1996; Weng 2012; Zhang et al. 2015a)....
[...]
...…it is also reported that the distribution of IS plays a crucial role in estimating numerous socioeconomic factors such as urban development, population distribution and density, social conditions, and fluctuation of housing prices (Wu and Murray 2003; Yuan and Bauer 2007; Zhang et al. 2015a)....
[...]
...This algorithm is based on Bayesian theory in estimating parameters of a probabilistic model (Zhang et al. 2015b)....
[...]
...Nevertheless, accuratemapping of impervious surfaces using satellite passive sensor data has been a challenging task due to the diversity of urban land cover classes, where confusion often occurs between pervious and impervious surfaces (Weng 2012; Zhang et al. 2015a, 2016; Ma et al. 2017b)....
[...]
...A number of studies on the extraction of IS, including Slonecker et al. (2001), Bauer et al. (2005), Yuan and Bauer (2007), Weng (2012), Wang et al. (2015), Zhang et al. (2015a), and Wei and Blaschke (2018), have shown the effectiveness and reliability of remote sensing in the monitoring of UIS....
[...]
References
More filters
TL;DR: A focused literature survey on recent neural network developments in computer-aided diagnosis, medical image segmentation and edge detection towards visual content analysis, and medical image registration for its pre-processing and post-processing is provided to increase awareness of how neural networks can be applied to these areas.
306 citations
"Combining MLC and SVM classifiers f..." refers methods in this paper
...Future work will focus on combining other classifiers such as neural network for applications in medical imaging [31–33] and recognition and classification tasks [34, 35]....
[...]
TL;DR: Although ANN outperforms SVM when balanced learning is absent, the performance from the two classifiers becomes very comparable when both balanced learning and optimized decision making are employed and this has fully validated the effectiveness of the proposed method for the successful classification of clustered microcalcifications.
Abstract: Classification of microcalcification clusters from mammograms plays essential roles in computer-aided diagnosis for early detection of breast cancer, where support vector machine (SVM) and artificial neural network (ANN) are two commonly used techniques. Although some work suggest that SVM performs better than ANN, the average accuracy achieved is only around 80% in terms of the area under the receiver operating characteristic curve Az. This performance may become much worse when the training samples are imbalanced. As a result, a new strategy namely balanced learning with optimized decision making is proposed to enable effective learning from imbalanced samples, which is further employed to evaluate the performance of ANN and SVM in this context. When the proposed learning strategy is applied to individual classifiers, the results on the DDSM database have demonstrated that the performance from both ANN and SVM has been significantly improved. Although ANN outperforms SVM when balanced learning is absent, the performance from the two classifiers becomes very comparable when both balanced learning and optimized decision making are employed. Consequently, an average improvement of more than 10% in the measurements of F"1 score and Az measurement are achieved for the two classifiers. This has fully validated the effectiveness of our proposed method for the successful classification of clustered microcalcifications.
232 citations
TL;DR: Results show that the SVM performs better than maximum likelihood, univariate decision tree and backpropagation neural network classifiers, even with small training data sets, and is almost unaffected by the Hughes phenomenon.
216 citations
"Combining MLC and SVM classifiers f..." refers methods in this paper
...Taking the application in remote sensing, for example, in Pal and Mather [12] and Huang et al....
[...]
...Taking the application in remote sensing for example, in Pal and Mather [12] and Huang et al [13], it is found SVM outperforms MLC and several other classifiers....
[...]
...[12] Pal, M., Mather, P.M., Assessment of the effectiveness of support vector machines for hyperspectral data, Future Generation Computer Systems, 20(7):...
[...]
TL;DR: In this article, the performance of the support vector machine image classification technique in the context of a tropical coastal zone exhibiting low to medium scale development was compared to the maximum likelihood classifier and the artificial neural network techniques.
168 citations
"Combining MLC and SVM classifiers f..." refers result in this paper
...In Szuster et al [15], SVM only yields slightly better results than MLC for land cover analysis....
[...]
TL;DR: The study shows that the predicted results using the estimated model from LS-SVM are good agreement with the actual test results, and Bayesian framework is also applied to infer the hyperparameters used in LS- SVM so as to eliminate the work of cross-validation.
131 citations