A novel deep learning by combining discriminative model with generative model
12 Jul 2015-pp 1-6
TL;DR: Experimental results show that the proposed deep structure model allows for an easier classification of the uncertain data through multiple-layer training and it gives more accurate results.
Abstract: Deep learning methods allow a classifier to learn features automatically through multiple layers of training. In a deep learning process, low-level features are abstracted into high-level features. In this paper, we propose a new probabilistic deep learning method that combines a discriminative model, namely, Support Vector Machine (SVM), with a generative model, namely, Gaussian Mixture Model (GMM). Combining the SVM with the GMM, we can represent a new input feature for deeper layer training of uncertain data in current layer construction. Bayesian rule is used to re-represent the output data of the previous layer of the SVM with GMM to serve as the input data for the next deep layer. As a result, deep features are reliably extracted without additional feature extraction efforts, using multiple layers of the SVM with GMM. Experimental results show that the proposed deep structure model allows for an easier classification of the uncertain data through multiple-layer training and it gives more accurate results.
Citations
More filters
TL;DR: This paper designed and simulated a smart city model and suggests solution for privacy issues which are to be considered at top priority to ensure secrecy and privacy of smart city residents.
Abstract: With the recent technological development, there is prevalent trend for smart infrastructure deployment with intention to provide smart services for inhabitants. City governments of current era are under huge pressure to facilitate their residents by offering state of the art services equipped with modern technology gadgets. To achieve this goal they have been forced for massive investment in IT infrastructure deployment, eventually they are collecting huge amount of data from users with intention of providing them better or improved services. These services are very exciting but on the other side they also pose a big threat to the privacy of individuals. This paper designed and simulated a smart city model. This model is connected with some mandatory communication devices which also produce data for different sensors, Based on simulation results and possible threats for alteration of this data, it suggests solution for privacy issues which are to be considered at top priority to ensure secrecy and privacy of smart city residents.
19 citations
Cites methods from "A novel deep learning by combining ..."
...This research intends to introduce a secure framework of Smart City infrastructure, where user data could be taken by network model which will be extracted by deep learning algorithm [22]....
[...]
01 Jul 2016
TL;DR: Assessment of the effectiveness of cross- company (CC) and within-company (WC) projects in STE estimation indicates that the application of the MILP framework yielded similar results for both WC and CC modeling.
Abstract: Software Testing Effort (STE), which contributes about 25-40% of the total development effort, plays a significant role in software development. In addressing the issues faced by companies in finding relevant datasets for STE estimation modeling prior to development, cross-company modeling could be leveraged. The study aims at assessing the effectiveness of cross-company (CC) and within-company (WC) projects in STE estimation. A robust multi-objective Mixed-Integer Linear Programming (MILP) optimization framework for the selection of CC and WC projects was constructed and estimation of STE was done using Deep Neural Networks. Results from our study indicate that the application of the MILP framework yielded similar results for both WC and CC modeling. The modeling framework will serve as a foundation to assist in STE estimation prior to the development of new a software project.
10 citations
Cites methods from "A novel deep learning by combining ..."
...DNN was considered for the within-company (WC) and crosscompany (CC) modeling approach since it makes use of multiple layers to automatically learn from a set of features and gives better predictive results [8]....
[...]
01 Jul 2016
TL;DR: A new content-based image retrieval system that can solve the object and scene recognition problems and categorize similar images is proposed using growing fuzzy topology adaptive resonant theory (GFTART) network and scene understanding using GIST.
Abstract: An image retrieval system is a technique for browsing, searching and retrieving images from a big database of digital images. In this paper, we propose a new content-based image retrieval system that can solve the object and scene recognition problems and categorize similar images. The proposed model consists of a deep structure support vector machine with Gaussian mixture model, which is combined with human-like top-down selective attention model using growing fuzzy topology adaptive resonant theory (GFTART) network and scene understanding using GIST. The results suggest that the proposed model has better performance than other recent methods used in this field.
2 citations
Cites background or methods from "A novel deep learning by combining ..."
...In this section, a layered structure is proposed [7], where each layer consists of discriminative and generative models....
[...]
...A deep structure support vector machine (SVM) with Gaussian mixture model (GMM) [7] is applied to find the class of the attended parts in query and candidate images....
[...]
01 Jan 2021
TL;DR: This research focuses on applying the deep learning methods to educational data for classification and prediction and shows that the proposed model SVM-LCDBN gives better performance as compared to simple hybrid model and hybrid model with traditional cuckoo search.
Abstract: The goodness measure of any institute lies in minimising the dropouts and targeting good placements. So, predicting students' performance is very interesting and an important task for educational information systems. Machine learning and deep learning are the emerging areas that truly entice more research practices. This research focuses on applying the deep learning methods to educational data for classification and prediction. The educational data of students from engineering domain with cognitive and non-cognitive parameters is considered. The hybrid model with support vector machine (SVM) and deep belief network (DBN) is devised. The SVM predicts class labels from preprocessed data. These class labels and actual class labels act as input to the DBN to perform final classification. The hybrid model is further optimised using cuckoo search with levy flight. The results clearly show that the proposed model SVM-LCDBN gives better performance as compared to simple hybrid model and hybrid model with traditional cuckoo search.
1 citations
References
More filters
TL;DR: Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.
Abstract: LIBSVM is a library for Support Vector Machines (SVMs). We have been actively developing this package since the year 2000. The goal is to help users to easily apply SVM to their applications. LIBSVM has gained wide popularity in machine learning and many other areas. In this article, we present all implementation details of LIBSVM. Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.
40,826 citations
Journal Article•
TL;DR: It is shown that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.
Abstract: Deep neural nets with a large number of parameters are very powerful machine learning systems. However, overfitting is a serious problem in such networks. Large networks are also slow to use, making it difficult to deal with overfitting by combining the predictions of many different large neural nets at test time. Dropout is a technique for addressing this problem. The key idea is to randomly drop units (along with their connections) from the neural network during training. This prevents units from co-adapting too much. During training, dropout samples from an exponential number of different "thinned" networks. At test time, it is easy to approximate the effect of averaging the predictions of all these thinned networks by simply using a single unthinned network that has smaller weights. This significantly reduces overfitting and gives major improvements over other regularization methods. We show that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.
33,597 citations
"A novel deep learning by combining ..." refers methods in this paper
...The Dropout [6] and DropConnect [7] methods have been particularly proposed for artificial neural networks which have large-fully connected layer....
[...]
TL;DR: A fast, greedy algorithm is derived that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory.
Abstract: We show how to use "complementary priors" to eliminate the explaining-away effects that make inference difficult in densely connected belief nets that have many hidden layers. Using complementary priors, we derive a fast, greedy algorithm that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory. The fast, greedy algorithm is used to initialize a slower learning procedure that fine-tunes the weights using a contrastive version of the wake-sleep algorithm. After fine-tuning, a network with three hidden layers forms a very good generative model of the joint distribution of handwritten digit images and their labels. This generative model gives better digit classification than the best discriminative learning algorithms. The low-dimensional manifolds on which the digits lie are modeled by long ravines in the free-energy landscape of the top-level associative memory, and it is easy to explore these ravines by using the directed connections to display what the associative memory has in mind.
15,055 citations
TL;DR: A simple explanation of how and why the Gibbs sampler works is given and analytically establish its properties in a simple case and insight is provided for more complicated cases.
Abstract: Computer-intensive algorithms, such as the Gibbs sampler, have become increasingly popular statistical tools, both in applied and theoretical work. The properties of such algorithms, however, may sometimes not be obvious. Here we give a simple explanation of how and why the Gibbs sampler works. We analytically establish its properties in a simple case and provide insight for more complicated cases. There are also a number of examples.
2,656 citations
"A novel deep learning by combining ..." refers methods in this paper
...The RBM was initially developed by Smolensky in 1986 [4] and it was considered as a generative stochastic neural network, using Gibbs sampling [5], to model the probability distribution over its set of inputs....
[...]