Journal ArticleDOI
A fast learning algorithm for deep belief nets
Reads0
Chats0
TLDR
A fast, greedy algorithm is derived that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory.Abstract:
We show how to use "complementary priors" to eliminate the explaining-away effects that make inference difficult in densely connected belief nets that have many hidden layers. Using complementary priors, we derive a fast, greedy algorithm that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory. The fast, greedy algorithm is used to initialize a slower learning procedure that fine-tunes the weights using a contrastive version of the wake-sleep algorithm. After fine-tuning, a network with three hidden layers forms a very good generative model of the joint distribution of handwritten digit images and their labels. This generative model gives better digit classification than the best discriminative learning algorithms. The low-dimensional manifolds on which the digits lie are modeled by long ravines in the free-energy landscape of the top-level associative memory, and it is easy to explore these ravines by using the directed connections to display what the associative memory has in mind.read more
Citations
More filters
Journal ArticleDOI
Deep Learning: A Comprehensive Overview on Techniques, Taxonomy, Applications and Research Directions.
Iqbal H. Sarker,Iqbal H. Sarker +1 more
TL;DR: In this paper, the authors present a structured and comprehensive view on deep learning techniques including a taxonomy considering various types of real-world tasks like supervised or unsupervised, and point out ten potential aspects for future generation DL modeling with research directions.
Proceedings ArticleDOI
The NTT CHiME-3 system: Advances in speech enhancement and recognition for mobile multi-microphone devices
Takuya Yoshioka,Nobutaka Ito,Marc Delcroix,Atsunori Ogawa,Keisuke Kinoshita,Masakiyo Fujimoto,Chengzhu Yu,Wojciech J. Fabian,Miquel Espi,Takuya Higuchi,Shoko Araki,Tomohiro Nakatani +11 more
TL;DR: NTT's CHiME-3 system is described, which integrates advanced speech enhancement and recognition techniques, which achieves a 3.45% development error rate and a 5.83% evaluation error rate.
Journal ArticleDOI
Spectral–Spatial Unified Networks for Hyperspectral Image Classification
TL;DR: A band grouping-based long short-term memory model and a multiscale convolutional neural network are proposed as the spectral and spatial feature extractors, respectively, for the hyperspectral image (HSI) classification.
Journal ArticleDOI
A deep learning ensemble approach for crude oil price forecasting
Yang Zhao,Jianping Li,Lean Yu +2 more
TL;DR: The approach is tested against some competing approaches and shows superior forecasting ability that is statistically proved by three tests and is especially suitable for oil price forecasting.
Journal ArticleDOI
A Unifying Review of Deep and Shallow Anomaly Detection
Lukas Ruff,Jacob R. Kauffmann,Robert A. Vandermeulen,Grégoire Montavon,Wojciech Samek,Marius Kloft,Thomas G. Dietterich,Klaus-Robert Müller +7 more
TL;DR: Deep learning approaches to anomaly detection (AD) have recently improved the state of the art in detection performance on complex data sets, such as large collections of images or text as mentioned in this paper, and led to the introduction of a great variety of new methods.
References
More filters
Journal ArticleDOI
Gradient-based learning applied to document recognition
Yann LeCun,Léon Bottou,Léon Bottou,Yoshua Bengio,Yoshua Bengio,Yoshua Bengio,Patrick Haffner +6 more
TL;DR: In this article, a graph transformer network (GTN) is proposed for handwritten character recognition, which can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters.
Book
Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
TL;DR: Probabilistic Reasoning in Intelligent Systems as mentioned in this paper is a complete and accessible account of the theoretical foundations and computational methods that underlie plausible reasoning under uncertainty, and provides a coherent explication of probability as a language for reasoning with partial belief.
Journal ArticleDOI
Shape matching and object recognition using shape contexts
TL;DR: This paper presents work on computing shape models that are computationally fast and invariant basic transformations like translation, scaling and rotation, and proposes shape detection using a feature called shape context, which is descriptive of the shape of the object.
Journal ArticleDOI
Training products of experts by minimizing contrastive divergence
TL;DR: A product of experts (PoE) is an interesting candidate for a perceptual system in which rapid inference is vital and generation is unnecessary because it is hard even to approximate the derivatives of the renormalization term in the combination rule.
Proceedings ArticleDOI
Best practices for convolutional neural networks applied to visual document analysis
TL;DR: A set of concrete bestpractices that document analysis researchers can use to get good results with neural networks, including a simple "do-it-yourself" implementation of convolution with a flexible architecture suitable for many visual document problems.