Open AccessProceedings Article
Framewise phoneme classification with bidirectional LSTM and other neural network architectures
Alex Graves,Jürgen Schmidhuber +1 more
- Vol. 18, pp 602-610
Reads0
Chats0
TLDR
In this article, a modified, full gradient version of the LSTM learning algorithm was used for framewise phoneme classification, using the TIMIT database, and the results support the view that contextual information is crucial to speech processing, and suggest that bidirectional networks outperform unidirectional ones.Abstract:
In this paper, we present bidirectional Long Short Term Memory (LSTM) networks, and a modified, full gradient version of the LSTM learning algorithm. We evaluate Bidirectional LSTM (BLSTM) and several other network architectures on the benchmark task of framewise phoneme classification, using the TIMIT database. Our main findings are that bidirectional networks outperform unidirectional ones, and Long Short Term Memory (LSTM) is much faster and also more accurate than both standard Recurrent Neural Nets (RNNs) and time-windowed Multilayer Perceptrons (MLPs). Our results support the view that contextual information is crucial to speech processing, and suggest that BLSTM is an effective architecture with which to exploit it'.read more
Citations
More filters
Journal ArticleDOI
Massive computational acceleration by using neural networks to emulate mechanism-based biological models
Shangying Wang,Kai Fan,Nan Luo,Yangxiaolu Cao,Feilun Wu,Carolyn Zhang,Katherine Heller,Lingchong You +7 more
TL;DR: It is shown that it is possible to explore the dynamics of a mechanistic model over a large set of parameters by training an artificial neural network on a smaller set of simulations.
Proceedings ArticleDOI
Gated Word-Character Recurrent Language Model
Yasumasa Miyamoto,Kyunghyun Cho +1 more
TL;DR: This paper introduced a gate that adaptively finds the optimal mixture of the character-level and word-level inputs to create the final vector representation of a word by combining two distinct representations of the word.
Journal ArticleDOI
Prediction of 3-D Ocean Temperature by Multilayer Convolutional LSTM
Kun Zhang,Xupu Geng,Xiao-Hai Yan +2 more
TL;DR: A model of multilayer convolutional long- and short-term memory (M-convLSTM) to predict 3-D ocean temperature is proposed, comprising Convolutional neural networks (CNNs), long-and-short- term memory (L STM), and multiple layer stacking to consider the horizontal and vertical temperature variations from sea surface to subsurface to be about 2000 m below.
Proceedings ArticleDOI
Learning temporal representation of transaction amount for fraudulent transaction recognition using CNN, Stacked LSTM, and CNN-LSTM
TL;DR: This study explores the effect of nonfraud to fraud sample ratio from 1 to 4 and three models: Convolutional Neural Network (CNN), Stacked Long Short-term Memory (SLSTM), and Hybrid of CNN-LSTM.
Journal ArticleDOI
A Feasibility of Respiration Prediction Based on Deep Bi-LSTM for Real-Time Tumor Tracking
TL;DR: The proposed seven-layer bidirectional long short term memory (Deep Bi-LSTM) and one output layer deep neural network is proposed to predict respiration motion for a latency of 400 ms and it is demonstrated that the prediction accuracy is about five times better than traditional autoregressive integrated moving average model and about three timesbetter than adaptive boosting and multi-layer perceptron neural network.
References
More filters
Journal ArticleDOI
Long short-term memory
TL;DR: A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
Book
Neural networks for pattern recognition
TL;DR: This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition, and is designed as a text, with over 100 exercises, to benefit anyone involved in the fields of neural computation and pattern recognition.
Journal ArticleDOI
Bidirectional recurrent neural networks
Mike Schuster,Kuldip K. Paliwal +1 more
TL;DR: It is shown how the proposed bidirectional structure can be easily modified to allow efficient estimation of the conditional posterior probability of complete symbol sequences without making any explicit assumption about the shape of the distribution.
Darpa Timit Acoustic-Phonetic Continuous Speech Corpus CD-ROM {TIMIT} | NIST
John S. Garofolo,Lori Lamel,W M. Fisher,Jonathan G. Fiscus,David S. Pallett,Nancy L. Dahlgren +5 more
Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies
Sepp Hochreiter,Yoshua Bengio +1 more
TL;DR: D3EGF(FIH)J KMLONPEGQSRPETN UCV.