Proceedings ArticleDOI
Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks
Alex Graves,Santiago Fernández,Faustino Gomez,Jürgen Schmidhuber +3 more
- pp 369-376
TLDR
This paper presents a novel method for training RNNs to label unsegmented sequences directly, thereby solving both problems of sequence learning and post-processing.Abstract:
Many real-world sequence learning tasks require the prediction of sequences of labels from noisy, unsegmented input data. In speech recognition, for example, an acoustic signal is transcribed into words or sub-word units. Recurrent neural networks (RNNs) are powerful sequence learners that would seem well suited to such tasks. However, because they require pre-segmented training data, and post-processing to transform their outputs into label sequences, their applicability has so far been limited. This paper presents a novel method for training RNNs to label unsegmented sequences directly, thereby solving both problems. An experiment on the TIMIT speech corpus demonstrates its advantages over both a baseline HMM and a hybrid HMM-RNN.read more
Citations
More filters
Proceedings ArticleDOI
A Comparative Study on Transformer vs RNN in Speech Applications
Shigeki Karita,Nanxin Chen,Tomoki Hayashi,Takaaki Hori,Hirofumi Inaguma,Ziyan Jiang,Masao Someki,Nelson Yalta,Ryuichi Yamamoto,Xiaofei Wang,Shinji Watanabe,Takenori Yoshimura,Wangyou Zhang +12 more
TL;DR: An emergent sequence-to-sequence model called Transformer achieves state-of-the-art performance in neural machine translation and other natural language processing applications, including the surprising superiority of Transformer in 13/15 ASR benchmarks in comparison with RNN.
Proceedings ArticleDOI
Improved Training of End-to-end Attention Models for Speech Recognition
TL;DR: In this article, a sequence-to-sequence attention-based model on subword units was proposed to achieve competitive results on the Switchboard 300h and LibriSpeech 1000h tasks.
Journal ArticleDOI
Light Gated Recurrent Units for Speech Recognition
TL;DR: This paper revise one of the most popular RNN models, namely, gated recurrent units (GRUs), and proposes a simplified architecture that turned out to be very effective for ASR, and proposes to replace hyperbolic tangent with rectified linear unit activations.
Journal ArticleDOI
A Deep Neural Framework for Continuous Sign Language Recognition by Iterative Training
TL;DR: This work develops a continuous sign language (SL) recognition framework with deep neural networks, which directly transcribes videos of SL sentences to sequences of ordered gloss labels, and proposed architecture adopts deep convolutional neural networks with stacked temporal fusion layers as the feature extraction module.
Journal ArticleDOI
Performance Evaluation of Deep Neural Networks Applied to Speech Recognition: RNN, LSTM and GRU
TL;DR: Evaluated RNN, LSTM, and GRU networks are evaluated to compare their performances on a reduced TED-LIUM speech data set and the results show that L STM achieves the best word error rates, however, the GRU optimization is faster while achieving worderror rates close to LSTm.
References
More filters
Journal ArticleDOI
Long short-term memory
TL;DR: A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
Journal ArticleDOI
A tutorial on hidden Markov models and selected applications in speech recognition
TL;DR: In this paper, the authors provide an overview of the basic theory of hidden Markov models (HMMs) as originated by L.E. Baum and T. Petrie (1966) and give practical details on methods of implementation of the theory along with a description of selected applications of HMMs to distinct problems in speech recognition.
Book
Neural networks for pattern recognition
TL;DR: This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition, and is designed as a text, with over 100 exercises, to benefit anyone involved in the fields of neural computation and pattern recognition.
Proceedings Article
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
TL;DR: This work presents iterative parameter estimation algorithms for conditional random fields and compares the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.