Proceedings ArticleDOI
Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks
Alex Graves,Santiago Fernández,Faustino Gomez,Jürgen Schmidhuber +3 more
- pp 369-376
TLDR
This paper presents a novel method for training RNNs to label unsegmented sequences directly, thereby solving both problems of sequence learning and post-processing.Abstract:
Many real-world sequence learning tasks require the prediction of sequences of labels from noisy, unsegmented input data. In speech recognition, for example, an acoustic signal is transcribed into words or sub-word units. Recurrent neural networks (RNNs) are powerful sequence learners that would seem well suited to such tasks. However, because they require pre-segmented training data, and post-processing to transform their outputs into label sequences, their applicability has so far been limited. This paper presents a novel method for training RNNs to label unsegmented sequences directly, thereby solving both problems. An experiment on the TIMIT speech corpus demonstrates its advantages over both a baseline HMM and a hybrid HMM-RNN.read more
Citations
More filters
Proceedings ArticleDOI
Text Recognition in Images Based on Transformer with Hierarchical Attention
TL;DR: A new Transformer-like structure for text recognition in images, referred to as the Hierarchical Attention Transformer Network (HATN), which can be trained end-to-end by using only images and sentence-level annotations.
Book ChapterDOI
2006: celebrating 75 years of AI - history and outlook: the next 25 years
TL;DR: A look back at important milestones of AI history, mention essential recent results, and speculate about what to expect from the next 25 years, emphasizing the significance of the ongoing dramatic hardware speedups, and discussing Godel-inspired, self-referential,Self-improving universal problem solvers.
Proceedings ArticleDOI
Term extraction via neural sequence labeling a comparative evaluation of strategies using recurrent neural networks
TL;DR: This paper built a system that identifies terms via directly performing sequence-labeling with a BILOU scheme on word sequences and investigated which network types and topologies are best suited when applying the authors' term extraction systems to other domains than that of the training data of the networks.
Journal ArticleDOI
CTC regularized model adaptation for improving LSTM RNN based multi-accent Mandarin speech recognition
TL;DR: A novel regularized adaptation method to improve the performance of multi-accent Mandarin speech recognition task by forces the conditional probability distribution estimated from the adapted model to be close to the accent independent model.
Proceedings ArticleDOI
Curriculum Learning for Handwritten Text Line Recognition
TL;DR: This article proposed to first learn to recognize short sequences before training on all available training sequences, which can significantly speed up the training of RNN for text recognition, and even significantly improve performance in some cases.
References
More filters
Journal ArticleDOI
Long short-term memory
TL;DR: A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
Journal ArticleDOI
A tutorial on hidden Markov models and selected applications in speech recognition
TL;DR: In this paper, the authors provide an overview of the basic theory of hidden Markov models (HMMs) as originated by L.E. Baum and T. Petrie (1966) and give practical details on methods of implementation of the theory along with a description of selected applications of HMMs to distinct problems in speech recognition.
Book
Neural networks for pattern recognition
TL;DR: This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition, and is designed as a text, with over 100 exercises, to benefit anyone involved in the fields of neural computation and pattern recognition.
Proceedings Article
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
TL;DR: This work presents iterative parameter estimation algorithms for conditional random fields and compares the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.