From the Publisher:
Connectionist Speech Recognition: A Hybrid Approach describes the theory and implementation of a method to incorporate neural network approaches into state-of-the-art continuous speech recognition systems based on Hidden Markov Models (HMMs) to improve their performance. In this framework, neural networks (and in particular, multilayer perceptrons or MLPs) have been restricted to well-defined subtasks of the whole system, i.e., HMM emission probability estimation and feature extraction. The book describes a successful five year international collaboration between the authors. The lessons learned form a case study that demonstrates how hybrid systems can be developed to combine neural networks with more traditional statistical approaches. The book illustrates both the advantages and limitations of neural networks in the framework of a statistical system. Using standard databases and comparing with some conventional approaches, it is shown that MLP probability estimation can improve recognition performance. Other approaches are discussed, though there is no such unequivocal experimental result for these methods. Connectionist Speech Recognition: A Hybrid Approach is of use to anyone intending to use neural networks for speech recognition or within the framework provided by an existing successful statistical approach. This includes research and development groups working in the field of speech recognition, both with standard and neural network approaches, as well as other pattern recognition and/or neural network researchers. This book is also suitable as a text for advanced courses on neural networks or speech processing.

Connectionist Speech Recognition: A Hybrid Approach

A new method is developed using the hidden Markov model (HMM) based technique. To handle nongesture patterns, we introduce the concept of a threshold model that calculates the likelihood threshold of an input pattern and provides a confirmation mechanism for the provisionally matched gesture patterns. The threshold model is a weak model for all trained gestures in the sense that its likelihood is smaller than that of the dedicated gesture model for a given gesture. Consequently, the likelihood can be used as an adaptive threshold for selecting proper gesture model. It has, however, a large number of states and needs to be reduced because the threshold model is constructed by collecting the states of all gesture models in the system. To overcome this problem, the states with similar probability distributions are merged, utilizing the relative entropy measure. Experimental results show that the proposed method can successfully extract trained gestures from continuous hand motion with 93.14% reliability.

An HMM-based threshold model approach for gesture recognition

The performance of current speech recognition systems is far below that of humans. Neural nets offer the potential of providing massive parallelism, adaptation, and new algorithmic approaches to problems in speech recognition. Initial studies have demonstrated that multilayer networks with time delays can provide excellent discrimination between small sets of pre-segmented difficult-to-discriminate words, consonants, and vowels. Performance for these small vocabularies has often exceeded that of more conventional approaches. Physiological front ends have provided improved recognition accuracy in noise and a cochlea filter-bank that could be used in these front ends has been implemented using micro-power analog VLSI techniques. Techniques have been developed to scale networks up in size to handle larger vocabularies, to reduce training time, and to train nets with recurrent connections. Multilayer perceptron classifiers are being integrated into conventional continuous-speech recognizers. Neural net architectures have been developed to perform the computations required by vector quantizers, static pattern classifiers, and the Viterbi decoding algorithm. Further work is necessary for large-vocabulary continuous-speech problems, to develop training algorithms that progressively build internal word models, and to develop compact VLSI neural net hardware.

Review of neural networks for speech recognition

The problem of audio information retrieval is familiar to anyone who has returned from vacation to find ananswering machine full of messages. While there is not yetan "AltaVista" for the audio data type, many workers arefinding ways to automatically locate, index, and browse audio using recent advances in speech recognition and machinelistening. This paper reviews the state of the art in audioinformation retrieval, and presents recent advances in automatic speech recognition, word spotting, speaker and music identification, and audio similarity with a view towardsmaking audio less "opaque". A special section addresses intelligent interfaces for navigating and browsing audio andmultimedia documents, using automatically derived information to go beyond the tape recorder metaphor.

http://marco.uminho.pt/disciplinas/UCAN/BD/Artigos%20Recomendados/acm98.pdf

An overview of audio information retrieval

One of the attractions of neural network approaches to pattern recognition is the use of a discrimination-based training method. We show that once we have modified the output layer of a multilayer perceptron to provide mathematically correct probability distributions, and replaced the usual squared error criterion with a probability-based score, the result is equivalent to Maximum Mutual Information training, which has been used successfully to improve the performance of hidden Markov models for speech recognition. If the network is specially constructed to perform the recognition computations of a given kind of stochastic model based classifier then we obtain a method for discrimination-based training of the parameters of the models. Examples include an HMM-based word discriminator, which we call an 'Alphanet'.

/pdf/training-stochastic-model-recognition-algorithms-as-networks-393em1d00h.pdf

Training Stochastic Model Recognition Algorithms as Networks can Lead to Maximum Mutual Information Estimation of Parameters

Algorithms for a speaker-dependent wordspotting system based on hidden Markov models (HMMs) are described. The system allows a user to specify keywords dynamically and to train the associated HMMs via a single repetition of a keyword. Nonkeyword speech is modeled using an HMM trained from a prerecorded sample of continuous speech. The wordspotter is intended for interactive applications, such as the editing of voice mail or mixed-media documents, and for keyword indexing in audio or video recordings. The forward-backward search algorithm used in the wordspotter is compared with the Viterbi decoder on the basis of speed and accuracy. In addition, an algorithm for speaker adaptation is described which allows indexing by a user into another talker's speech. >

Training and search algorithms for an interactive wordspotting system

Experiments comparing artificial neural network (ANN), k-nearest-neighbor (KNN), and Bayes' rule with Gaussian distributions and maximum-likelihood estimation (BGM) classifiers were performed. Classifier error rate as a function of training set size was tested for synthetic data drawn from several different probability distributions. In cases where the true distributions were poorly modeled, ANN was significantly better than BGM. In some cases, ANN was also better than KNN. Similar experiments were performed on a voiced/unvoiced speech classification task. ANN had a lower error rate than KNN or BGM for all training set sizes, although BGM approached the ANN error rate as the training set became larger. It is concluded that there are pattern classification tasks in which an ANN is able to make better use of training data to achieve a lower error rate with a particular size training set. >

How limited training data can allow a neural network to outperform an 'optimal' statistical classifier

A set of experiments in which the LVQ2 (learning vector quantization) algorithm of T. Kohonen et al (Proc. 1988 IEE Int. Conf. on Neural Networks, p.I-61-68, 1988) is used to generate vector codebooks for a discrete-observation hidden Markov model (HMM) classifier is described. Input feature vectors consist of single-frame linear predictive coding (LPC)-based cepstra and/or differenced cepstra. Classification accuracies using conventional k-means, class-specific k-means, and LVQ2 codebooks are compared for a 16-way speaker-independent vowel classification task. In contrast to speaker-dependent phonetic classification results previously published, no significant performance advantages are observed with LVQ2. These conflicting results are discussed relative to differences in the recognition tasks and the feature sets used. It is also argued that the single-observation Bayesian decision boundaries approximated by LVQ2 are nonoptimal for HMM-based classification involving multiple observations. >

Speaker-independent vowel classification using hidden Markov models and LVQ2

A Gaussian-model classifier trained by maximum mutual information estimation (MMIE) is compared to one trained by maximum-likelihood estimation (MLE) and to an artificial neural network (ANN) on several classification tasks. Similarity of MMIE and ANN results for uniformly distributed data confirm that the ANN is better than the MLE in some cases due to the ANNs use of an error-correcting training algorithm. When the probability model fits the data well, MLE is better than MMIE if the training data are limited, but they are equal if there are enough data. When the model is a poor fit, MMIE is better than MLE. Training dynamics of MMIE and ANN are shown to be similar under certain assumptions. MMIE seems more susceptible to overtraining and computational difficulties than the ANN. Overall, ANN is the most robust of the classifiers. >

Neural networks, maximum mutual information training, and maximum likelihood training (speech recognition)

The authors describe preliminary CV (consonant-vowel syllable) recognition experiments using neural network learning and retrieval paradigms. They have trained both one and two speaker systems and report on the results of both speaker dependent and speaker independent testing. The one-speaker systems performed at 94 percent correct classifying the three voiced stop consonants learned in 3 different vowel contexts using 40ms of data from burst onset per CV token (all results involve tokens not used in training). Vowel performance was also good when at least 55 ms of data from each CV were used. The system receives no segmentation information on consonant-vowel boundary. When both voiced and voiceless CV types were learned together by a one-speaker system, consonant performance dropped to just over 80 percent. This fall-off was mostly due to weak performance on velar stops. Two-speaker systems trained on the voiced CV tokens in three vowel contexts also performed at or above 90 percent. The two-speaker system also demonstrated speaker independent ability with over 80 percent correct consonant classification with voiced CV tokens from a third speaker. >

M.A. Bush

Papers

Training and search algorithms for an interactive wordspotting system

How limited training data can allow a neural network to outperform an 'optimal' statistical classifier

Speaker-independent vowel classification using hidden Markov models and LVQ2

Neural networks, maximum mutual information training, and maximum likelihood training (speech recognition)

A connectionist model for consonant-vowel syllable recognition