Showing papers by "Lawrence K. Saul published in 1997"
Proceedings Article•
01 Jun 1997
TL;DR: This work considers the use of language models whose size and accuracy are intermediate between different order n-gram models and examines smoothing procedures in which these models are interposed between different orders.
Abstract: We consider the use of language models whose size and accuracy are intermediate between different order n-gram models. Two types of models are studied in particular. Aggregate Markov models are classbased bigram models in which the mapping from words to classes is probabilistic. Mixed-order Markov models combine bigram models whose predictions are conditioned on different words. Both types of models are trained by ExpectationMaximization (EM) algorithms for maximum likelihood estimation. We examine smoothing procedures in which these models are interposed between different order n-grams. This is found to significantly reduce the perplexity of unseen word combinations.
192 citations
Patent•
AT&T1
TL;DR: In this article, factor analysis is used to model acoustic correlation in automatic speech recognition by introducing a small number of parameters to model the covariance structure of a speech signal, which are estimated by an Expectation Maximization (EM) technique that can be embedded in the training procedures for the HMMs, and then further adjusted using Minimum Classification Error (MCE) training, which demonstrates better discrimination and produces more accurate recognition models.
Abstract: Hidden Markov models (HMMs) rely on high-dimensional feature vectors to summarize the short-time properties of speech correlations between features that can arise when the speech signal is non-stationary or corrupted by noise. These correlations are modeled using factor analysis, a statistical method for dimensionality reduction. Factor analysis is used to model acoustic correlation in automatic speech recognition by introducing a small number of parameters to model the covariance structure of a speech signal. The parameters are estimated by an Expectation Maximization (EM) technique that can be embedded in the training procedures for the HMMs, and then further adjusted using Minimum Classification Error (MCE) training, which demonstrates better discrimination and produces more accurate recognition models.
23 citations
Posted Content•
TL;DR: The authors consider the use of language models whose size and accuracy are intermediate between different order n-gram models and examine smoothing procedures in which these models are interposed between different orders of ngrams, which significantly reduce the perplexity of unseen word combinations.
Abstract: We consider the use of language models whose size and accuracy are intermediate between different order n-gram models. Two types of models are studied in particular. Aggregate Markov models are class-based bigram models in which the mapping from words to classes is probabilistic. Mixed-order Markov models combine bigram models whose predictions are conditioned on different words. Both types of models are trained by Expectation-Maximization (EM) algorithms for maximum likelihood estimation. We examine smoothing procedures in which these models are interposed between different order n-grams. This is found to significantly reduce the perplexity of unseen word combinations.
10 citations
Proceedings Article•
01 Dec 1997TL;DR: This work evaluates the combined use of mixture densities and factor analysis in HMMs that recognize alphanumeric strings and finds that these methods, properly combined, yield better models than either method on its own.
Abstract: Hidden Markov models (HMMs) for automatic speech recognition rely on high dimensional feature vectors to summarize the short-time properties of speech. Correlations between features can arise when the speech signal is non-stationary or corrupted by noise. We investigate how to model these correlations using factor analysis, a statistical method for dimensionality reduction. Factor analysis uses a small number of parameters to model the covariance structure of high dimensional data. These parameters are estimated by an Expectation-Maximization (EM) algorithm that can be embedded in the training procedures for HMMs. We evaluate the combined use of mixture densities and factor analysis in HMMs that recognize alphanumeric strings. Holding the total number of parameters fixed, we find that these methods, properly combined, yield better models than either method on its own.
7 citations
14 Dec 1997
TL;DR: A statistical method for improved acoustic modeling in continuous density hidden Markov models (HMMs) using factor analysis, which uses a small number of parameters to model the covariance structure of the speech signal.
Abstract: Modeling acoustic correlation in automatic speech recognition systems is essential when the speech signal is non stationary or corrupted by noise. We present a statistical method for improved acoustic modeling in continuous density hidden Markov models (HMMs). Factor analysis uses a small number of parameters to model the covariance structure of the speech signal. These parameters are estimated by an Expectation-Maximization algorithm, then further adjusted using discriminative minimum classification error training. Experimental results on 1219 New Jersey town names demonstrate that the proposed method produces faster, smaller and more accurate recognition models.
2 citations