Showing papers by "Vincent Vanhoucke published in 2003"

Mixtures of inverse covariances

Abstract: We introduce a model that approximates full and block-diagonal covariances in a Gaussian mixture, while reducing significantly both the number of parameters to estimate and the computations required to evaluate the Gaussian likelihoods. The inverse covariance of each Gaussian is expressed as a mixture of a small set of prototype matrices. Estimation of both the mixture weights and the prototypes is performed using maximum likelihood estimation. Experiments on a variety of speech recognition tasks show that this model significantly outperforms a diagonal covariance model, while using the same number of Gaussian-dependent parameters.

Variable length mixtures of inverse covariances.

Abstract: The mixture of inverse covariances model is a low-complexity, approximate decomposition of the inverse covariance matrices in a Gaussian mixture model which achieves high modeling accuracy with very good computational efficiency. In this model, the inverse covariances are decomposed into a linear combination of K shared prototype matrices. In this paper, we introduce an extension of this model which uses a variable number of prototypes per Gaussian for improved efficiency. The number of prototypes per Gaussian is optimized using a maximum likelihood criterion. This variable length model is shown to achieve significantly better accuracy at a given complexity level on several speech recognition tasks.