The present invention provides improved foreground-speech signal endpointing by computing a spectral stationarity statistic. This statistic is used by a finite state machine to endpoint speech. Endpointing using the spectral stationarity statistic is less susceptible to background noise than endpointing using conventional measures. The present invention uses frame-synchronous quantile estimation to generate a mask signal for signal to Noise Ratio Normalization.

Method and apparatus to detect and delimit foreground speech

Automatic speech recognition (ASR) systems have made dramatic performance leaps in the recent past. Yet, the notion that the key to making recognition more robust is to reduce the difference between training and test conditions is still commonly held. As ASR applications move from tightly controlled to more natural environments with a varying number of unpredictable sound sources, this assumption is becoming less and less viable. Decoding the speech source of interest while listening to several sound sources at the same time seems a more accurate description of the ASR process that suits these challenging environments. This thesis discusses the theoretical and practical issues which arise from this viewpoint. The aim is to explore the division of the problem of robust ASR into two subproblems: (a) identification/separation of the speech and noise using speech properties alone; and (b) recognition based on the resulting partial evidence. The basic assumption is that some regions of the speech time-frequency representation remain relatively unaffected by the noise, that they can be identified and that they alone are sufficient for ASR. In contrast to conventional techniques which require models of all sources in the auditory scene and their subsequent decoding even when only one of the sources is of interest, the techniques described in this thesis make no such requirement. However, they are flexible enough to use this information if it is available. Two techniques are used to adapt a conventional Hidden Markov model (HMM) based ASR system to use partial evidence: (i) marginalisation of the state distributions, so that only the likelihood of the reliable regions is assessed; and (ii) imputation of the unreliable regions by replacing the unreliable features with a single point from the state conditional distributions. In both cases, the ”counterevidence” assessing which states are unlikely to have generated the speech underlying the unreliable regions dominated by noise further constrains the decoding. The techniques are evaluated on the Aurora 2 connected digit recognition task, and seem to perform competitively. In the experiments, the reliable features are identified via local SNR estimates derived through stationary and adaptive on-line noise estimates. The potential of the techniques is indicated by using the clean speech to identify the reliable regions in the noisy speech, where the accuracy is maintained even at -5 dB. The simple all-or-nothing assumption (the feature is either reliable or unreliable) gives rise to a model linking the recognition and the separation as two interdependent sides of the search for the most likely explanation of the noisy data.

Robust Automatic Speech Recognition with Missing and Unreliable Data

We propose a formalism, called vector filtering of spectral trajectories, that allows the integration of a number of speech parameterization approaches (cepstral analysis, /spl Delta/ and /spl Delta//spl Delta/ parameterizations, auto-regressive vector modeling, ...) under a common formalism. We then propose a new filtering, called contextual principal components (CPC) or time-frequency principal components (TFPC). This filtering consists in extracting the principal components of the contextual covariance matrix, which is the covariance matrix of a sequence of vectors expanded by their context. We apply this new filtering in the framework of closed-set speaker identification, using a subset of the POLYCOST database. When using speaker-dependent TFPC filters, our results show a relative improvement of approximately 20% compared to the use of the classical cepstral coefficients augmented by their /spl Delta/-coefficients, which is significantly better with a 90% confidence level.

Application of time-frequency principal component analysis to text-independent speaker identification

Acoustic signals are analyzed by two-dimensional (2-D) processing of the one-dimensional (1-D) speech signal in the time-frequency plane. The short-space 2-D Fourier transform of a frequency-related representation (e.g., spectrogram) of the signal is obtained. The 2-D transformation maps harmonically-related signal components to a concentrated entity in the new 2-D plane (compressed frequency-related representation). The series of operations to produce the compressed frequency-related representation is referred to as the “grating compression transform” (GCT), consistent with sine-wave grating patterns in the frequency-related representation reduced to smeared impulses. The GCT provides for speech pitch estimation. The operations may, for example, determine pitch estimates of voiced speech or provide noise filtering or speaker separation in a multiple speaker acoustic signal.

2-d processing of speech

The paper explores the use of 2-dimensional cepstral-time features for the utilisation of correlation among successive speech spectral vectors, within a hidden-Markov-model (HMM) framework. A cepstral-time-feature matrix is obtained from a 2-dimensional discrete cosine transform of a spectral-time matrix. Advantages of cepstral-time features are that cepstral-time-feature matrices are a simple and robust method of representing short-time variation of speech spectral parameters; a cepstral-time matrix contains information on the transitional dynamics of feature vectors within the matrix; speech recognition based on cepstral time matrices is more robust in noisy environments; and use of a matrix of M cepstral vectors implies a minimum HMM-state duration constraint of M vector units. A simple framework investigated in the paper for applications of cepstral-time features is a finite-state-matrix quantiser (FSMQ), a special case of the HMM. It is used for initialisation of the training phase of HMMs.

Speech modelling using cepstral-time feature matrices in hidden Markov models

The paper investigates the robustness of cepstral based features with respect to additive noise, and details two methods of increasing the robustness with minimal need for a-priori knowledge of the noise statistics. The first approach is a form of noise masking which adds a fixed offset to the linear spectral estimate. The second is a form of sub-band filtering, again in the linear domain, which estimates the dynamic content of the speech using Fourier transforms. This avoids negative values normally inherent in such filtering and which presents difficulties in deriving log estimates. Both methods are shown to provide useful levels of robustness to additive noise, for example, speaker identification error rates in SNR mis-matched conditions of 15 dB are reduced from 60.5% for standard mel cepstra to 13.3% and 24.1% for the two approaches respectively.

J.S. Mason

Papers

Noise robust estimate of speech dynamics for speaker recognition