Robust automatic speech recognition with missing and unreliable acoustic data

Do listeners process noisy speech by taking advantage of "glimpses"-spectrotemporal regions in which the target signal is least affected by the background? This study used an automatic speech recognition system, adapted for use with partially specified inputs, to identify consonants in noise. Twelve masking conditions were chosen to create a range of glimpse sizes. Several different glimpsing models were employed, differing in the local signal-to-noise ratio (SNR) used for detection, the minimum glimpse size, and the use of information in the masked regions. Recognition results were compared with behavioral data. A quantitative analysis demonstrated that the proportion of the time-frequency plane glimpsed is a good predictor of intelligibility. Recognition scores in each noise condition confirmed that sufficient information exists in glimpses to support consonant identification. Close fits to listeners' performance were obtained at two local SNR thresholds: one at around 8 dB and another in the range -5 to -2 dB. A transmitted information analysis revealed that cues to voicing are degraded more in the model than in human auditory processing.

/pdf/a-glimpsing-model-of-speech-perception-in-noise-35tohj1zpa.pdf

A glimpsing model of speech perception in noise.

In his famous treatise of computational vision, Marr (1982) makes a compelling argument for separating different levels of analysis in order to understand complex information processing. In particular, the computational theory level, concerned with the goal of computation and general processing strategy, must be separated from the algorithm level, or the separation of what from how. This chapter is an attempt at a computational-theory analysis of auditory scene analysis, where the main task is to understand the character of the CASA problem.

/pdf/on-ideal-binary-mask-as-the-computational-goal-of-auditory-3m6673dmtc.pdf

On Ideal Binary Mask As the Computational Goal of Auditory Scene Analysis

New waves of consumer-centric applications, such as voice search and voice interaction with mobile devices and home entertainment systems, increasingly require automatic speech recognition (ASR) to be robust to the full range of real-world noise and other acoustic distorting conditions. Despite its practical importance, however, the inherent links between and distinctions among the myriad of methods for noise-robust ASR have yet to be carefully studied in order to advance the field further. To this end, it is critical to establish a solid, consistent, and common mathematical foundation for noise-robust ASR, which is lacking at present. This article is intended to fill this gap and to provide a thorough overview of modern noise-robust techniques for ASR developed over the past 30 years. We emphasize methods that are proven to be successful and that are likely to sustain or expand their future applicability. We distill key insights from our comprehensive overview in this field and take a fresh look at a few old problems, which nevertheless are still highly relevant today. Specifically, we have analyzed and categorized a wide range of noise-robust techniques using five different criteria: 1) feature-domain vs. model-domain processing, 2) the use of prior knowledge about the acoustic environment distortion, 3) the use of explicit environment-distortion models, 4) deterministic vs. uncertainty processing, and 5) the use of acoustic models trained jointly with the same feature enhancement or model adaptation process used in the testing stage. With this taxonomy-oriented review, we equip the reader with the insight to choose among techniques and with the awareness of the performance-complexity tradeoffs. The pros and cons of using different noise-robust ASR techniques in practical application scenarios are provided as a guide to interested practitioners. The current challenges and future research directions in this field is also carefully analyzed.

/pdf/an-overview-of-noise-robust-automatic-speech-recognition-u8zbz18oa3.pdf

An overview of noise-robust automatic speech recognition

inability to deal with signals in the high -freq uency range. Psychoacoustic evidence sugge sts that different perceptual mechanisms are involved to handle resolved and unresolved hannonics. We propose a system for speech segre gation that deals with low-frequency and high-frequency signals differently. For low-frequency signals , our model generates segm ents based on temporal continuity and cross-channel correlation , and groups them according to periodicity. For high-frequency signals, the model generates segments based on common amplitude modulation (AM) in addition to temporal contin uity, and grou ps them according to AM repetition rates. Underlying the group ing process is a p itch contour that is first est imated from segregated speech based on global pitch and then verified by psychoacoustic constraints. Our system is systematically evaluated, and it yields substantially better performan ce than previous CASA systems, especially in the high-frequency range.

Monaural speech segregation based on pitch tracking and

In the missing data approach to robust Automatic Speech Recognition (ASR), time-frequency regions which carry reliable speech information are identified. Recognition is then based on these regions alone. In this paper, we address the problem of identifying reliable regions and propose two criteria to solve this based on negative energy ( $ s < 0 ) and SNR ( $ s s n 2 2 2 < + ). These criteria are evaluated on the TIDigits corpus for several noise sources and compared with spectral subtraction. We show that in this task the missing data method performs considerably better than spectral subtraction and the combination of the two techniques outperforms either technique used alone. We report robust performance at 0dB SNR for car noise and 10dB SNR for factory noise.

/pdf/missing-data-theory-spectral-subtraction-and-signal-to-noise-3dg938qrn7.pdf

Missing data theory, spectral subtraction and signal-to-noise estimation for robust ASR: an integrated study.

Within the context of continuous-density HMM speech recognition in noise, we report on imputation of missing time-frequency regions using emission state probability distributions. Spectral subtraction and local signal–to– noise estimation based criteria are used to separate the present from the missing components. We consider two approaches to the problem of classification with missing data: marginalization and data imputation. A formalism for data imputation based on the probability distributions of individual Hidden Markov model states is presented. We report on recognition experiments comparing state based data imputation to marginalization in the context of connected digit recognition of speech mixed with factory noise at various global signal-to-noise ratios, and wideband restoration of speech. Potential advantages of the approach are that it can be followed by conventional techniques like cepstral features or artificial neural networks for speech recognition.

/pdf/state-based-imputation-of-missing-data-for-robust-speech-3tv5jhd3tj.pdf

State based imputation of missing data for robust speech recognition and speech enhancement.

Monitoring surface waves in coastal waters by integrating HF radar measurement and modelling

Human speech perception withstands a wide variety of distortions, both experimentally applied and naturallyoccurring A novel approach to these situations in robust ASR identifies the spectro-temporal regions which carry reliable speech evidence and treats the remainder as missing or uncertain This standpoint makes minimal assumptions about any noise background This paper describes two approaches to the adaptation of continuous-density hidden Markov model-based speech recognisers to deal with missing and uncertain acoustic data The first computes output probabilities on the basis of the reliable evidence only, while the second estimates values for the unreliable regions by conditioning on the reliable parts Both techniques are evaluated on the TIDigits corpus for several NOISEX noise sources, using spectral subtraction to identify reliable regions These studies demonstrate that the two schemes behave comparably, and that both produce a significant performance advantage over spectral subtraction alone

Ascension Vizinho

Papers

Robust automatic speech recognition with missing and unreliable acoustic data

Missing data theory, spectral subtraction and signal-to-noise estimation for robust ASR: an integrated study.

State based imputation of missing data for robust speech recognition and speech enhancement.

Monitoring surface waves in coastal waters by integrating HF radar measurement and modelling

Robust asr with unreliable data and minimal assumptions