As speech processing devices like mobile phones, voice controlled devices, and hearing aids have increased in popularity, people expect them to work anywhere and at any time without user intervention However, the presence of acoustical disturbances limits the use of these applications, degrades their performance, or causes the user difficulties in understanding the conversation or appreciating the device A common way to reduce the effects of such disturbances is through the use of single-microphone noise reduction algorithms for speech enhancement The field of single-microphone noise reduction for speech enhancement comprises a history of more than 30 years of research In this survey, we wish to demonstrate the significant advances that have been made during the last decade in the field of discrete Fourier transform domain-based single-channel noise reduction for speech enhancementFurthermore, our goal is to provide a concise description of a state-of-the-art speech enhancement system, and demonstrate the relative importance of the various building blocks of such a system This allows the non-expert DSP practitioner to judge the relevance of each building block and to implement a close-to-optimal enhancement system for the particular application at hand Table of Contents: Introduction / Single Channel Speech Enhancement: General Principles / DFT-Based Speech Enhancement Methods: Signal Model and Notation / Speech DFT Estimators / Speech Presence Probability Estimation / Noise PSD Estimation / Speech PSD Estimation / Performance Evaluation Methods / Simulation Experiments with Single-Channel Enhancement Systems / Future Directions

DFT-Domain Based Single-Microphone Noise Reduction for Speech Enhancement: A Survey of the State of the Art

In this paper, we present an overview on the topic of noise reduction in the short-time Fourier transform (STFT) domain. First, we briefly review the conventional literature in the single- and multichannel cases separately. In the single-channel scenario, we focus on the spectral subtractive methods, Wiener filter based methods, speech amplitude estimators and estimators of the complex STFT coefficients. In the multi-channel scenario, we investigate in short a selection of key beamforming approaches as well as conventional post-filtering methods. Next, a detailed survey of the most recent advances in the STFT-based noise reduction methods is provided. This includes STSA estimators with super-Gaussian priors, noise power spectral density (PSD ) estimation, estimation methods in the modulation domain, estimation of spectral phase and noise PSD matrix estimation for multi-channel applications. Finally, we summarize the presented material and draw important conclusions on each of the investigated topics.

Recent Developments in Speech Enhancement in the Short-Time Fourier Transform Domain

Estimating a wideband spectral envelope having only narrowband speech at hand is a challenging task. In this paper, we explore ways to do so in the context of an artificial speech bandwidth extension (ABE) framework. Starting from a typical hidden Markov model (HMM)/Gaussian mixture model baseline scheme, we investigate two types of features, topologies, and regularization approaches of deep neural networks (DNNs) to obtain estimates of wideband spectral envelopes with smallest cepstral distance to the original ones. In order to draw realistic conclusions, we employ a database for test, which is acoustically different to the training and validation speech material. Interestingly, it turns out that a DNN regression approach outperforms all other investigated methods, although the HMM has been dropped. Cepstral distance was reduced by 1.18 dB, wideband PESQ was improved by 0.23 MOS points, and a subjective comparison category rating listening test showed a significant preference of the best DNN ABE approach versus narrowband speech of 1.37 CMOS points.

Artificial Speech Bandwidth Extension Using Deep Neural Networks for Wideband Spectral Envelope Estimation

Handbook of Probability and Statistics with Tables

Regression based on neural networks (NNs) has led to considerable advances in speech enhancement under non-stationary noise conditions. Nonetheless, speech distortions can be introduced when employing NNs trained to provide strong noise suppression. We propose to address this problem by first suppressing noise and subsequently restoring speech with specifically chosen NN topologies for each of these distinct tasks. A mask-estimating long short-term memory (LSTM) network is employed for noise suppression, while the speech restoration is performed by a fully convo-lutional encoder-decoder (CED) network, where we introduce temporal modeling capabilities by using a convolutional LSTM layer in the bottleneck. We show considerable performance gains over reference methods of up to 0.26 MOS points (PESQ) and the ability to significantly improve intelligibility in terms of STOI for low-SNR conditions.

Separated Noise Suppression and Speech Restoration: Lstm-Based Speech Enhancement in Two Stages

Several investigations showed that speech enhancement approaches can be improved by speech presence uncertainty (SPU) estimation. Although there has been a strong focus on the use of correct statistical models for spectral weighting rules for the last few decades, there is just a few publications about SPU estimation based on a speech prior consistent with the spectral weighting rule. This contribution presents a new consistent solution for MMSE speech amplitude (SA) estimation under SPU, being based on the generalized gamma distribution representing a variety of speech priors. Employing the gamma speech model which is a special case of the generalized gamma distribution, the new approach is shown to outperform both the SPU-based MMSE-SA estimator relying on a Gaussian speech prior, and the gamma MMSE-SA estimation without SPU.

MMSE speech enhancement under speech presence uncertainty assuming (generalized) gamma speech priors throughout

In many applications non-stationary Gaussian or stationary non-Gaussian noises can be observed. In this paper we present a maximum a posteriori estimation jointly of spectral amplitude and phase (JMAP). It principally allows for arbitrary speech models (Gaussian, super-Gaussian, …), while the noise DFT coefficients pdf is modeled as Gaussian mixture (GMM). Such a GMM covers both a non-Gaussian stationary noise process, but also a non-stationary process that changes between Gaussian noise modes of different variance with probability of the GMM weight. Accordingly, we provide results for these two types of noise, showing superiority over the Gaussian noise model JMAP estimator even in case of ideal noise power estimation.

Speech enhancement using a joint map estimator with Gaussian mixture model for (non-)stationary noise

Signal-to-noise (SNR) measurement of noisy speech signals is an important topic, e. g., in automotive environments. This paper presents an approach which aims to measure the SNR of a speech signal distorted by stationary noise as close as possible to a reference-based SNR measurement according to the ITU-T Recommendation P.56, however, without using any reference signals. The proposal offers low computational complexity, and it can be applied to narrowband and wideband signals within a typical SNR range. The measured SNR is calculated by means of a separate estimate of speech and noise power, resulting in raw SNR measurements which have to be corrected by a mapping function. The proposal was evaluated thoroughly by means of speech data in 15 different languages and car noise data. The proposed method achieves a correlation coefficient larger than 0.99 compared to state-of-theart reference-based SNR measurement employing P.56 reference levels of speech and noise. The method is proposed for a new ITU-T Recommendation within Study Group 12, Focus Group Carcom.

Reference-free SNR Measurement for Narrowband and Wideband Speech Signals in Car Noise

Explicit information about speech presence or absence is needed in many speech processing applications. In a Bayesian estimation framework, this information can be provided by an a posteriori speech presence probability (SPP) estimator. Recent improvements in SPP estimation include likelihoods of speech presence based on a super-Gaussian speech model or, alternatively, based on averaged observations. In this paper, we combine these aspects and derive a closed form solution for the likelihood of speech presence based on both averaged observations and a super-Gaussian speech model. The new approach is shown to outperform competing methods that either include averaging or super-Gaussian speech models.

A posteriori speech presence probability estimation based on averaged observations and a super-gaussian speech model

In many applications non-Gaussian noises, such as babble noise, can be observed. In this paper we present a minimum mean square error (MMSE) estimation of the speech spectral amplitude. It principally allows for arbitrary speech spectral amplitude probability density function (pdf) models (Rayleigh, Chi, …), while the pdf of the noise DFT coefficients is modeled by a Gaussian mixture (GMM). Applying for both approaches an idealized a priori SNR estimator that works well in babble noise, we can show clear improvements compared to the MMSE spectral amplitude estimator with Gaussian noise assumption.

Balázs Fodor

Papers

MMSE speech enhancement under speech presence uncertainty assuming (generalized) gamma speech priors throughout

Speech enhancement using a joint map estimator with Gaussian mixture model for (non-)stationary noise

Reference-free SNR Measurement for Narrowband and Wideband Speech Signals in Car Noise

A posteriori speech presence probability estimation based on averaged observations and a super-gaussian speech model

MMSE speech spectral amplitude estimation assuming non-Gaussian noise