Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

Digital processing of speech signals

An intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Intelligent Automated Assistant

When building a unified vision system or gradually adding new capabilities to a system, the usual assumption is that training data for all tasks is always available. However, as the number of tasks grows, storing and retraining on such data becomes infeasible. A new problem arises where we add new capabilities to a Convolutional Neural Network (CNN), but the training data for its existing capabilities are unavailable. We propose our Learning without Forgetting method, which uses only new task data to train the network while preserving the original capabilities. Our method performs favorably compared to commonly used feature extraction and fine-tuning adaption techniques and performs similarly to multitask learning that uses original task data we assume unavailable. A more surprising observation is that Learning without Forgetting may be able to replace fine-tuning with similar old and new task datasets for improved new task performance.

Learning without Forgetting

Data-driven speech enhancement employing deep neural networks (DNNs) can provide state-of-the-art performance even in the presence of non-stationary noise. During the training process, most of the speech enhancement neural networks are trained in a fully supervised way with losses requiring noisy speech to be synthesized by clean speech and additive noise. However, in a real implementation, only the noisy speech mixture is available, which leads to the question, how such data could be advantageously employed in training. In this work, we propose an end-to-end non-intrusive PESQNet DNN which estimates perceptual evaluation of speech quality (PESQ) scores, allowing a reference-free loss for real data. As a further novelty, we combine the PESQNet loss with denoising and dereverberation loss terms, and train a complex mask-based fully convolutional recurrent neural network (FCRN) in a "weakly" supervised way, each training cycle employing some synthetic data, some real data, and again synthetic data to keep the PESQNet up-to-date. In a subjective listening test, our proposed framework outperforms the Interspeech 2021 Deep Noise Suppression (DNS) Challenge baseline overall by 0.09 MOS points and in particular by 0.45 background noise MOS points.

Deep Noise Suppression with Non-Intrusive PESQNet Supervision Enabling the Use of Real Training Data

Conventional narrowband (NB) telephony suffers from limited acoustic bandwidth at the receiver side, leading to degraded speech quality and intelligibility. In this paper, artificial speech bandwidth extension (ABE) of NB speech toward missing frequencies below about 300 Hz (low-frequency band, LB) is proposed to enhance the speech quality. The LB-ABE in this paper is employed together with a preexisting ABE toward high-frequency components to obtain spectrally balanced speech signals. In an instrumental quality assessment, the spectral distance in the LB was improved by more than 5 dB compared to NB speech. In a subjective listening test, the gap of speech quality between wideband and NB speech was significantly reduced when employing the proposed ABE toward low frequencies. The LB extension was found to further improve the preexisting ABE toward higher frequencies by a significant 0.26 CMOS points.

/pdf/sinusoidal-based-lowband-synthesis-for-artificial-speech-4j3k68441l.pdf

Sinusoidal-Based Lowband Synthesis for Artificial Speech Bandwidth Extension

In-car communication (ICC) systems supporting speech communication in noise by reproducing amplified speech from the car cabin in the car cabin ask for low-delay acoustic echo cancellation (AEC). In this paper we propose a delay-flexible DFTbased stereo AEC capable of cancelling also the echoes stemming from the audio player or FM radio. For the price of a somewhat higher complexity we are able to reduce the 32 ms delay of the baseline down to 4 ms, loosing only 1 dB in ERLE while even preserving system distance properties.

A Delay-Flexible Stereo Acoustic Echo Cancellation for DFT-Based In-Car Communication (ICC) Systems.

Hearing impaired persons, and particularly hearing-aid and cochlear implant (CI) users, often have difficulties communicating over the telephone. The intelligibility of classical so-called narrowband telephone speech is considerably lower than the intelligibility of face-to-face speech. This is partly because of the lack of visual cues, limited telephone bandwidth, and background noise. This work proposes to artificially extend the standard bandwidth of telephone speech to improve its intelligibility for CI users. Artificial speech bandwidth extension (ABE) is obtained through a front-end signal processing algorithm that estimates missing speech components in the high-frequency spectrum from learned data. A state-of-the-art ABE approach, which already led to superior speech quality for people with normal hearing, is used for processing telephone speech for CI users. Two different parameterizations are evaluated, one being more aggressive than the other. Nine CI users were tested with and without the proposed ABE algorithm. The experimental evaluation shows a significant improvement in speech intelligibility and speech quality over the phone for both versions of the ABE algorithm. These promising results support the potential of ABE, which could be incorporated into a commercial speech processor or a smartphone-based pre-processor that streams the telephone speech to the CI.

Artificial speech bandwidth extension improves telephone speech intelligibility and quality in cochlear implant users

The typical goals of transmission system design such as high capacity, but also high signal quality and robustness usually contradict each other. Thanks to Shannon’s landmark contribution to information theory [Shannon 1948] one can treat the issues of bit rate reduction (source coding) and error protection (channel coding) separately. This however in principle presumes unlimited computational power and delay. In realtime conversational and streaming services for speech, audio, and video transmission in particular these assumptions are of course not met. To guarantee a limited delay and to achieve a certain level of robustness, frame-based transmission is employed, which limits the ability to remove interframe correlations. In virtually all source coders, this fact is practically observed in terms of residual redundancy in the bit stream.

Tim Fingscheidt

Papers

Deep Noise Suppression with Non-Intrusive PESQNet Supervision Enabling the Use of Real Training Data

Sinusoidal-Based Lowband Synthesis for Artificial Speech Bandwidth Extension

A Delay-Flexible Stereo Acoustic Echo Cancellation for DFT-Based In-Car Communication (ICC) Systems.

Artificial speech bandwidth extension improves telephone speech intelligibility and quality in cochlear implant users

Parameter Models and Estimators in Soft Decision Source Decoding