All-Round Ambisonic Panning (AllRAP) is an algorithm for arbitrary loudspeaker arrangements, aiming at the creation of phantom sources of stable loudness and adjustable width. The equivalent All-Round Ambisonic Decoding (AllRAD) fits into the Ambisonic format concept. Conventional Ambisonic decoding is only simple with optimal loudspeaker arrangements for which it achieves direction-independent energy and energy spread, the estimated phantom source loudness and width. AllRAP/AllRAD is still simple but more versatile and utilizes the combination of a virtual optimal loudspeaker arrangement with Vector-Base Amplitude Panning.

/pdf/all-round-ambisonic-panning-and-decoding-i4ozh50f91.pdf

All-Round Ambisonic Panning and Decoding

Removing background noise from speech audio has been the subject of considerable effort, especially in recent years due to the rise of virtual communication and amateur recordings. Yet background noise is not the only unpleasant disturbance that can prevent intelligibility: reverb, clipping, codec artifacts, problematic equalization, limited bandwidth, or inconsistent loudness are equally disturbing and ubiquitous. In this work, we propose to consider the task of speech enhancement as a holistic endeavor, and present a universal speech enhancement system that tackles 55 different distortions at the same time. Our approach consists of a generative model that employs score-based diffusion, together with a multi-resolution conditioning network that performs enhancement with mixture density networks. We show that this approach significantly outperforms the state of the art in a subjective test performed by expert listeners. We also show that it achieves competitive objective scores with just 4-8 diffusion steps, despite not considering any particular strategy for fast sampling. We hope that both our methodology and technical contributions encourage researchers and practitioners to adopt a universal approach to speech enhancement, possibly framing it as a generative task.

Universal Speech Enhancement with Score-based Diffusion

Despite recent progress in generative adversarial network (GAN)-based vocoders, where the model generates raw waveform conditioned on acoustic features, it is challenging to synthesize high-fidelity audio for numerous speakers across various recording environments. In this work, we present BigVGAN, a universal vocoder that generalizes well for various out-of-distribution scenarios without fine-tuning. We introduce periodic activation function and anti-aliased representation into the GAN generator, which brings the desired inductive bias for audio synthesis and significantly improves audio quality. In addition, we train our GAN vocoder at the largest scale up to 112M parameters, which is unprecedented in the literature. We identify and address the failure modes in large-scale GAN training for audio, while maintaining high-fidelity output without over-regularization. Our BigVGAN, trained only on clean speech (LibriTTS), achieves the state-of-the-art performance for various zero-shot (out-of-distribution) conditions, including unseen speakers, languages, recording environments, singing voices, music, and instrumental audio. We release our code and model at: https://github.com/NVIDIA/BigVGAN

BigVGAN: A Universal Neural Vocoder with Large-Scale Training

Method and preliminary results of multiple sound sources localization in free field using the acoustic vector sensor were presented in this study. Direction of arrival (DOA) for considered source was determined based on sound intensity method supported by Fourier analysis. Obtained spectrum components for considered signal allowed to determine the DOA value for the particular frequency independently. The accuracy of the developed and practically implemented algorithm was evaluated on the basis of laboratory tests. Both synthetic acoustic signals (pure tones and noises) and real sounds were used during the measurements. Real signals had the same or different energy distribution both on time and frequency domain. The setup of the experiment and obtained results were described in details in the text. Taking the obtained results into consideration is important to emphasize that the localization of the multiple sound sources using single acoustic vector sensor is possible. The localization accuracy was the best for signals which spectral energy distribution was different.

/pdf/multiple-sound-sources-localization-in-free-field-using-4ro4kapvzi.pdf

Multiple sound sources localization in free field using acoustic vector sensor

An end-to-end AV broadcast system providing an immersive, interactive experience for live events is the development aim for the EU FP7 funded project, FascinatE. The project has developed real time audio object event detection and localisation, scene modelling and processing methods for multimedia data, which will allow users to navigate the event by creating their own unique user-defined scene. As part of the first implementation of the system a test shoot was carried out capturing a live Premier League football game and methods have been developed to detect, analyse, extract and localise salient audio events from a range of sensors and represent them within an audio scene in order to allow free navigation within the scene. Within this context, this paper describes a procedure for the detection, extraction and localisation of ball-kicks and whistle-blows from the pitch-side microphones used in the broadcast of football and describes a potential audio streaming format for an object-based broadcast.

/pdf/object-based-audio-for-interactive-football-broadcast-5ggu3abr4s.pdf

Object-based audio for interactive football broadcast

A number of recent advances in neural audio synthesis rely on upsampling layers, which can introduce undesired artifacts. In computer vision, upsampling artifacts have been studied and are known as checkerboard artifacts (due to their characteristic visual pattern). However, their effect has been overlooked so far in audio processing. Here, we address this gap by studying this problem from the audio signal processing perspective. We first show that the main sources of upsampling artifacts are: (i) the tonal and filtering artifacts introduced by problematic upsampling operators, and (ii) the spectral replicas that emerge while upsampling. We then compare different upsampling layers, showing that nearest neighbor upsamplers can be an alternative to the problematic (but state-of-the-art) transposed and subpixel convolutions which are prone to introduce tonal artifacts.

Upsampling artifacts in neural audio synthesis

A number of recent advances in neural audio synthesis rely on up-sampling layers, which can introduce undesired artifacts. In computer vision, upsampling artifacts have been studied and are known as checkerboard artifacts (due to their characteristic visual pattern). However, their effect has been overlooked so far in audio processing. Here, we address this gap by studying this problem from the audio signal processing perspective. We first show that the main sources of upsampling artifacts are: (i) the tonal and filtering artifacts introduced by problematic upsampling operators, and (ii) the spectral replicas that emerge while upsampling. We then compare different upsampling layers, showing that nearest neighbor upsamplers can be an alternative to the problematic (but state-of-the-art) transposed and subpixel convolutions which are prone to introduce tonal artifacts.

Upsampling Artifacts in Neural Audio Synthesis

A gain contribution of the audio signal for each of the N audio objects to at least one of M speakers may be determined. Determining the gain contribution may involve determining a center of loudness position that is a function of speaker (or cluster) positions and gains assigned to each speaker (or cluster). Determining the gain contribution also may involve determining a minimum value of a cost function. A first term of the cost function may represent a difference between the center of loudness position and an audio object position.

Giulio Cengarle

Papers

Upsampling artifacts in neural audio synthesis

Upsampling Artifacts in Neural Audio Synthesis

Panning of audio objects to arbitrary speaker layouts

A New Technology for the Assisted Mixing of Sport Events: Application to Live Football Broadcasting

Comparison of Anemometric Probe and Tetrahedral Microphones for Sound Intensity Measurements