Representations of spatial audio scenes using higher-order Ambisonics (HOA) technology typically require a large number of coefficients per time instant. This data rate is too high for most practical applications that require real-time transmission of audio signals. According to the invention, the compression is carried out in spatial domain instead of HOA domain. The (N+1)2 input HOA coefficients are transformed into (N+1)2 equivalent signals in spatial domain, and the resulting (N+1)2 time-domain signals are input to a bank of parallel perceptual codecs. At decoder side, the individual spatial-domain signals are decoded, and the spatial-domain coefficients are transformed back into HOA domain in order to recover the original HOA representation.

Method and apparatus for encoding and decoding successive frames of an ambisonics representation of a 2- or 3-dimensional sound field

Conventional audio compression technologies perform a standardized signal transformation, independent of the type of the content. Multi-channel signals are decomposed into their signal components, subsequently quantized and encoded. This is disadvantageous due to lack of knowledge on the characteristics of scene composition, especially for e.g. multi-channel audio or Higher-Order Ambisonics (HOA) content. An improved method for encoding pre-processed audio data comprises encoding the pre-processed audio data, and encoding auxiliary data that indicate the particular audio pre-processing. An improved method for decoding encoded audio data comprises determining that the encoded audio data had been pre-processed before encoding, decoding the audio data, extracting from received data information about the pre-processing, and post-processing the decoded audio data according to the extracted pre-processing information.

Method and device for improving the rendering of multi-channel audio signals

Informed source separation (ISS) aims at reliably recovering sources from a mixture. To this purpose, it relies on the assumption that the original sources are available during an encoding stage. Given both sources and mixture, a side-information may be computed and transmitted along with the mixture, whereas the original sources are not available any longer. During a decoding stage, both mixture and side-information are processed to recover the sources. ISS is motivated by a number of specific applications including active listening and remixing of music, karaoke, audio gaming, etc. Most ISS techniques proposed so far rely on a source separation strategy and cannot achieve better results than oracle estimators. In this study, we introduce Coding-based ISS (CISS) and draw the connection between ISS and source coding. CISS amounts to encode the sources using not only a model as in source coding but also the observation of the mixture. This strategy has several advantages over conventional ISS methods. First, it can reach any quality, provided sufficient bandwidth is available as in source coding. Second, it makes use of the mixture in order to reduce the bitrate required to transmit the sources, as in classical ISS. Furthermore, we introduce Nonnegative Tensor Factorization as a very efficient model for CISS and report rate-distortion results that strongly outperform the state of the art.

https://hal.inria.fr/hal-00869603/document

Coding-Based Informed Source Separation: Nonnegative Tensor Factorization Approach

The invention relates to a method for encoding a multi-channel audio signal representing a sound scene comprising a plurality of sound sources. The method comprises a step of decomposing (T) the multi-channel signal into frequency bands, and the steps, carried out by frequency band, of obtaining (OBT) data representative of the direction of the sound sources of the sound scene, selecting (Select) a set of sound sources forming the main sources, adapting (DiA_M) data representative of the direction of the selected sound sources based on restitution characteristics of the multi-channel signal, determining a mixing matrix of the main sources on the basis of the adapted data, mastering (M) the main sources with the determined matrix in order to obtain a sum signal with a reduced amount of channels, and encoding (Cod.Di) data representative of the direction of the sound sources and forming a binary stream including encoded data, wherein the binary stream can be transmitted in parallel with the sum signal. The present invention also relates to a decoding method for decoding the sum signal and direction information in order to determine the mixing matrix and to obtain a multi-channel signal, and to an encoder and decoder for same.

Advanced encoding of multi-channel digital audio signals

Spatially squeezed surround audio coding (S3AC) has been previously shown to provide efficient coding with perceptually accurate soundfield reconstruction when applied to ITU 5.1 multichannel audio. This paper investigates the application of S3AC to the coding of Ambisonic audio recordings. Traditional ambisonics achieve compression and backward compatibility through the use of the UHJ matrixing approach to obtain a stereo signal. In this paper the relationship to Ambisonic B-format signals is described and alternative approaches that derive a stereo or mono-downmix signal based on S3AC are presented and evaluated. The mono-downmix approach utilizes side information consisting of spatial cues that are quantized based on novel source localization listening experiments. Objective and subjective tests demonstrate significant improvements in the localization of sound sources resulting from decoding the compressed B-format signals to a 5.1 speaker playback.

/pdf/a-spatial-squeezing-approach-to-ambisonic-audio-compression-1d0zoz7u27.pdf

A Spatial Squeezing approach to Ambisonic audio compression

This paper presents a novel solution to multichannel spatial audio coding: spatial squeezing surround audio coding (S3AC). The S3AC scheme analyses a multichannel audio signal and downmixes it into a stereo signal pair containing both the monophonic properties of audio sources and their localization information; this avoids the need for side information. The approach uses time-frequency analysis of a spatial audio scene and exploits virtual sources and amplitude panning techniques to 'squeeze' 360deg of a horizontal soundfield to a 60deg stereo signal pair. In comparison with other spatial audio coding techniques, S3AC significantly advances in-band encoding of the localization information in the original sound scene and achieves accurate recoverability of dynamic localized sources.

/pdf/principles-and-analysis-of-the-squeezing-approach-to-low-bit-2za8i2oppw.pdf

Principles and Analysis of the Squeezing Approach to Low Bit Rate Spatial Audio Coding

This paper presents a technique for low bit rate compression of three-dimensional (3D) audio produced by multiple loudspeaker channels. The approach is based on the time-frequency analysis of the localization of spatial sound sources within the 3D space as rendered by a multi-channel audio signal (in this case 16 channels). This analysis results in the derivation of a stereo downmix signal representing the original 16 channels. Alternatively, a mono-downmix signal with side information representing the location of sound sources within the 3D spatial scene can also be derived. The resulting downmix signals are then compressed with a traditional audio coder, resulting in a representation of the 3D soundfield at bit rates comparable with existing stereo audio coders while maintaining the perceptual quality produced from separate encoding of each channel.

/pdf/a-general-compression-approach-to-multi-channel-three-qexi35oaz2.pdf

A General Compression Approach to Multi-Channel Three-Dimensional Audio

Spatially Squeezed Surround Audio Coding (S3AC) was introduced as an approach to multi-channel audio compression which specifically aims to preserve original source localization information. In this paper, extensions to S3AC that allow for the accurate coding of independent spatial sources overlapped in both frequency and time are described; these use compact side information. An evaluation of the coder applied to tone and band-pass spatial sources shows that S3AC offers improved source localization performance while maintaining bit-rates, when compared with other state-of-the-art spatial audio coders.

Encoding independent sources in spatially squeezed surround audio coding

The derivation of spatial cues representing source localisation information is a typical component of multichannel spatial audio coders. Efficient compression of spatial cues based on psychoacoustic localisation features is investigated. Results show that the proposed quantisation approach for spatial cue compression achieves bit-rates of less than 6 kbit/s while preserving critical source localisation information.

Bin Cheng

Papers

A Spatial Squeezing approach to Ambisonic audio compression

Principles and Analysis of the Squeezing Approach to Low Bit Rate Spatial Audio Coding

A General Compression Approach to Multi-Channel Three-Dimensional Audio

Encoding independent sources in spatially squeezed surround audio coding

Psychoacoustic-based quantisation of spatial audio cues