An audio “desktop” user-interface is provided in which services are represented by audio labels presented in an audio field through respective synthesized sound sources. A desired service is selected by identifying it through its sound source or audio label. A user can modify the layout of service-representing sound sources and preferably add and remove services.

Audio user interface

Numerous applications are currently envisioned for immersive audio systems. The principal function of such systems is to synthesize, manipulate, and render sound fields in real time. We examine several fundamental and technological limitations that impede the development of seamless immersive audio systems. Such limitations stem from signal processing requirements, acoustical considerations, human listening characteristics, and listener movement. We present a brief historical overview to outline the development of immersive audio technologies and discuss the performance and future research directions of immersive audio systems with respect to such limits. Last, we present a novel desktop audio system with integrated listener-tracking capability that circumvents several of the technological limitations faced by today's digital audio workstations.

/pdf/fundamental-and-technological-limitations-of-immersive-audio-pkod8fasx8.pdf

Fundamental and technological limitations of immersive audio systems

Described are systems and methods performed by computer to reduce crosstalk produced by loudspeakers when rendering binaural sound that is emitted from the loudspeakers into a room. The room may have sound-reflecting surfaces that reflect some of the sound produced by the loudspeakers. To reduce crosstalk, a room model stored by the computer, is accessed. The room model models at least sound reflected by one or more of the physical surfaces. The room model is used to calculate a model of an audio channel from the loudspeakers to a listener. The model of the audio channel models sound transmission from the loudspeakers to the listener. The computer uses the model of the audio channel to cancel crosstalk from the loudspeakers when rendering the binaural sound.

Audio spatialization using reflective room model

The authors discuss immersive audio systems and the signal processing issues that pertain to the acquisition and subsequent rendering of 3D sound fields over loudspeakers. On the acquisition side, recent advances in statistical methods for achieving acoustical arrays in audio applications are reviewed. Classical array signal processing addresses two major aspects of spatial filtering, namely localization of a signal of interest, and adaptation of the spatial response of an array of sensors to achieve steering in a given direction. The achieved spatial focusing in the direction of interest makes array signal processing a necessary component in immersive sound acquisition systems. On the rendering side, 3D audio signal processing methods are described that allow rendering of virtual sources around the listener using only two loudspeakers. Finally, the authors discuss the commercial implications of audio DSP.

https://ict.usc.edu/pubs/Surrounded%20by%20Sound-%20Acquisition%20and%20Rendering%20Methods%20for%20Immersive%20Audio.pdf

Surrounded by sound

Immersive audio systems can be used to render virtual sound sources in three-dimensional (3-D) space around a listener. This is achieved by simulating the head-related transfer function (HRTF) amplitude and phase characteristics using digital filters. In this paper, we examine certain key signal processing considerations in spatial sound rendering over headphones and loudspeakers. We address the problem of crosstalk inherent in loudspeaker rendering and examine two methods for implementing crosstalk cancellation and loudspeaker frequency response inversion in real time. We demonstrate that it is possible to achieve crosstalk cancellation of 30 dB using both methods, but one of the two (the Fast RLS Transversal Filter Method) offers a significant advantage in terms of computational efficiency. Our analysis is easily extendable to nonsymmetric listening positions and moving listeners.

/pdf/inverse-filter-design-for-immersive-audio-rendering-over-3qy6tekdo5.pdf

Inverse filter design for immersive audio rendering over loudspeakers

Signal Processing, Acoustics, and Psychoacoustics for High Quality Desktop Audio

Immersive audio systems are being envisioned for applications that include teleconferencing and telepresence; augmented and virtual reality for manufacturing and entertainment; air traffic control, pilot warning, and guidance systems; displays for the visually- or aurally-impaired; home entertainment; distance learning; and professional sound and picture editing for television and film. The principal function of such systems is to synthesize, manipulate, and render sound fields in real time. In this paper we examine the limitations that are inherent in spatial sound delivery over loudspeakers and propose a method that generates virtual sound sources based on synthetic head-related transfer functions with the same spectral characteristics as those of the real source.

Head-related transfer function synthesis for immersive audio

Virtual Loudspeaker Rendering for Multiple Listeners

Immersive audio systems are being envisioned for applications that include teleconferencing and telepresence; augmented and virtual reality for manufacturing and entertainment; air traffic control, pilot warning, and guidance systems; displays for the visually-impaired; distance learning; and professional sound and picture editing for television and film. In this paper we examine signal processing issues that pertain to immersive audio rendering over loudspeakers. We propose two methods that can be used to implement the necessary filters for generating virtual sound sources based on synthetic head-related transfer functions with the same spectral characteristics as those of the real source. Furthermore, several factors are presented that pertain to high quality immersive audio reproduction at the desktop including acoustical and psychoacoustical considerations and the importance of dynamically adapting to listener’s head movement.

Jong-Soong Lim

Papers

Signal Processing, Acoustics, and Psychoacoustics for High Quality Desktop Audio

Head-related transfer function synthesis for immersive audio

Virtual Loudspeaker Rendering for Multiple Listeners

Signal Processing Considerations for Immersive Audio Rendering