Speech enhancement and separation are core problems in audio signal processing, with commercial applications in devices as diverse as mobile phones, conference call systems, hands-free systems, or hearing aids. In addition, they are crucial preprocessing steps for noise-robust automatic speech and speaker recognition. Many devices now have two to eight microphones. The enhancement and separation capabilities offered by these multichannel interfaces are usually greater than those of single-channel interfaces. Research in speech enhancement and separation has followed two convergent paths, starting with microphone array processing and blind source separation, respectively. These communities are now strongly interrelated and routinely borrow ideas from each other. Yet, a comprehensive overview of the common foundations and the differences between these approaches is lacking at present. In this paper, we propose to fill this gap by analyzing a large number of established and recent techniques according to four transverse axes: 1 the acoustic impulse response model, 2 the spatial filter design criterion, 3 the parameter estimation algorithm, and 4 optional postfiltering. We conclude this overview paper by providing a list of software and data resources and by discussing perspectives and future trends in the field.

/pdf/a-consolidated-perspective-on-multimicrophone-speech-qnbaqxmtvq.pdf

A Consolidated Perspective on Multimicrophone Speech Enhancement and Source Separation

As machines become more and more portable, and part of our everyday life, it becomes apparent that developing interactive and ubiquitous systems is an important aspect of new music applications created by the research community. We are interested in developing a robust layer for the automatic annotation of audio signals, to be used in various applications, from music search engines to interactive installations, and in various contexts, from embedded devices to audio content servers. We propose adaptations of existing signal processing techniques to a real time context. Amongst these annotation techniques, we concentrate on low and mid-level tasks such as onset detection, pitch tracking, tempo extraction and note modelling. We present a framework to extract these annotations and evaluate the performances of different algorithms. The first task is to detect onsets and offsets in audio streams within short latencies. The segmentation of audio streams into temporal objects enables various manipulation and analysis of metrical structure. Evaluation of different algorithms and their adaptation to real time are described. We then tackle the problem of fundamental frequency estimation, again trying to reduce both the delay and the computational cost. Different algorithms are implemented for real time and experimented on monophonic recordings and complex signals. Spectral analysis can be used to label the temporal segments; the estimation of higher level descriptions is approached. Techniques for modelling of note objects and localisation of beats are implemented and discussed. Applications of our framework include live and interactive music installations, and more generally tools for the composers and sound engineers. Speed optimisations may bring a significant improvement to various automated tasks, such as automatic classification and recommendation systems. We describe the design of our software solution, for our research purposes and in view of its integration within other systems.

/pdf/automatic-annotation-of-musical-audio-for-interactive-2t6jfn3vud.pdf

Automatic annotation of musical audio for interactive applications

The development of a geometric invariant audio watermarking scheme without degrading acoustical quality is challenging work. This paper proposes a multi-bit spread-spectrum audio watermarking scheme based on a geometric invariant log coordinate mapping (LCM) feature. The LCM feature is very robust to audio geometric distortions. The watermark is embedded in the LCM feature, but it is actually embedded in the Fourier coefficients which are mapped to the feature via LCM, so the embedding is actually performed in the DFT domain without interpolation, thus eliminating completely the severe distortion resulted from the non-uniform interpolation mapping. The watermarked audio achieves high auditory quality in both objective and subjective quality assessments. A mixed correlation between the LCM feature and a key-generated PN tracking sequence is proposed to align the log-coordinate mapping, thus synchronizing the watermark efficiently with only one FFT and one IFFT. Both the theoretical analysis and experimental results show that the proposed audio watermarking scheme is not only resilient against common signal processing operations, including low-pass filtering, MP3 recompression, echo addition, volume change, normalization, test functions in the Stirmark benchmark, and DA/AD conversion, but also has conquered the challenging audio geometric distortion and achieves the best robustness against simultaneous geometric distortions, such as pitch invariant time-scale modification (TSM) by ±20%, tempo invariant pitch shifting by 20%, resample TSM with scaling factors between 75% and 140%, and random cropping by 95%. This is mainly contributed by the proposed geometric invariant LCM feature. To our best knowledge, audio watermarking based on LCM has not been reported before.

Geometric Invariant Audio Watermarking Based on an LCM Feature

After covering the basics of sound perception and giving an overview of commonly used audio effects (using a perceptual categorization), we propose a new concept called adaptive digital audio effects (A-DAFx). This consists of combining a sound transformation with an adaptive control. To create A-DAFx, low-level and perceptual features are extracted from the input signal, in order to derive the control values according to specific mapping functions. We detail the implementation of various new adaptive effects and give examples of their musical use

/pdf/adaptive-digital-audio-effects-a-dafx-a-new-class-of-sound-19xvpfxsqy.pdf

Adaptive digital audio effects (a-DAFx): a new class of sound transformations

Audio equalization is a vast and active research area. The extent of research means that one often cannot identify the preferred technique for a particular problem. This review paper bridges those gaps, systemically providing a deep understanding of the problems and approaches in audio equalization, their relative merits and applications. Digital signal processing techniques for modifying the spectral balance in audio signals and applications of these techniques are reviewed, ranging from classic equalizers to emerging designs based on new advances in signal processing and machine learning. Emphasis is placed on putting the range of approaches within a common mathematical and conceptual framework. The application areas discussed herein are diverse, and include well-defined, solvable problems of filter design subject to constraints, as well as newly emerging challenges that touch on problems in semantics, perception and human computer interaction. Case studies are given in order to illustrate key concepts and how they are applied in practice. We also recommend preferred signal processing approaches for important audio equalization problems. Finally, we discuss current challenges and the uncharted frontiers in this field. The source code for methods discussed in this paper is made available at https://code.soundsoftware.ac.uk/projects/allaboutaudioeq.

/pdf/all-about-audio-equalization-solutions-and-frontiers-550t19t9nq.pdf

All About Audio Equalization: Solutions and Frontiers

From the Publisher:
Digital Audio Effects stand for the highest possible sound quality and the finest level of control in the modern world of music and sound.
Digital Audio Effects (DAFX) is also the name chosen for the European Research Project COST G6 which investigates the use of digital signal processing, its application to sounds, and its musical use designed to put effects on a sound. The aim of the project and this book is to present the main fields of digital audio effects. It systematically introduces the reader to digital signal processing concepts as well as software implementations using MATLAB. Highly acclaimed contributors analyse the latest findings and developments in filters, delays, modulators, and time-frequency processing of sound. 
Features include: 
Chapters on time-domain, non-linear, time-segment, time-frequency, source-filter, spectral, bitstream signal processing; spatial effects, time and frequency warping and control of DAFX. 
MATLAB implementations throughout the book illustrate essential DSP algorithms for sound processing. 
Accompanying website with sound examples available 
The approach of applying digital signal processing to sound will appeal to sound engineers as well as to researchers and engineers in the field of signal processing. 
DAFX - Digital Audio Effects features contributions from Daniel Arfib, Xavier Amatrain, Jordi Bonada, Giovanni de Poli, Pierre Dutilleux, Gianpaolo Evangelista, Florian Keiler, Alex Loscos, Davide Rocchesso, Mark Sandler, Xavier Serra, and Todor Todoroff.

/pdf/dafx-digital-audio-effects-19nh43ug3t.pdf

Dafx : Digital Audio Effects

Equalizers for audio signal processing play an important role in microphones, mixing systems, mastering processors, monitor loudspeakers, and consumer devices such as smart tablets, smart phones, wireless headphones, and hearing devices. We will introduce low-complexity filter designs for equalizers based on first- and second order recursive filters. Low-complexity filter design means having the lowest number of parameters and design equations for first- and second-order difference equation realizing prescribed filter transfer functions such as low-pass, high-pass, band-pass, and all-pass filters. Based on these basic filters, we derive all-pass realizations of these standard filters and then apply them to design parametric low- and high-frequency shelving filters and peak filters. These last two versions of weighting filters are based on three parameters, namely, the cut-off or center frequency, the bandwidth or Q factor, and the gain in dB for a low-, mid-, or high frequency band and are, therefore, named parametric equalizers. These parametric equalizers' PEQs occur in a channel-strip of a mixing console and are an integral part of the mixing process. We will describe several further applications of PEQs for amp and loudspeaker modeling, headphone equalization and head-related transfer function modeling, and audio coding. All applications allow a low-complexity filter design, parameter update, and efficient recursive filter implementations with much lower complexity compared to non-recursive filters.

Udo Zoelzer

Papers

Dafx : Digital Audio Effects

Low-complexity equalizers and applications