2017 25th European Signal Processing Conference (EUSIPCO)

Computational Science and Engineering (CSE) is the multi-disciplinary field of computer-based modelling and simulation for studying scientific phenomena and engineering designs. Modelling and simulation helps to validate theory, and makes it possible to analyse scenarios that would otherwise be too time-consuming, expensive, or dangerous to study by experiment. Data exploration helps to turn numbers into insight which is especially challenging in times of Big Data.

Computational Science and Engineering

This paper reports the organization and results for the 2018 community-based Signal Separation Evaluation Campaign (SiSEC 2018). This year’s edition was focused on audio and pursued the effort towards scaling up and making it easier to prototype audio separation software in an era of machine-learning based systems. For this purpose, we prepared a new music separation database: MUSDB18, featuring close to 10 h of audio. Additionally, open-source software was released to automatically load, process and report performance on MUSDB18. Furthermore, a new official Python version for the BSS Eval toolbox was released, along with reference implementations for three oracle separation methods: ideal binary mask, ideal ratio mask, and multichannel Wiener filter. We finally report the results obtained by the participants.

The 2018 Signal Separation Evaluation Campaign

Perceiving a scene most fully requires all the senses. Yet modeling how objects look and sound is challenging: most natural scenes and events contain multiple objects, and the audio track mixes all the sound sources together. We propose to learn audio-visual object models from unlabeled video, then exploit the visual context to perform audio source separation in novel videos. Our approach relies on a deep multi-instance multi-label learning framework to disentangle the audio frequency bases that map to individual visual objects, even without observing/hearing those objects in isolation. We show how the recovered disentangled bases can be used to guide audio source separation to obtain better-separated, object-level sounds. Our work is the first to learn audio source separation from large-scale “in the wild” videos containing multiple audio sources per video. We obtain state-of-the-art results on visually-aided audio source separation and audio denoising. Our video results: http://vision.cs.utexas.edu/projects/separating_object_sounds/.

/pdf/learning-to-separate-object-sounds-by-watching-unlabeled-3vla02iggj.pdf

Learning to Separate Object Sounds by Watching Unlabeled Video

/pdf/learning-to-separate-object-sounds-by-watching-unlabeled-5czfgsy9q8.pdf

The fast Fourier transform is an algorithm of paramount importance in signal processing as it allows to apply the Fourier transform in $\mathcal {O}(n \log n)$ instead of $\mathcal {O}(n^2)$ arithmetic operations. Graph signal processing is a recent research domain that generalizes classical signal processing tools, such as the Fourier transform, to situations where the signal domain is given by any arbitrary graph instead of a regular grid. Today, there is no method to rapidly apply graph Fourier transforms. In this paper, we propose a method to obtain approximate graph Fourier transforms that can be applied rapidly and stored efficiently. It is based on a greedy approximate diagonalization of the graph Laplacian matrix, carried out using a modified version of the famous Jacobi eigenvalues algorithm. The method is described and analyzed in detail, and then applied to both synthetic and real graphs, showing its potential.

https://hal.inria.fr/hal-01416110/document

Approximate Fast Graph Fourier Transforms via Multilayer Sparse Approximations

The computational cost of many signal processing and machine learning techniques is often dominated by the cost of applying certain linear operators to high-dimensional vectors. This paper introduces an algorithm aimed at reducing the complexity of applying linear operators in high dimension by approximately factorizing the corresponding matrix into few sparse factors. The approach relies on recent advances in nonconvex optimization. It is first explained and analyzed in details and then demonstrated experimentally on various problems including dictionary learning for image denoising and the approximation of large matrices arising in inverse problems.

Flexible Multilayer Sparse Approximations of Matrices and Applications

We consider a single-channel source separation problem consisting in separating speech from nonstationary background such as music. We introduce a novel approach called text-informed separation, where the source separation process is guided by the corresponding textual information. First, given the text, we propose to produce a speech example via either a speech synthesizer or a human. We then use this example to guide source separation and, for that purpose, we introduce a new variant of the nonnegative matrix partial co-factorization (NMPCF) model based on a so called excitation-filter-channel speech model. The proposed NMPCF model allows sharing the linguistic information between the example speech and the speech in the mixture. We then derive the corresponding multiplicative update (MU) rules for the parameter estimation. Experimental results over different types of mixtures and speech examples show the effectiveness of the proposed approach.

/pdf/text-informed-audio-source-separation-using-nonnegative-25180v86ab.pdf

Text-informed audio source separation using nonnegative matrix partial co-factorization

Dictionary learning aims at finding a frame (called dictionary) in which some training data admits a sparse representation. Traditional dictionary learning is limited to relatively small-scale problems, because high-dimensional dense dictionaries can be costly to manipulate, both at the learning stage and when used for tasks such as sparse coding. In this paper, inspired by usual fast transforms, we consider a multi-layer sparse dictionary structure allowing cheaper manipulation, and propose a learning algorithm imposing this structure. The approach is demonstrated experimentally with a factorization of the Hadamard matrix and on image denoising.

/pdf/chasing-butterflies-in-search-of-efficient-dictionaries-3ikqm04xm6.pdf

Chasing butterflies: In search of efficient dictionaries

The so-called informed audio source separation, where the separation process is guided by some auxiliary information, has recently attracted a lot of research interest since classical blind or non-informed approaches often do not lead to satisfactory performances in many practical applications. In this paper we present a novel text-informed framework in which a target speech source can be separated from the background in the mixture using the corresponding textual information. First, given the text, we propose to produce a speech example via either a speech synthesizer or a human. We then use this example to guide source separation and, for that purpose, we introduce a new variant of the non-negative matrix partial co-factorization (NMPCF) model based on a so-called excitation-filter-channel speech model. Such a modeling allows sharing the linguistic information between the speech example and the speech in the mixture. The corresponding multiplicative update (MU) rules are eventually derived for the parameters estimation and several extensions of the model are proposed and investigated. We perform extensive experiments to assess the effectiveness of the proposed approach in terms of source separation and alignment performance.

/pdf/text-informed-audio-source-separation-example-based-approach-bladg203q4.pdf

Luc Le Magoarou

Papers

Approximate Fast Graph Fourier Transforms via Multilayer Sparse Approximations

Flexible Multilayer Sparse Approximations of Matrices and Applications

Text-informed audio source separation using nonnegative matrix partial co-factorization

Chasing butterflies: In search of efficient dictionaries

Text-Informed Audio Source Separation. Example-Based Approach Using Non-Negative Matrix Partial Co-Factorization