Open AccessPosted Content
Music Source Separation in the Waveform Domain
Reads0
Chats0
TLDR
Demucs is proposed, a new waveform-to-waveform model, which has an architecture closer to models for audio generation with more capacity on the decoder, and human evaluations show that Demucs has significantly higher quality than Conv-Tasnet, but slightly more contamination from other sources, which explains the difference in SDR.Abstract:
Source separation for music is the task of isolating contributions, or stems, from different instruments
recorded individually and arranged together to form a song. Such components include voice, bass, drums and any other accompaniments.
Contrarily to many audio synthesis tasks where the best performances are achieved by models that directly generate the waveform, the state-of-the-art in source separation for music is to compute masks on the magnitude spectrum.
In this paper, we compare two waveform domain architectures. We first adapt Conv-Tasnet, initially developed for speech source separation,
to the task of music source separation. While Conv-Tasnet beats many existing spectrogram-domain methods, it suffers
from significant artifacts, as shown by human evaluations. We propose instead Demucs, a novel waveform-to-waveform model,
with a U-Net structure and bidirectional LSTM.
Experiments on the MusDB dataset show that, with proper data augmentation, Demucs beats all
existing state-of-the-art architectures, including Conv-Tasnet, with 6.3 SDR on average, (and up to 6.8 with 150 extra training songs, even surpassing the IRM oracle for the bass source).
Using recent development in model quantization, Demucs can be compressed down to 120MB
without any loss of accuracy.
We also provide human evaluations, showing that Demucs benefit from a large advantage
in terms of the naturalness of the audio. However, it suffers from some bleeding,
especially between the vocals and other source.read more
Citations
More filters
Proceedings ArticleDOI
Real Time Speech Enhancement in the Waveform Domain.
TL;DR: Empirical evidence shows that the proposed causal speech enhancement model, based on an encoder-decoder architecture with skip-connections, is capable of removing various kinds of background noise including stationary and non-stationary noises, as well as room reverb.
Journal ArticleDOI
Spleeter: a fast and efficient music source separation tool with pre-trained models
TL;DR: The performance of the pre-trained models are very close to the published state-of-the-art and is one of the best performing 4 stems separation model on the common musdb18 benchmark to be publicly released.
Posted Content
Voice Separation with an Unknown Number of Multiple Speakers
TL;DR: A new method is presented for separating a mixed audio sequence, in which multiple voices speak simultaneously, that greatly outperforms the current state of the art, which, as it is shown, is not competitive for more than two speakers.
Proceedings ArticleDOI
Sudo RM -RF: Efficient Networks for Universal Audio Source Separation
TL;DR: The backbone structure of this convolutional network is the SUccessive DOwnsampling and Resampling of Multi-Resolution Features (SuDoRM-RF) as well as their aggregation which is performed through simple one-dimensional convolutions.
Journal ArticleDOI
High Fidelity Neural Audio Compression
TL;DR: In this article , the authors propose a loss balancer mechanism to stabilize training, where the weight of a loss now defines the fraction of the overall gradient it should represent, thus decoupling the choice of this hyper-parameter from the typical scale of the loss.
References
More filters
Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
Book ChapterDOI
U-Net: Convolutional Networks for Biomedical Image Segmentation
TL;DR: Neber et al. as discussed by the authors proposed a network and training strategy that relies on the strong use of data augmentation to use the available annotated samples more efficiently, which can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks.
Proceedings Article
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Sergey Ioffe,Christian Szegedy +1 more
TL;DR: Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin.
Posted Content
U-Net: Convolutional Networks for Biomedical Image Segmentation
TL;DR: It is shown that such a network can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks.
Posted Content
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Sergey Ioffe,Christian Szegedy +1 more
TL;DR: Batch Normalization as mentioned in this paper normalizes layer inputs for each training mini-batch to reduce the internal covariate shift in deep neural networks, and achieves state-of-the-art performance on ImageNet.