Spleeter: a fast and ecient music source separation

tool with pre-trained models

Romain Hennequin

1

, Anis Khlif

1

, Felix Voituret

1

, and Manuel

Moussallam

1

1 Deezer Research, Paris

DOI: 10.21105/joss.02154

Software

Submitted: 06 March 2020

Published: 24 June 2020

License

Authors of papers retain

copyright and release the work

under a Creative Commons

Attribution 4.0 International

License (CC BY 4.0).

Summary

The performance of the pre-trained models are very close to the published state-of-the-art and

is one of the best performing 4 stems separation model on the common musdb18 benchmark

(Rai, Liutkus, Stöter, Mimilakis, & Bittner, 2017) to be publicly released. Spleeter is also

very fast as it can separate a mix audio le into 4 stems 100 times faster than real-time (we

note, though, that the model cannot be applied in real-time as it needs buering) on a single

Graphics Processing Unit (GPU) using the pre-trained 4-stems model.

Purpose

has reached a level of maturity that makes it worth considering for these tasks and that spe-

cic features computed from isolated vocals, drums or bass may help increase performances,

especially in low data availability scenarios (small datasets, limited annotation availability) for

which supervised learning might be dicult. Spleeter also makes it possible to ne-tune the

provided state-of-the-art models in order to adapt the system to a specic use-case. Finally,

having an available source separation tool such as Spleeter will allow researchers to compare

performances of their new models to a state-of-the-art one on their private datasets instead

of musdb18, which is usually the only used dataset for reporting separation performances for

unreleased models. Note that we cannot release the training data for copyright reasons, and

thus, sharing pre-trained models were the only way to make these results available to the

The pre-trained models are U-nets (Jansson et al., 2017) and follow similar specications as

in (Prétet, Hennequin, Royo-Letelier, & Vaglio, 2019). The U-net is an encoder/decoder

Convolutional Neural Network (CNN) architecture with skip connections. We used 12-layer

U-nets (6 layers for the encoder and 6 for the decoder). A U-net is used for estimating a

soft mask for each source (stem). Training loss is a L

-norm between masked input mix

on Central Processing Unit (CPU) or GPU.

Speed

As the whole separation pipeline can be run on a GPU and the model is based on a CNN,

computations are eciently parallelized and model inference is very fast. For instance, Spleeter

is able to separate the whole musdb18 test dataset (about 3 hours and 27 minutes of audio)

into 4 stems in less than 2 minutes, including model loading time (about 15 seconds), and

audio wav les export, using a single GeForce RTX 2080 GPU, and a double Intel Xeon Gold

6134 CPU @ 3.20GHz (CPU is used for mix les loading and stem les export only). In this

setup, Spleeter is able to process 100 seconds of stereo audio in less than 1 second, which

makes it very useful for eciently processing large datasets.

in the following table compared to Open-Unmix (Stöter, Uhlich, Liutkus, & Mitsufuji, 2019)

and Demucs (Défossez, Usunier, Bottou, & Bach, 2019) (only SDR are reported for Demucs

since other metrics are not available in the paper) which are, to the authors’ knowledge, the

only released system that performs near state-of-the-art performances. We present results

for soft masking and for multi-channel Wiener ltering (applied using Norbert (Liutkus &

Stöter, 2019)). As can be seen, for most metrics Spleeter is competitive with Open-Unmix

and especially on SDR for all instruments, and is almost on par with Demucs.

Spleeter Mask Spleeter MWF Open-Unmix Demucs

Vocals SDR 6.55 6.86 6.32 7.05

Vocals SIR 15.19 15.86 13.33 13.94

Bass ISR 9.18 9.61 9.23 9.99

Drums SDR 5.93 6.71 5.73 7.08

Drums SIR 12.24 13.67 11.12 13.74

Drums SAR 5.78 6.54 6.02 7.04

Drums ISR 10.50 10.69 10.51 11.96

Other SDR 4.24 4.55 4.02 4.47

Other SIR 7.86 8.16 6.59 7.11

Other SAR 4.63 4.88 4.74 5.26

Other ISR 9.83 9.87 9.31 10.86

Spleeter (Hennequin, Khlif, Voituret, & Moussallam, 2019) source code and pre-trained models

Acknowledgements

We acknowledge contributions from Laure Pretet who trained rst models and wrote the rst

piece of code that lead to Spleeter.

References

Abadi, M. et a. (2015). TensorFlow: Large-scale machine learning on heterogeneous systems.

Retrieved from https://www.tensorflow.org/

Défossez, A., Usunier, N., Bottou, L., & Bach, F. (2019). Music source separation in the

waveform domain. Retrieved from http://arxiv.org/abs/1911.13254

Hennequin, R., Khlif, A., Voituret, F., & Moussallam, M. (2019). Spleeter. Retrieved from

https://www.github.com/deezer/spleeter

doi:10.5281/zenodo.3269749

Prétet, L., Hennequin, R., Royo-Letelier, J., & Vaglio, A. (2019). Singing voice separation: A

study on training data. In ICASSP 2019 - 2019 ieee international conference on acoustics,

MUSDB18 corpus for music separation. doi:10.5281/zenodo.1117372

Stöter, F.-R., Liutkus, A., & Ito, N. (2018). The 2018 signal separation evaluation campaign.

In Latent variable analysis and signal separation. LVA/ICA. Springer, Cham. doi:10.1007/

978-3-319-93764-9_28

Stöter, F.-R., Uhlich, S., Liutkus, A., & Mitsufuji, Y. (2019). Open-unmix - a reference

implementation for music source separation. Journal of Open Source Software. doi:10.

21105/joss.01667