In their testing, Deezer have developed new software called Spleeter that uses AI to isolate the vocals from a record, resulting in two separate tracks at a minimum – a instrumental and acapella. The now open source software is available to all, however it will not be for everyone – some expertise is necessary.
Yesterday the company released it as an open-source package, putting the code up on Github for anyone to download and use. Just feed Spleeter an audio file and it splits it into two, four, or five separate audio tracks known as stems. The results aren’t perfect but they are eminently usable and Spleeter itself is very fast. When running on a dedicated GPU it can split audio files into four stems 100 times faster than real time.
Deezer notes that this is not the first time people have used machine learning to automate this particular task, and that the company has built on a lot of earlier research. Deezer’s chief data and research officer Aurelien Herault says the company trained its software on 20,000 musical tracks with pre-isolated vocals in a range of genres. From this information, the software learned how to isolate the tracks itself.
Deezer says it has no plans to turn Spleeter into a consumer tool, but others could take their work and slap a simple interface on it. The obvious applications are for DJs and producers looking to integrate isolated vocals into mixes, or for people looking to create homebrew karaoke backing tracks. (Such activities might not be in compliance with copyright law depending on how the final product is distributed.)
Deezer itself uses Spleeter for a range of research applications that help improve its streaming service. “Internally, we’re using it as a pre-processing tool for complex research tasks such as music categorization, transcription and language detection,” says Herault.
