For two singers in a duet from one song

Page 50 of 228 · Open the original GDoc ↗

(use on already separated vocals)

- MVSep SATB Choir (in case of crossbleeding “when i put them in the DAW and inverted it gave really satisfactory results”, worked for two main male vocals where the below failed - lekt0rs)

- becruily & frazer BS Karaoke (sometimes can separate even 3 singers if the vocals aren't completely glued into one, especially if it's male and female - maxerv19,

"It's getting better than MedleyVox" - ryanz48
for male/female separation “Is the best at this” - maxerv19, even better than the MVSEP’s)

- Becruily Mel Karaoke
- Dry Paint Dealer Undr MelBand Roformer Duet model (Singer 1 and Singer_2) | MK Colab fork

For RTX 5000 patch and torch._dynamo error, edit “use_torch_checkpoint: True” to False (or delete the line).

“only works on isolated vocals”, “It's not perfect but it can work, although how well it works really varies from song to song. I was originally going to hold off on releasing this to see if I could get it any better but I saw people wanted a model like this and thought I should probably just release it now.” - dpdu
“Surprisingly good. The model has slight bleed from the backing vocals but less than the karaoke models. Well, about the same. Still has quite a bit which is what i was expecting” - isling

- MedleyVox (trained by Cyrus, 24kHz SR - use e.g. AudioSR afterwards, MVSEP, Colab, local installation tutorial [use vocals 238 model], more info)

“best at it, but not guaranteed to work always. 5% chances of perfectly separating duets on MedleyVox, or else it always false detects and switches back and forth” “also does a pretty good job on other solo instruments”

- MVSEP Multispeaker model (Experimental section at the bottom)
Works well for rap overlapped with singing in one already separated vocal stem.

“Seems very picky with audio, most of the songs/files I tried didn't work

MedleyVox works on the other side (regardless that it's of lower quality)” - becruily

- MVSEP Male/Female:

a) Mel-Roformer Male/Female separation model 13.03 SDR

a) SCNet XL Male/Female separation model on MVSEP (same model base)

SDR on the same dataset: 11.8346 vs 6.5259 (Sucial)

Sometimes the old Sucial model might still do a better job at times, so feel free to experiment. Also becruily & frazer BS Karaoke can be better.

- Aufr33 BS-Roformer Male/Female beta model | config | MK Colab | x-minus (uses Kim-Mel-Band-Roformer-FT2 as preprocessor) | MVSEP

(based on BS-RoFormer Chorus Male Female by Sucial) SDR 8.18

- Male/female BS-Roformer model by Sucial | config for UVR | tensor match error fix
If they sing at intervals [one by one - not together] they cannot be separated. | MVSEP

- MVSep Karaoke BS-Roformer by MVSep team (works for double-tracked vocals)

- Mel-Roformer Karaoke on x-minus.pro (model files in Karaoke)

- bs_karaoke_3stem_giantailab (model | yaml | for use with this bs_roformer.py file for MSST | mvsepless: HF

Works for people talking over vocals where karaoke models failed and only extracted backing and lead vocals where the speech landed in the latter.

- MDX-UVR Karaoke models

- VR's 5_HP or ev. 6_HP in UVR

- BVE v2 on x-minus/uvronline (already uses voc_ft as preprocessor for separating vocals)

It might be still not enough, then continue and/or look for Dolby Atmos rip and retry; works for several backing vocals when lead vocal panning is set to center, “then running the bgv through bve v2 again, but this time set lead vocal panning to 80% but be aware the lead vocal quality will not be that good with this model” - Isling)

- Dry Paint Dealer Undr’s Melband Roformer and Demucs Lead and Rhythm guitar models

- duet-svs-diffusion (“mono/16kHz, 24kHz sample rate, and quality is lower than MedleyVox models”)

- RipX (paid)

- Melodyne (paid, “with polyphonic mode, with a lot of manual finetuning in the detection tab, and this can only work if the voices are not on same pitch”).

- SpectraLayers 11 (but it’s mainly dedicated for voice, not singing)

Spectral painting:

- ISSE (free, you can figure out which voice is who's just by frequencies alone; use on e.g. separated vocals too)

- RX Editor’s brush (video by Bas)

- Audacity (image) less effective

- Ampter

- Filter-Artist

- AudioPaint

Notes

If artists sing the same notes, Karaoke models will rather not work in this case.

(probably both already outperformed by Roformers in all cases:)

If BVs are heard in the center, don't use the MDX karaoke model but the VR karaoke model instead.

Use the chain algorithm with mdx (kar) v2 on x-minus which will use uvr (kar) v2 to solve the issue. It will be available after you process the song with MDX. (Aufr33/dca)

“The MDX models seem to have a cleaner separation between lead and backing/background vocals [vs VR], but they often don't do any actual separation, meanwhile the VR models are less clean, but they seem to be better at detecting lead and background”

“MDX models basically require the lead to be completely center and the BV to be stereo

whereas VR ones don't really care as much about stereo placement”

You could also ask playdasegunda/play da primeira/viperx for separation, as he has some decent private method/models for double vocals better than becruily model (although the latter can be still close), although newer frazer & becruily model is “no way inferior to the ViperX (Play da Segunda). The rumour says, viperx model on Play da Segunda was trained on 40GB dataset allegedly (it would be small), and the model can be bought for 500$ when you contact him via email. It's actually a set of models used for the final inference.
For the record, the open-sourced model by the becruily/frazer duo cost 600$ on compute and probably used a bigger dataset, achieved a bit smaller SDR, but it’s a single model.

For research:

“These archs are [...] really promising for multiple speakers separation, and should be working for multiple singers separation if trained on singing voice:

https://github.com/dmlguq456/SepReformer (current SOTA)

https://github.com/JusperLee/TDANet

https://github.com/alibabasglab/MossFormer2”

> Separating two main vocals

E.g. one panned about 30% left and the other right

- “use bve v2 and click the “lead vocal panning” button” on x-minus premium

For vocals with vocoder

(although lots of newer models were released since then)

- voc_ft

Alternatively, you can use:

- 5HP Karaoke (e.g. with aggression settings raised up) or

- Karaoke 2 model (UVR5 or Colabs). Try out separating the result obtained with voc_ft as well.