Fast inference models for general use
Above an hour on i3-7100u, small - the lightest Roformers, while most used to be 870 MB):
For vocals
- Unwa Resurrection BS-Roformer (yaml | Colab, 195 MB)
- BS-Roformer SW vocals only (mask_estimators.0 on the regular 6 stem model, 195 MB)
Models like BS_RoFormer_mag or Anvuew BS-Roformer 12.45 also have similar size, although not all small size models have to be similarly fast like above, but feel free to test (e.g. hyperacev2 is much slower than the Resurrection).
- Vocals from MVSep 53 stem model (single model | yaml, 77 MB)
(it’s even smaller, but its performance might be mediocre compared to the above)
Older models
- Aname Mel-Roformer small (203MB)
- Unwa Mel-Roformer small (203MB)
Older arch (faster; 25-60 minutes+ on weak i3u/C2Q respectively)
- voc_ft (probably the fastest, but uses outperformed MDX-Net v2 arch, also it’s narrowband)
- Kim Vocal 2 (or ev. 1, -||-, older model)
For instrumentals
- Unwa BS-Roformer Resurrection inst (yaml) | a.k.a. “unwa high fullness inst" on MVSEP | uvronline free/premium | Colab | UVR (don’t confuse with Resurrection vocals variant, 204 MB)
- Gabox BS_ResurrectioN (model | yaml, 204 MB)
For both
(dual stem model, it don’t invert - you might save time instead of using two models)
- Becruily Mel Deux (decent vocals and instrumentals, although sometimes bleedy)
—
Older models
- Unwa BS-Roformer-Inst-FNO (works only in MSST after modifying py file like in the model card, similar to decently performing Resurrection inst model, 332 MB)
- Gabox Mel-Roformer small_inst | yaml (experimental, 203 MB)
- Unwa BS-Roformer-Inst-EXP-Value-Residual (uses Mel v2 model type in UVR; If it wasn’t made compatible with MSST already, replace bs_roformer.py from this repo and
from bs_roformer.attend import attend
⇩
from models.bs_roformer.attend import attend
in bs_roformer.py file
generally not very good model, but sometimes capable: “successfully removed [vocals] and kept the digital choir atmosphere as well” vs deux, inst_gaboxFlowersV10 and HyperACE but it it’s considerably slower than deux - mohammedmehditber)
—-
4 stems
- Faster FP16 version of BS-Roformer 6 stems called splifft (by undef13; a tad lower SDR; only 334MB vs 700 MB in the OG weight, CPU/NVIDIA compatible, and potentially AMD ROCm, only bigger variant works in UVR; the OG “Conversion done after 2 hours for a 2 minute 49 second file” on 2/4 i3 7100u) - on CPU it might be slower than the OG, as it might not support FP16 natively due to even possible emulation. But probably Turing GPUs with tensors (e.g. RTX or T4) and newer, probably have FP16 acceleration, while non-RTX 16XX sometimes not.
Faster, lower quality:
- KUIELab-MDXNET23C (4 stems) - its first scores were probably from ensemble of its five models, and in that configuration it had better SDR than demucs_ft on its own, and drums had better SDR than “SCNet-large_starrytong” (so single models’ score of any of these MDX23C models is probably lower than in demucs_ft).
> Lighter “model1” drums sound surprisingly better than htdemucs non_ft v4 on previously separated instrumental. It handles trap really well and preserves hi-hats correctly, but at the cost of other stem bleeding. v4 model can be used to clean it a bit further,
- htdemucs v4 non-ft (UVR default) - it can clean up other stem bleeding of the above
The fastest models, usually lower quality, CPU-friendly.
If you only use CPU or CUDA, you can use this UVR fork of legacy non-Roformer version 5.6, separating the below MDX-Net V2 models up to 4 times faster with half of the model usage:
https://github.com/revlaydust/rld-uvr-opt/
Instrumentals
- MDX-Net HQ_3, 4, 5 (the last is the fastest, 56 MB)
- MDX-Net inst3, Kim inst (older, narrowband models, but can be useful too in some cases, 63 MB)
Vocals
- MDX-Net voc_ft
- MDX-Net Kim vocal 2/1
(all narrowband models)
4 stems
- htdemucs_mmi - iirc the fastest demucs (v3) model, but worse quality than the others
- kuielab_b - MDX-Net, lighting-fast, but quality is mediocre (but rather still better than Spleeter which might be even faster, but not necessarily)
________________
Older vocal models for general use (moved here for archiving purposes)
- Mel-Roformer unwa’s inst-voc model called “duality v1/2” (focused on both instrumental and vocal stem during training, but you can now test newer V1e+ single stem for this purpose too).
https://huggingface.co/pcunwa/Mel-Band-Roformer-InstVoc-Duality | Colab | MVSEP
Vocals sound similar to beta 4 model, but with more noise,
instrumentals are deprived of the noise present in inst v1 and later inst models, but as a downside, they’re more muddy for instrumentals.
v2 have slightly a bit better SDR and fewer residues
Because duality is a two stems target model.
"other" is output from model
"Instrumental" is inverted vocals against input audio.
The latter has lower SDR and more holes in the spectrum.
So, using MSST-GUI, leave the checkbox “extract instrumental” disabled for duality models.
You can use it in the Bas Curtiz’ GUI for ZFTurbo script (already added) or with the OG ZF’s repo, or in the Colab.
- Aname duality Mel model
- Aname Full Scratch Mel-Band Roformer model
bleedless 30.75 fullness 13.24, SDR: 8.01
- SYHFT (a.k.a. SYH99999/yukunelatyh) MelBandRoformer V3 | model
VS previous SYH’s models “this version is more consistent with separation. It's not what I'd call a clean model; It sometimes lets background noise bleed into the vocal stem. But only somewhat, and depending on how you look at it, it can be a good thing since it makes the vocals sound less muddy.” Musicalman
- MelBandRoformerBigSYHFTV1Fast | model - more vocal fullness metric, but more bleeding (although less than duality models and even Kim’s purely metric-wise). “same parameters size with Kim's. Other models are 2x scale parameter size to compare my model”
- Mel-Roformer model by Kim | model | config
Vocals bleedless: 36.75, fullness: 16.26, SDR: 11.07
(Colab/Huggingface/2/MVSEP/uvronline via special link for: free/premium (scroll down)/UVR beta Roformer (available in Download Center)/MSST-GUI/simple Colab/CML inference)
Usual base for lots of Mel fine-tunes on that list.
Sometimes might leave instrumental residues in vocals, but can be less muddy than other BS-Roformers - the same goes to any fine-tunes of this model vs BS 2024.08, so effectively all the Mel models above)
“godsend for voice modulated in synth/electronic songs” vs 1296 can be more problematic with wind instruments putting them in vocals.
- unwa’s instrumental Mel-Roformer v1e+
- unwa’s instrumental Mel-Roformer v2 model (similar to v1, but less noise, muddier, bigger, heavier model)
Model files | Colab | uvronline via special link for: free/premium (scroll down) | MSST-GUI (It's now included in ZFTurbo's repo, it's the "gui-wx.py" file)
Might miss some samples or adlibs while cleaning inverts. SDR got a bit bigger (16.845 vs 16.595) “Sounds very similar to v1 but has less noise, pretty good” “the aforementioned noise from the V1 is less noticeable to none at all, depending on the track”. “V2 is more muddy than V1 (on some songs), but less muddy than the Kim model. (...) [As for V1,] sometimes it's better at high frequencies” Aufr33
- older BS-Roformer 2024.02 on MVSEP (generally BS-Roformer models “can be slappy with choir-like vocals and background vocals” but “hot on pre-2000 rock”)
These older Roformers “kinda does poorly on large screams” in metal music, but not always. Sometimes even HQ_4 can catch them better than, e.g. viperx models.
- Mel-Roformer fine-tuned 17.48 model on MVSEP (works e.g. for live shows that have crowd)
(it’s different from the one on x-minus)
- Gabox BS-Roformer instrumental, which doesn’t struggle so much with choirs like most Mel-Roformers, although it may not help in all cases (link)
- “ver. 2024.04” SDR 17.55 on MVSEP - fine-tuned viperx model v1 (can pick in adlibs better, occasionally picks some SFX’, sometimes one, sometimes the other is “slightly worse at pulling out difficult vocals”)
- BS-Roformer Large v1 unwa’s vocal model (viperx 1297 model fine-tuned) download | mirror | yaml | Colab
More muddy than Kim’s Roformer, potentially a bit less of residues, a bit more artificial sound. Better than viperx model - “captures more nuances, subtle elements and details” ~A5
It can be better for some older music like The Beatles than above models.
- BS-Roformer viperx 1297 model (UVR/MVSEP a.k.a. SDR 17.17 model | yaml
- BS-Roformer viperx a.k.a playdasegunda 1296 variant model | yaml
previously called just “BS-Roformer” on uvronline via special link for: free/premium and in nextgen.uvronline.app (legacy)
- Mel-Roformer viperx 1143 vocal model (UVR>Download More Models)
(don't confuse with 1053 which separates drums and bass in one stem).
The first Mel-Roformer vocal model trained by viperx. It was before Kim model which introduced changes to the config, and fixed the problem of lower SDR vs models trained on BS-Roformer.
Most people back then preferred Kim Mel-Roformer instead, but Mel viperx’ “does background voices correctly unlike the Kim's (it does not recognise background 'breee's)” “Iirc Viperx Mel Rofo doesn't struggle with instruments counted as vocals”.
Also, both Mel and BS variants of viperx model struggle with saxophone and e.g. some Arabic guitars. It can still depend on a song whether these are better than even the second oldest Roformer than on MVSEP (from before viperx model got fine-tuned version). Beside problems with recognizing instruments, they're very good for vocals (although Mel-Roformer by Kim on x-minus tends to be better).
Muddy instrumentals when not ensembled with other archs (but we didn’t have typically instrumental stem target models back then), maybe Mel variant less.
Be aware that names of these models on UVR refer to SDR measurements of vocals conducted on private viperx dataset, not even older Synthetic dataset, instead of on multisong dataset on MVSEP, hence the numbers are higher than in the multisong chart on MVSEP.
Older ensembles for vocals
- Models ensembled option on x-minus.pro (available only for premium users)
> Mel-Roformer + MDX23C (can be picked after you uploaded/processed a track [at least with Mel-Roformer model chosen]).
> Mel-Roformer + demudder
“I recommend mel-roformer + demudder to remove vocals from songs that contain only backing vocals that are so faint that our ears can barely hear them.”
- MDX23 by ZFTurbo (v. 2.5 jarredou Colab fork)
- Ensembles on MVSEP.com (for premium users)
- Ensembles in UVR 5:
a) 1296 + 1143 (BS-Roformer in beta UVR) + Inst HQ4 (dopfunk)
(there might be instrumental residues from HQ4 in some cases)
b) 1296 + 1297 + MDX23C HQ
c) Manual ensemble in UVR of models BS-Roformer 1296 + copy of the result + MDX23C HQ (jarredou) - for faster result and similar quality vs the one above
More ensembles beneath
- KaraFan (preset 4, but may give worse results than Mel-Roformer)
___
Older single models for vocals (available in UVR 5 | MDX-Net Colab for non-23C models | MVSEP)
- UVR-MDX-Net-Voc_FT (narrowband, further trained, fine-tuned version of the Kim vocal model; Roformers might be better now)
>If you still have instrumental bleeding, process the result with Kim vocal 2
>Alternatively use MDX23C narrowband (D1581) then Voc-FT, "great combination" (or MDX23C-InstVoc HQ instead of D1581)
(so separate with the D1581 or InstVoc model first, then use the separated result as input, and separate it further with voc_ft)
- Kim Vocal 1 (can bleed less than 2, but more than voc_ft, might depend on a song)
- Kim Vocal 2
>MDX-Net HQ_3/4/5 (HQ_4 can be sometimes not bad on vocals too, even less muddy than voc_ft, though more noisy, and e.g. HQ_3 had more vocal residues then Kim Vocal 2 in general, HQ_5 have stronger and fuller vocals than HQ_4)
>MDX23C-InstVoc HQ (can have some instruments residues at times, but it’s fullband - better clarity vs voc_ft and Kim Vocal 1/2 -
“This new model is [vs the narrowband vocal models], by far, the best in removing the most non-vocal information from an audio and recovering formants from buried passages... But in some cases, it also removes some airy parts from specific words, and some non-verbal sounds (breathing, moaning).”
- newer MDX23C epochs available on MVSEP like 16.66.
MDX23C models are go-to models for live recorded vocals
(available also in MDX23 Colab v2.3/2.4 when weight set only for InstVoc model)
- ZFTurbo’s VitLarge23 (yaml) e.g. on MVSEP or MK Colab or MDX23 2.3/2.4 Colab (as part of an ensemble with older models there) - it's based on a new transformers arch. SDR-wise it's not better than MDX23C (9.78 vs 10.17), but works "great" for an ensemble consisting of two models with weights 2, 1. It was added in one old 4 models ensemble on MVSEP (although it could have changed since then)
Older UVR ensembles (from before Roformer models release)
>Voc FT + MDX23C_D1581 (avg/avg)
>292, 496, 406, 427, Kim Vocal 1, Kim Inst + Demucs ft (#1449)
>Kim Inst, Kim Vocal 1 (or/and voc_ft), Kim Vocal 2, UVR-MDX-NET Inst HQ 2 (or 3/4), UVR-MDX-NET_Main_427, htdemucs_ft (avg/avg IRC)
>Kim Vocal 1+2, MDX23C-InstVoc HQ, UVR-MDX-NET-Voc_FT
(jaredou)
>You can also check some ensembles for instrumentals
Your choice of the best vocal models only (up to 4-5 max for the best SDR - more)
If your separation still bleeds, consider processing it further with models in Debleeding section further below.
___