(old) More descriptions of models
and AIs, with troubleshooting and tips
(most models here are dated as it lacks Roformers)
(Instruction here was moved to Reading advice)
Older models descriptions
- Inst fullband (fb) HQ_3/4/5 x-minus, MVSEP, Colabs
HQ_4 vs 3 has some problems with fadeouts when occasionally it can leave some vocal residues
HQ_3 generally has problems with strings. mdx_extra from Demucs 3/4 had better result with strings here, sometimes 6s model can be good compensation in ensemble for these lost instruments, but HQ_3 gives some extra details compared to those.
HQ_3/4 are generally muddy models at times, but with not much of vocal residues (near Gsep at times, but more than BS-Roformer v2).
For more clarity, use MDX23C HQ model (HQ_2 can have less vocal residues at times).
Another possibly problematic instruments are those wind ones (flute, trumpet etc.)
- use Kim inst or inst 3 then
HQ3 has worse SDR vs:
- voc_ft, but given that HQ_3 is an instrumental model, the latter can leave less vocal residues at times.
https://mvsep.com/quality_checker/leaderboard2.php?id=4029
https://mvsep.com/quality_checker/leaderboard2.php?id=3710
These are SDR results from the same patch, so the voc_ft vs HQ_3 comparison is valid.
- MDX23C_D1581 (narrowband) - usually worse results than voc_ft and probably worse SDR if evaluation for both models was made on the same patch
Can be a bit better for instrumentals
“The new model is very promising
although having noise, it seems to pick vocals more accurately and the instrumentals don't have that much of the filtering effect (where entire frequencies are being muted).”
While others say it’s worse than demucs_ft
- GSEP AI an online closed source service (cannot be installed on your computer or your own site). mp3 only, 20kHz cutoff.
Decent results in some cases, click on the link above to read more about GSEP in the specific section below. This SDR leaderboard underestimates it very much, probably due to some kind of post-processing used in GSEP [probably noise gate and/or slight reverb or chunking). As a last resort, you can use 4-6 stems option and perform mixdown without vocal stem in e.g. Audacity or other DAW. 4-6 stem option has additional noise cancellation vs 2 stem.
GSEP is good with some tracks with a busy mix or acoustic songs where everything else simply fails, or you’re forced to use the RX10 De-bleed feature.
- GSEP is also better than MDX-UVR instrumental models on at least tracks with flute and possibly duduk/clarinet or oriental tracks, and possibly tracks with only piano, as it has a decent dedicated piano model.
- To address the issue with flute using MDX-UVR, use the following ensemble: Kim_Inst, HQ1, HQ2, INST 3, Max Spec/Max Spec (Anjok).
- Sometimes kim inst and inst3 models are less vulnerable to the issue (not in all cases).
- Also, main 406 vocal model keeps most of these trumpets/saxes or other similar instruments
- Passing through a Karaoke model may help a bit with this issue (Mateus Contini method).
- inst HQ_1 (450)/HQ_2 (498)/HQ_3 MDX-UVR fullband models in the Download center of UVR5 - great high quality models to use in most cases. The latter a bit better SDR, possibly a bit less vocal residues. Not so few like inst3 or kim ft other in specific cases, but a good point to start.
What you need to know about MDX-UVR models is that they're divided into instrumental and vocal models and that instrumental models will always leave some instrumental residues in vocals and vice versa - vocal models will more likely to leave some vocal residues in instrumentals. But you can still encounter specific cases of songs when breaking that rule will benefit you - that might depend on the specific song. Usually, instrumental model should give better instrumental if you’re fighting with vocal residues.
Also, MDX-UVR models can sometimes pick up sound midi effects which won’t be recovered.
- kim inst (a.k.a. ft other) - cutoff, cleaner results and better SDR than inst3/464 but tends to be more noisy than inst3 at times. Use:
- inst3/464 - to get more muddy, but less noisy results, although it all depends on a song, and sometimes HQ_1/2/3 models provide generally less vocal residues (or more detestable).
- MDX23 by ZFTurbo v1 - the third place in the newest MDX challenge. 4 stem. Already much better SDR than Demucs ft (4) model. More vocal residues than e.g. HQ_2 or Kim inst, but very clean results, if not the cleanest among all at the time. Jarredou in his fork fixed lots of those issues and further enhanced the SDR so it’s comparable with Ensemble on MVSEP, which was also further enhanced since the first version of the code released in 2023, and also has newer models and various enhancements.
- Demucs 4 (especially ft 4 stem model; UVR5, Colab, MVSEP, 6s available) - Demucs models don't have so aggressive noise cancellation and missing instruments issue like in GSEP. Check it out too in some cases (but it tend to have more vocal bleeding than GSEP and MDX-UVR inst3/464 and HQ_3 (not always, though), and 6 stem has more bleeding than 4 stem, but not so much like the old mdx_extra 4 stem model).
- Models ensemble in UVR5 GUI (one of the best results so far for both instrumentals and vocals SDR-wise). Decent Nvidia GPU required, or brace for 4 hours processing on 2/4 Sandy Bridge per whole ensemble of one song. How to set up ensemble video.
General video guide about UVR5.
"UVR-MDX still struggles with acoustic songs (with a lot of pianos, guitars, soft drums etc.)" so in this case use e.g. GSEP instead.
Description of vocal models by Erosunica
"That's my list of useful MDX-NET models (vocal primary), best to worst:
- MDX23C-8KFFT-InstVoc_HQ (Attenuates some non-verbal vocalizations: short low-level and/or high-frequency sounds)
- Kim Vocal 2
- UVR-MDX-NET-Voc_FT
- Kim Vocal 1
- Main (Attenuates some low level non-verbal vocalizations)
- Main_340 (Attenuates some non-verbal vocalizations)
- Main_406 (Attenuates some non-verbal vocalizations)
- Kim Inst (Attenuates some non-verbal vocalizations)
- Inst_HQ_3 (Attenuates some non-verbal vocalizations)
- MDXNET_2_9682 (Attenuates some non-verbal vocalizations)"
and it’s also worth to check HQ_4.
“UVR BVE v2 model [currently on x-minus] is actually full band. There is, however, a small nuance. This model uses MDX VocFT preprocessing, which is not full band. MDX VocFT model is rebalancing the song. The music is slightly mixed with the vocals (25% music + 100% vocals). This mix is then processed by the BVE model. A small amount of music can help the model better understand the context (it's important for harmony separation). We train the model on a rebalanced dataset. It contains 25% of music.” aufr33
_____
All the tips moved to Tips to enhance separation section
_____