Fixed in the beta patch #3 and #4 for all platforms
- Don't set overlap higher than 11 for 1101 dim_t (at the bottom of yaml file in the “inference” section, not above) and overlap 8 for 801 - these two are the fastest settings before stem misalignment issues occur. Otherwise, it can lead occasionally to some effects or synths missing from the instrumental stem (although some rules can be broken here with various settings). Also, the problems with clicks are alleviated with these good settings.
- In beta #2 patch, best measured SDR for both Mel and BS-Roformers is when dim_t = 1101 in the inference section of yaml config and when overlap is set to 2 in GUI (although 1 wasn’t tested, and is actually lower). But the last beta patches, all bigger overlap values are slower, so SDR might be higher with higher values.
Be aware that it will increase separation time. Maximum allowed value before error is 1801, but 1501 or 1601 depending on a model will be the max reasonable for experiments before some unwanted downsides of too high or too low dim_t appear (disappearing of some stem elements). In some specific cases, 1333 (or potentially 1301) was giving better results than 1101 or 1501, but it depended on song length - usually it happened on short fragments.
- Instruction for overlap and dim_t above applies to other Roformer models as well, and not only those in Download Center. With the instructions, you can achieve faster separation times, as you’re not forced to use the most time-consuming overlap 2 in older patches to avoid stem misalignment issues
_______________
Infos and fixes for older patch #1/2
(with matching overlap (reversed) and dim_t necessity)
- To avoid separation errors for 4GB VRAM and AMD/Intel GPUs using Roformers, set segments 32, overlap 2 and dim_t 201 with num_overlap 2 both at the bottom of yaml config in \models\MDX_Net_Models\model_data\mdx_c_configs
(dim_t 301 and overlap 3 also works, although not on all models [e.g. not for beta 3, but inst v1] and seems to be less muddy and fewer clicks appear).
dim_t 201 is not optimal setting and might lead to more occasional quiet residues, clicks or sudden volume changes (like chunk was changing every 2 seconds), although there’s no stem misalignment issue with these settings (they work both for Mel and BS Roformers). dim_t 301 with lighter models seems to be a bare minimum to avoid the majority of audible artefacts (after patch #3 dim_t 256 is allowed - “make sure you check the "Segment Default" in MDXNET23 Only Options for it to take effect”).
Using the settings above on patch #2, with GPU acceleration it will take 39m 28s for 3:28 song using 1296 model on RX 470 4GB and 18 minutes for Kim Mel-Roformer and 3:01 song.
Using HQ_4 is much faster than realtime using default settings, but even longer than accelerated Roformer, when on CPU only using old Core 2 Quad @3.6 DDR2 800MHz.
On Mac M1 using the patch above, it takes 9 minutes to process a 3-minute song using BS-Roformer (dim_t 1101, batch size 2, overlap 8) with “constant throttling”. Click
And below 4 minutes for Kim Mel-Roformer (overlap 1, dim 801). Click
- Settings working for 6GB AMD GPUs: dim_t 601 or 701 at the bottom of the yaml file and overlap 6 or 7 in GUI.
- Like I mentioned, overlap 8 can be good enough too when dim_t=801 is set (the fastest setting before SDR getting drastically reduced), at least in other cases you shouldn’t exceed 6, while 2 should provide the best quality in most cases.
- 1602 (or rather 1601) dim_t might lead to less wateriness, but turns out in cost of a bit more of vocal residues.
- “In theory, max overlap value [for Roformer separations without mentioned issues in UVR] can be known with formula:
(dim_t - 1) / 100 = Max_overlap_value
if dim_t = 801:
(801 - 1) / 100 = 8
if_dim_t = 1101:
(1101 -1) / 100 = 10 [jarredou wrote 10 here, but it’s actually 11]
Above that max value, some parts of the input will not be processed.
The lower the overlap value is, the more overlap is used, so better SDR.
- Some Rofos models still have wrong config by default, with dim_t=256, so max overlap value for that is 2. That's why I've advised to stick to overlap=2” - jarredou
So in times before dim_t was known how to be correctly set, so now overlaps can be even set to 8 now when dim_t=801 is set]).”
The same thing applies for both BS and Mel Roformers in UVR.
“audio.dim_t value is not used with Roformers in ZFTurbo script, it uses audio.chunk_size and then it's parameters in the model part of config.”
- Using older Roformer beta patches for Mac M1 doesn’t allow you to choose the Roformer parameter to check for custom Roformer models and only config name can be chosen, but no confirm button is available. So the error “File "libv5/tfctdfv3.py", line 152, in __init” appears.
> Place the corresponding json file with your model from this repo into: models\MDX_Net_Models\model_data beforehand, to fix the issue.
In some cases, you may still get the same error anyway and to get rid of it, you need to edit manually model_data.json adding desired model line at the end like your custom model was downloaded from download center. On example of unwa’s beta 3:
},
"d43f93520976f1dab1e7e20f3c540825":{
"config_yaml": "config_melbandroformer_big.yaml",
"is_roformer": true
}
Additionally, you need the model at the end of model_data_mapper.json:
"model melband_roformer_big_beta3.ckpt": "config_melbandroformer_big"
}
Now copy the hash-named json file (d43f93520976f1dab1e7e20f3c540825.json for beta 3) to model_data folder.
All the three modified files for beta 3 and other models here.
If you have problems generating hash on first launch of the model and your model is not uploaded in the repo above or json is not generated then use Windows installation in VM, or ask some PC user for the config. Potentially reading Hash decoding can be helpful.
But maybe your hashed config name will be generated correctly already after you imported the model into UVR (although no confirmation button might prevent it), and now it will be enough to just place the following line like in the jsons presented above: "is_roformer": true” (so after “,” in the yaml line above).
- More in-depth - Settings per model SDR vs Time elapsed -||- (incl. dim_t and overlap evaluation for Roformers) - click or here | conclusion - made before patch #3