MDX23 by ZFTurbo (jarredou fork) - 2, 4 stems
(2-4 stems, max 32-bit float output)
As of October 2025, Colabs are defunct due to Google’s runtime changes
(possible fix - make the dependencies look like here).
v2.5 (broken mirror to change repo), /w HQ_5 (experimental - muddiness, residues - set HQ_5 weight to 2.5 or lower), /w SCNet XL (weights set on ears, not measured), WebUI fork (for local installation), 2.4 (dead, added BS-Roformer model), 2.3 (Kubinka fork of jarredou’s Colab /w FLAC conversion, ZIP unpacking, new fullband preservation), 2.1 (voc_ft instead of Kim Vocal 2, a bit better SDR over 2.0 in overall), 2.2 (dead, with MDX23C model, may have more vocal residues vs 2.1), org. 2.3 (dead, with VitLarge model instead instr-HQ3), GUI/CML (GUI only for older original 1.x release by ZFTurbo with older models), instructions for local installation at the button
The ZFTurbo 1.0 Colab was further modified by jarredou to alleviate vocal residues. It adds better models and volume compensation, fullband trick for narrowband vocal models, higher frequency bleeding fix and much more. It achieved not much worse SDR as “Ensemble 4 models” on MVSEP back then (utilizes some newer private models available only on MVSEP). Initially released 1.x code by ZFTurbo received 3rd place in the latest MDX 2023 challenge.
“I have successfully processed a ~30min track with vocals_instru_only mode [on Colab] while I was working on that 2.3 version, but it was probably with minimal settings.
[Errors/freezes are] already happening during Demucs separation when you do 4-stem separation with files longer than ~10-15 min” jarredou
With v. 2.4, a 30 minute file was too long, and the Colab hung on Roformer model separation.
The Colab combines results of then the best public models of different architectures using custom weights for every model (like a manually set volume for every stem, then mixed with others together), instead of usual methods of ensembling as in UVR, which in e.g. “avg” averages results of all models (so there the same volume is used for every stem). More tricks in the Colab explained further below.
As of v. 2.5 “Baseline ensemble is made with Kim Melband Rofo, InstVocHQ and selected 1296 or 1297 BS Rofo” (so Kim Rofo was added, and VitLarge is no longer default).
~“Free Google Colab gives you 3h per day, then you need to wait 24h, and next day it gives you 2 free hours, after 24h wait you'll get only 1h, and 24h later, 2h of free credits, the day after 1h of free credits, etc... and once in that pattern, you have to wait 48h to recover the 3h back.” You can just change your Google account when the GPU limit is reached, but remember to use the same new account during mounting GDrive, otherwise you may get an error.
Troubleshooting
- “PytorchStreamReader error”
simply restart the environment, it’s a random issue occurring in the Colab.
- “usage: inference.py [-h] --input_audio INPUT_AUDIO [INPUT_AUDIO ...] --output_folder”
(and the whole list of arguments is shown below)
launch mount to GDrive cell (it’s not being done automatically) or change file input and output path
- “ValueError: Mountpoint must not already contain files”
(on attempt of mounting GDrive), go to file manager on the left, and you probably have GDrive folder with empty folders you need to delete from there first, and retry (might happen when you use GDrive on this account while it’s nto mounted yet, but Colab works).
- "no such file"
(error in v2.3 while batch processing)
“it's square brackets [ ]
when it sees a [ in the filename, it then thinks there's two additional [ ] in the name
changing to regular parentheses does work”
- “If I input more than a single song it just starts building up on model data without clearing the old one, so it slowly starts running out of VRAM and then gets stuck”
Experimenting with settings
- Default settings of the Colab are a good starting point in general
- Some people like to increase BigShifts to 20 or even 30 with all other default settings (some songs might be less muddy that way), but default 3 is already balanced value, but exceeding 5 or 7 may not give a noticeable difference, while increasing separation time severely.
- Switching from 1296 to 1297 model produces more muddy/worse instrumentals in this Colab (more sudden jumps of dynamics from residues). Similar situation with decreasing BigShifts to 1.
- voc_ft enabled might give less muddy results, but with more residues in instrumentals
- In 2.5 you can try out the following settings by mesk:
“Set the weights of BS-RoFormer & MDX23C to 0, enable VitLarge, and set the weights of Mel-RoFormer & VitLarge to 8 & 3 respectively.
You can set BigShifts to whatever you'd like, I think 5 or 7 is optimal” but mesk uses even 9.
VitLarge overlap can no longer be changed in v2.5 of the Colab, probably only in CLI version.
- Or you can test ensemble of only Kim weight 10 + Vit weight 5, BigShifts e.g. 9
- Or BS-Roformer with MDX23C
- Experimentally set “Separation_mode:” to 4 stems (slower) and "filename_instrum2" will be the sum of the Drums + Bass + Other stems that are obtained by processing "instrum" with multiple Demucs models. It might have a bit less vocal residues or be a bit muddier. Vs 2.1 denoiser is less aggressive as its disabled for some stems to save on VRAM.
- Increasing overlap might give muddier results, but potentially better if you hear some vocal residues
- In e.g. older v. 2.4 you might want to disable VitLarge to experiment (it’s disabled in 2.5) - the model increases some noise at times
- Older versions than 2.4 have very clean results for instrumentals, although it can rarely fail in getting rid of some vocals in quiet fragments of a track, but it has a bigger SDR than the best ensembles in UVR. Versions 2.4 and newer started to utilize BS-Roformer arch, which is pretty muddy itself, but deprived of the majority of vocal residues.
- For instrumentals, I’d rather stick to instrum2 results (so sum of all 3 stems instead of inversion with e.g. inst only enabled) but some fragments can sound better in instrum and it also slightly better SDR, so e.g. instrum can give louder snares at times, while instrum2 is muddier but sometimes less noisy/harsh. It can all depend on a song. Most people can’t tell the difference between both.
- If you suffer from some vocal residues in v. 2.2.2, try out these settings
BigShifts_MDX: 0
overlap_MDX: 0.65
overlap_MDXv3: 10
overlap demucs: 0.96
output_format: float
vocals_instru_only: disabled (it will additionally give instrum2 output file for less vocal residues in some cases)
- You can manipulate with weights.
E.g. different weight balance, in 2.2 with less MDXv3 and more VOC-FT.
- For vocals in 2.2 (dead) you can test out these (dead link) settings (21, 0, 20, 6, 5, 2, 0.8)
- In older versions of the Colab Overlap large and small control overlap of song during processing. The larger value, the slower processing but better quality (for both), but bad setting will crash your separation at least on certain songs.
Q: is it possible to use v.2.5 for Melband inference without the need to run the BS model?
A: You can comment out the model(s) you don't want to disable them L621-627 in inference.py [in the line called “vocals_model_names”
Probably, you could also set BS weight to 0, but it might trigger separation of that model anyway, making it slower.]
- To experiment with parameters for just 4 stems separation, you can use:
- "overlap_demucs" in the Colab (not sure how in this Colab, but for demucs_ft, shifts 10 and overlap 0.1 worked the best for original instrumentals as input)
- shifts for demucs are in line probably 511 (formerly 618 in some other versions iirc): https://github.com/jarredou/MVSEP-MDX23-Colab_v2/blob/v2.5/inference.py (dead)
- In order to bypass models for 2 stem separation to use just instrumentals as input for 4 stem separation, “comment out/delete the name of the models you want to bypass” in the line 621 (screen). “If you want to use only VOCFT, you have to activate InstVoc too, else it will crash (as it's using InstVoc to fill the spectrum part that is missing because of VOCFT cutoff)” - jarredou
Using other models not included in the Colab
- (dead) https://github.com/jarredou/MVSEP-MDX23-Colab_v2/blob/v2.5/inference.py
E.g. in line 452 you can replace Kim model by any other vocal model, and replace that edited in file manager once Colab has executed initialization cell or fork the repo. As for using instrumental Roformer models instead of vocal models, I can't guarantee it will work correctly.
- “Easiest way [to replace MDX HQ model] should be to replace Inst-HQ4 link with HQ5 link line 480 https://github.com/jarredou/MVSEP-MDX23-Colab_v2/blob/36909309efd4a75dab9f1d093a112785a8f560fb/inference.py#L480 (dead)
If models parameters are same (iirc they are), drop-in replacement should work (and then you control HQ5 with HQ4 settings in colab GUI)”
- Change the args awaited by inference.py accordingly to the ones you've changed in the Colab notebook [if you decide to change models names in he Colab], it's at bottom of inference.py (line 874 and so on)
- Adding e.g. SCNet is not that easy task, it will also require to really add SCNet arch to the script, not only words (add its core files to "modules" folder, import them in main script, check if that work with existing "demix" functions, etc... else it can't work).
https://github.com/ZFTurbo/Music-Source-Separation-Training/tree/main/models/scnet
You can study how ZFTurbo is doing it with his script and then try to adapt it to MDX23 Colab. ~jarredou
- What weight should you use for your custom model?
“You must process an evaluation dataset with each model individually, download the separated audio and then use my "weight finder" script (here [mirrored]) with all the separated audios from each model. It will try many different weights until it find the best ones for the given model inputs.
Else you can set "random" weights, process the multisong dataset from MVSEP and upload the separated audios to the quality checker to get the evaluation scores https://mvsep.com/en/quality_checker (and repeat until you're satisfied)
Download the mutlisong eval dataset provided on the quality checker link I've shared above. Process the 100 tracks with the model/ensemble you want to evaluate. Download the separated audio.
Rename the files accordingly to guidelines provided in quality checker link, zip them, upload them and wait for the results
All in same folder, and named:
song_000_instrum.wav
song_000_vocals.wav
song_001_instrum.wav
song_001_vocals.wav
song_002_instrum.wav
song_002_vocals.wav
etc...
Software like https://www.advancedrenamer.com/ can be useful for this”
About
The Colab produces one of the best SDR scores for 4 stems (maybe with slightly better implementation on MVSEP as “Ensemble” 4/5 or more models, although it could be 24 or 32 bit output used for that evaluation which increases SDR (jarredou’s v2.3 evaluation was made using 16 bit).
In version 2.4, for 2 stems, UVR/ZFTurbo/Viperx following models are used:
MDX23C Inst Voc HQ/MDX-Net HQ_4 and voc_ft (optionally)/VitLarge/BS-Roformer
and for 4 stems:
How MDX23 Colab works under the hood in 2.3 iirc (more or less)
- MDX models vocal outputs (so inversion of one inst model there) + Demucs only vocals>inversion of these to get instrumental>demucs_ft+demucs 6s+demucs+mmi to get remaining 3 stems (weighted) to get remaining 3 stems (all steps weighted). Something in this recipe could be changed since then.
Or differently - “The process is:
1. Separate vocals independently with InstVocHQ, VitLarge (and VOC-FT as opt)
2. Mix the vocals stems together as a weighted ensemble to create final vocals stem
3. Create instrumental by inverting vocals stem against source
4. Save vocals & instrumental stems
5 (if 5). Take the instrumental to create the 3 others stems with the multiple demucs models weighted ensembles + phase inversion trick and save them.”
Modified inference will probably work locally too, e.g. if you use that 2.1 repo locally (and probably newer too), but the modified inferences from jarredou crashes the GUI, so you can only use CML version locally in that case.
Usage:
python inference.py --input_audio mixture1.wav mixture2.wav --output_folder ./results/
To separate locally, it generally requires a 8GB VRAM Nvidia card. 6GB VRAM is rather not enough but lowering overlaps (e.g. 500000 instead of 1000000) or chunking track manually might be necessary in this case. Also, now you can control everything from options: so you can set chunk_size 200000 and single ONNX. It can possibly work with 6GB VRAM that way.
If you have failed to allocate memory error, use --large_gpu parameter.
Chunks option have been deleted from newer Colab options.
Jarredou made some fixes in the 2.2.2.x version in order to handle memory better with the MDX23C fullband model.
“I've only removed the denoise double pass for demucs_6s, it's activated for other demucs models.”
jarredou:
“you can use a workaround to have MDX23C InstVoc-HQ results only with (dead) settings:
(all weights beside MDXv3 set to 0, BigShifts_MDX set to min. of 1, and demucs overlap 0 [at least for vocal_instru_only)
You can use a higher "overlap_MDXv3" value than in the screenshot to get slightly better results.
(and also, as it's only a workaround, it will still process the audio with other models, but they will not be used for final result as their weights = 0)
(MDX23C InstVoc-HQ = MDXv3 here)
You can also use the defaults settings & weights, as it scores a bit higher SDR than InstVoc alone ”
Be aware that 2.0 version wasn’t updated with:
!python -m pip install ort-nightly-gpu --index-url=https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ort-cuda-12-nightly/pypi/simple/
Or in case of credential issues, you can try out this instead:
!python -m pip -q install onnxruntime-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/
Hence, it’s slow (so use 2.1-2.3 instead as they work as intended or add the line at the end of the first cell yourself)
Explanations on features added in 2.2 Colab (2.2 might have more residues vs 2.1) by jarredou