Training models guides
1. Read mesk’s guide (new link #2), then proceed below for arch explanations and more details.
2. More thorough training guide (just not that step-by-step, condensed form, all the training content was moved from here, there, to avoid 1.02M character limitation of Google Documents)
As for a training code for Roformers and MDX23C, SCNet or adding new archs, most people use MSST by ZFTurbo. It’s also provided with a bunch of documentation.
“You can start with Sucial MSST WebUI (link). I use that to train all my models” - Gabox
For weird errors with depth and stuff like:
Wrong shape: expected 4 dims. Received 3-dim tensor:
ZFTurbo "has been testing some things with Roformers lately, I already notified him that the Rofos have bugs.
You'll have to use an older version of mel_band_roformer.py, for example 1-2 months ago" - becruily
https://github.com/ZFTurbo/Music-Source-Separation-Training/tree/v.1.0.4
An old ass one, it doesn't even have sage attention, LoRA and all that shit - mesk
But probably you won’t need LoRA anyway.
Local SDR testing script
https://drive.google.com/file/d/1GC9pwch0WQXZXwBNTz_QnXE_UyxdKmQF/view?usp=sharing by Dill
https://drive.google.com/file/d/1BeqNw3TnRTDMwnoQGMbOqwrcGRwe4Zht/view?usp=sharing GUI by zmis (but it scores a bit lower for some reason):
“Here's a handy little python script I made using the help of Ai that can calculate the SDR of a track based off of the actual instrumental or vocal of the song.
You can do python sdr.py --help for an explanation on how to use the script.
You just need numpy and scipy for it to work, and python ofc!
I'm not sure if you would like to pin this or not, but I've been using this script to help me improve my separation methods.
https://github.com/ZFTurbo/Audio-separation-models-checker/tree/main
Based on MUSDB18-HQ dataset”
“Q: Why SDR goes <0 in silence parts? (song_006)
A: SDR and SISDR behave weirdly when 1 of the input is silent, and that's why log_WMSE was made: https://github.com/crlandsc/torch-log-wmse/
Interestingly, L1freqMag metrics is giving same results than some users here (1296 a bit better for instrumentals, 1297 a bit better for vocals).” jarredou
Best ensemble finder for a song script
by Vinctekan
https://drive.google.com/file/d/1LUtBsCSym1iDHqADEusmACs-LF2lNYLw/view?usp=sharing
Currently, this optimized version can find the best combo of 9, 3 minute audio files in about 2 minutes and 40 seconds in Colab.
Refactored best weighted ensemble finder by jarredou
https://drive.google.com/file/d/1Rm09z1wpj0Pi-6bFQ15u767n1XV95pDz/view?usp=sharing
“That's what I've used to find optimal weights for my MDX23 fork v2.5 update.
It's still Nelder-Mead based optimizer, but code is way more simple/clean than 1st version.
To use it, you need:
A dataset of clean sources (with exact same filename scheme than mvsep multisong dataset).
Process dataset mixtures with all the models you want to ensemble and put the outputs in different folders, 1 for each model (and still with exact same filename scheme than mvsep multisong dataset).
librosa and scipy python libs
then run (for example):
weight_finder_v2.py \
--ref c:\reference_dataset \
--est c:\InstVoc c:\bsrofo1296 c:\kimrofo \
--stem vocals \
--extension flac \
--tracks 100
--ref is clean sources' folder path
--est is estimates (separations) folder paths (multiple inputs)
--stem is stem name (based on multisong dataset filename scheme)
--extension is audio file extension (flac/wav...)
--tracks is a number of tracks in a dataset.
It will process the datasets many times and change weights each time until it find the best balance. When finished, it will output weights scaled to 10 max value.
Warning: it can take hours (or even days, depending on the number of models to ensemble, size of dataset and resources of computer)
A python lib to align audio:
https://github.com/nomonosound/fast-align-audio”
Universal function to make different types of ensembles by ZFTurbo
https://cdn.discordapp.com/attachments/911050124661227542/1192220574982881320/ensemble.py
I think it’s the same or newer:
https://github.com/ZFTurbo/Music-Source-Separation-Training/blob/main/ensemble.py
“In my experiments SDR for avg_wave always the max.”
Now also jarredou made his Colab with the above implemented with comfy GUI:
_____________________________________________________
____________________________________________________________________
Volume compensation for MDX v2 models
How to automate calculation of volume compensation value for all older MDX models
(results are not perfect and need to be fine-tuned)
by jarredou
So, I have maybe a protocol to find accurate volume compensation:
- Use a short .wav file of just noise (I've used pink noise here) and pass it through the model you wanna evaluate
- Take the resulting audio, the one that will have all the noise in it and compare it to the original noise with this little python script that will give you the difference in dBTP and the quivalent VC ratio (you'll need to
pip install librosa
if you don't have it installed already). The results I've found with it are coherent with the ones you've found by ears ! (1.035437 for HQ2 / 1.022099 for KimFT other)
Here's the script :
import numpy as np
import argparse
import librosa
def Diff_dBTP(file1,file2):
y1, sr1 = librosa.load(file1)
y2, sr2 = librosa.load(file2)
true_peak1 = np.max(np.abs(y1))
true_peak2 = np.max(np.abs(y2))
difference = 20 * np.log10(true_peak1 / true_peak2)
print(f"Diff_dBTP : The difference in true peak between the two audio files is {difference:.6f} dB.")
ratio = 10 ** (difference / 20)
print(f"The volume of sound2 is {ratio:.6f} times that of sound1.\n")
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Find volume difference of two audio files.")
parser.add_argument("file1", help="Path to original audio file")
parser.add_argument("file2", help="Path to extracted audio file")
args = parser.parse_args()
Diff_dBTP(args.file1, args.file2)
Volume compensation values for various models (in reality they may differ +/- e.g. by 0.00xxxx, but maybe not much more)
All values according to the script made by **jarredou**
*(All default but Spectral Inversion - Off; Denoise Output: On; - the latter shouldn't affect the results if turned off)*:
- Kim Vocal_1 - 1.012819
- Kim Vocal 2 - 1.009
- voc_ft - 1.021
- Kim ft other - 1.020 (Bas' fine-tuned and SDR-validated)
- UVR-MDX-NET 1 - 1.017194
- UVR-MDX-NET Inst 2 - 1.037748
- UVR-MDX-NET Inst 3 - 1.043115
- UVR-MDX-NET Inst HQ 1 - 1.052259
- UVR-MDX-NET Inst HQ 2 - 1.047476
- UVR-MDX-NET Inst Main - 1.037812 (actually it turned out to be 1.025)
- UVR-MDX-NET Main - 1.002124
- UVR-MDX-NET-Inst_full_292 - 1.056003
- UVR-MDX-NET_Inst_82_beta - 1.088610
- UVR-MDX-NET_Inst_90_beta - 1.151219 (wtf)
- UVR-MDX-NET_Main_340 - 1.002742
- UVR-MDX-NET_Main_406 - 1.001850
- UVR-MDX-NET_Main_427 - 1.002091
- UVR-MDX-NET_Main_438 - 1.001799
- UVR_MDXNET_9482 - 1.007059
"denoise is just processing twice with the second try inverted, after separation reinverted, to amplify the result, but remove the noise introduced by MDX, and then deamplified by 6dbs, so it still the same volume, just without MDX noise.
Basically HV noise removal trick"
UVR-MDX parameters & hashes decoded by Bas Curtiz
https://github.com/Anjok07/ultimatevocalremovergui/blob/master/models/MDX_Net_Models/model_data/model_data.json - the link with hashes possess MDX models parameters.
The above probably still doesn’t possess all the models added in the update, e.g. Foxy model, but there are only 4-5 combinations of settings so far.
File with newer models parameters:
https://raw.githubusercontent.com/TRvlvr/application_data/main/mdx_model_data/model_data_new.json
All MDX-Net model parameters in UVR consist of these combinations:
- HQ_4:
self.n_fft = 6144 dim_f = 2560 dim_t = 8
- All older HQ fullbands:
self.n_fft = 6144 dim_f = 3072 dim_t = 8
- kim vocal 1/2, kim ft other (inst), inst 1-3 (415-464), 427, voc_ft:
self.n_fft = 7680 dim_f = 3072 dim_t = 8
- 496, Karaoke, 9.X (NET-X)
self.n_fft = 6144 dim_f = 2048 dim_t = 8 (and 9 kuielab_a_vocals only)
- Karaoke 2
self.n_fft = 5120 dim_f = 2048 dim_t = 8
- De-reverb by FoxJoy
self.n_fft = 7680 dim_f = 3072 dim_t = 9
UVR model hash decode
“I've made this little script a while back to find those hashes.
Use with model_hash_finder.py path_to_model_file.”
https://drive.google.com/file/d/1D4TNKjuObNn6MSiss1PtmXPQoR3XJOwJ/view?usp=sharing
It's a checksum hash but based only on the last 10MB of model files.” (jarredou)
full_band_inst_model_new_epoch_309.onnx fea6de84f625c6413d0ee920dd3ec32f
full_band_inst_model_new_epoch_337.onnx 4bc04e98b6cf5efeb581a0f382b60499
kim_ft_other.onnx b6bccda408a436db8500083ef3491e8b
Kim_Vocal_1.onnx 73492b58195c3b52d34590d5474452f6
Kim_vocal_2.onnx 970b3f9492014d18fefeedfe4773cb42
UVR-MDX-NET-Voc_FT.onnx 77d07b2667ddf05b9e3175941b4454a0
kuielab_a_bass.onnx 6703e39f36f18aa7855ee1047765621d
kuielab_a_drums.onnx dc41ede5961d50f277eb846db17f5319
kuielab_a_other.onnx 26d308f91f3423a67dc69a6d12a8793d
kuielab_a_vocals.onnx 5f6483271e1efb9bfb59e4a3e6d4d098
kuielab_b_bass.onnx c3b29bdce8c4fa17ec609e16220330ab
kuielab_b_drums.onnx 4910e7827f335048bdac11fa967772f9
kuielab_b_other.onnx 65ab5919372a128e4167f5e01a8fda85
kuielab_b_vocals.onnx 6b31de20e84392859a3d09d43f089515
Reverb_HQ_By_FoxJoy.onnx cd5b2989ad863f116c855db1dfe24e39
UVR-MDX-NET-Inst_1.onnx 2cdd429caac38f0194b133884160f2c6
UVR-MDX-NET-Inst_2.onnx ceed671467c1f64ebdfac8a2490d0d52
UVR-MDX-NET-Inst_3.onnx e5572e58abf111f80d8241d2e44e7fa4
UVR-MDX-NET-Inst_full_292.onnx b06327a00d5e5fbc7d96e1781bbdb596
UVR-MDX-NET-Inst_full_338.onnx 13819d85cad1c9d659343ba09ccf77a8
UVR-MDX-NET-Inst_full_382.onnx 734b716c193493a49f8f1ad548451c48
UVR-MDX-NET-Inst_full_386.onnx 2e4fcd9ec905f35d2b8216933b5009ff
UVR-MDX-NET-Inst_full_403.onnx 94ff780b977d3ca07c7a343dab2e25dd
UVR-MDX-NET-Inst_HQ_1.onnx 291c2049608edb52648b96e27eb80e95
UVR-MDX-NET-Inst_HQ_2.onnx cc63408db3d80b4d85b0287d1d7c9632
UVR-MDX-NET-Inst_HQ_2.onnx 55657dd70583b0fedfba5f67df11d711
UVR-MDX-NET-Inst_Main.onnx 1c56ec0224f1d559c42fd6fd2a67b154
UVR-MDX-NET_Inst_187_beta.onnx d2a1376f310e4f7fa37fb9b5774eb701
UVR-MDX-NET_Inst_82_beta.onnx f2df6d6863d8f435436d8b561594ff49
UVR-MDX-NET_Inst_90_beta.onnx 488b3e6f8bd3717d9d7c428476be2d75
UVR-MDX-NET_Main_340.onnx 867595e9de46f6ab699008295df62798
UVR-MDX-NET_Main_390.onnx 398580b6d5d973af3120df54cee6759d
UVR-MDX-NET_Main_406.onnx 5d343409ef0df48c7d78cce9f0106781
UVR-MDX-NET_Main_427.onnx b33d9b3950b6cbf5fe90a32608924700
UVR-MDX-NET_Main_438.onnx e7324c873b1f615c35c1967f912db92a
UVR_MDXNET_1_9703.onnx a3cd63058945e777505c01d2507daf37
UVR_MDXNET_2_9682.onnx d94058f8c7f1fae4164868ae8ae66b20
UVR_MDXNET_3_9662.onnx d7bff498db9324db933d913388cba6be
UVR_MDXNET_9482.onnx 0ddfc0eb5792638ad5dc27850236c246
UVR_MDXNET_KARA.onnx 2f5501189a2f6db6349916fabe8c90de
UVR_MDXNET_KARA_2.onnx 1d64a6d2c30f709b8c9b4ce1366d96ee
UVR_MDXNET_Main.onnx 53c4baf4d12c3e6c3831bb8f5b532b93
VR de-reverb models decode
UVR-De-Echo-Normal.pth = f200a145434efc7dcf0cd093f517ed52
UVR-De-Echo-Aggressive.pth = 6857b2972e1754913aad0c9a1678c753
UVR-DeEcho-DeReverb.pth = 0fb9249ffe4ffc38d7b16243f394c0ff
So they’re all "4band_v3.json" config file (from here)
More thorough chart by David Duchamp a.k.a. Captain FLAM:
https://docs.google.com/spreadsheets/d/1XZAyKmgJkKE3fVKrJm9pBGIXIcSQC3GWYYI90b_ul1M
___
Voice Cloning
“RVC and some of its forks (Applio, Mangio, etc) are genuine free, open source ones for inference and training. For realtime voice changer that uses RVC models, there's w-okada: https://rentry.co/VoiceChangerGuide” no guide for Linux though.
“Url downloads, local files, massive database of models, both huggingface and weightsgg, in built separation models, options to skip that part if you have vocals, ability to use multiple ai models for one particular result, and the option to either merge or just get multiple results at the end, plus whatever else, de-reverb and stuff” it has voc_ft vocal model from UVR5.
“even my old laptop still can inferencing using applio
i3 3217u 1.8ghz
intel hd 4000”
And you’re probably aware already that RVC Colabs to train voice cloned models are banned.
Stable Audio Open Gen
Available on MVSEP in the Experimental section. It’s not for separation, but generating sounds.
ZFTurbo: “Algorithm based on model:
https://huggingface.co/stabilityai/stable-audio-open-1.0
Audio is generated in Stereo format with a sample rate of 44.1 kHz and duration up to 47 seconds. The quality is quite high. It's better to make prompts in English.
Example prompts:
1) Sound effects generation: cats meow, lion roar, dog bark
2) Sample generation: 128 BPM tech house drum loop
3) Specific instrument generation: A Coltrane-style jazz solo: fast, chaotic passages (200 BPM), with piercing saxophone screams and sharp dynamic changes
Examples:
Cat meow: https://mvsep.com/result/20250612092110-b297c082fb-generated.wav
Dog bark: https://mvsep.com/result/20250612115517-b297c082fb-generated.wav
128 BPM tech house drum loop: https://mvsep.com/result/20250612115841-b297c082fb-generated.wav
Violin solo: https://mvsep.com/result/20250612120111-b297c082fb-generated.wav
Woman sing song "Happy Birthday to you": https://mvsep.com/result/20250612120433-b297c082fb-generated.wav”
__________
Visit our #dev-talk channel for more
Anjok’s interview on YT
TL;DW: UVR’s documentary + training, archs and demudder explained
Anjok is the developer of Ultimate Vocal Remover 5 (UVR5 GUI).
He intended UVR to be a Swiss army tool - to contain everything you need for separation, and also contain models made by the community (e.g. dereverb/denoise/deecho).
History of UVR
Anjok in times where Spleeter was still a thing, found a VR arch made by Japanese developer, tsurumeso, and received better results than Spleeter. He started to make his own model on laptop 1060 6GB on 100 or 150 pairs with the absolute minium parameters, and it turned out to be a better model than tsurumeso's one. Later he transitioned to faster GPU (probably before 3090 yet).
Anjok wanted GUI for VR, and found BoskanDilan on Fiver and simply contracted him, paying to build the foundations of what UVR is today. BoskanDilan turned out to be a very good and talented coder.
They put the work on GitHub, and Aufr33 contacted Anjok with ideas on how the VR models can be improved etc.
Then BoskanDilan left in mid 2021 for personal reasons. Then the GUI work was taken by Anjok who was mentored by BoskanDilan to improve on understanding the coding. Anjok started to working on UVR exclusively, spending 10 hours a day for UVR in 2022.
He decided to make a simple installer in one package, as he received lots of issues on GutHub, from people not knowing how to install it. He also re-coded the UVR to make the code easier to maintain. Then Bas Curtiz helped Anjok on design aspects of UVR, e.g. designed new logo, and gave some advice, and good amount of feedback from UVR user perspective. Early 2022 phase of UVR development took a lot of advice from early users of UVR.
In May 2022 there was a first installer released to make UVR more accessible without e.g. installing Python or other dependencies and specialized programming knowledge to set up a proper environment.
Anjok was still in charge of introducing other archs than VR into UVR, being simply the only one behind the process, while normally bigger teams work on projects of that scale, when e.g. different archs could be coded into UVR by different developers. It was a stressful period of time, because Anjok intended to make the software which is free of bugs, and still not fully rely on the community in terms of bug reporting.
Then the Mac version came out and M1/M2/M3 support for faster GPU acceleration. Anjok found out in Demucs repo a part of the code, making it easier to port UVR to Macs, and it is used by every model. Music community is pretty Mac-centered, and he devoted a considerable amount of time to make it work reliably on Macs too.
In the new UVR version there's a planned demudder to be introduced (described later), and possibly translations.
Anjok currently trains a new model coming in several weeks.
It's intended to be a little smaller in order to be not so resource intensive, but also better than the best current MDX-Net model.
Update 01.03.24
“I'm going to allow HQ4 to continue training beyond 1500+ epochs as an experiment (it's currently at 1200), and interestingly, the SDR has been steadily increasing. It has significantly surpassed HQ3 in terms of SDR and listening tests, and it also outperformed MDXC23 in listening tests, though not in SDR (yet!). The most recent evaluation on the multi-dataset showed a score of 15.85, using the default settings. Clearly, there's a limit to how much further training can enhance performance, but up to this point, improvements are still being observed. This model has been in training since October! I'm chipping away at the next GUI update as well, and the demudder will be in it.”
The model was released, with already HQ_5 scheduled in following month/s.
The archs in UVR and their technicalities summarized
VR
VR uses audio spectrograms and converts them to FFT spectrograms.
VR uses only magnitude spectrograms, not phase.
Phase represents timing where the data is, while magnitude represents the intensity of each frequency.
Phase is much harder to predict.
Actually VR uses original phase from the mixture and saves it during the process "and it just does the magnitude".
That's the reason why VR tends to have more artefacts in it. The smearing in instrumentals of VR is because the phase from the mixture is still in there.
Aufr33 later introduced 4 bands support for UVR.
Let's say for first of three bands between 0-700Hz there will be different resolution, for all other frequency ranges there will be different. E.g. knowing that vocals are in specific frequency range, you can optimize it further.
That feature made UVR and VR arch much better.
Later they introduced -
Ensembling
So a way to use multiple models to potentially get better results.
The three ways of ensembling:
avg - gets the average of vocals/instrumentals
max - is maximum result of each stem, e.g. in a vocal you'll get the heaviest weighted vocal from each model, and the same goes for instrumental, giving a bit cleaner results, but more artefacts
min
MDX-Net
Uses full spectrogram with phase and magnitude
Tradeoff is muddier results, but natural, cleaner sound.
Training
Anjok separated on nearly every genre you can think of, and stated that the hardest genre for separation is metal and vocal-centered mixes. Also, if the instrumental has lot of noise, e.g. distorted guitars, the instrumental will come out muddier.
MDX-Net was the arch, addressing lots of VR issues in its core.
Tracks from 70-80s can separate well. 50-60s will be harder, e.g. recorded in mono. Early stereo era gets a little better.
A good model needs to be as good as the dataset for a model.
There was lots of work scrapping it from the internet.
Aufr33 was the mastermind behind Karaoke model and its dataset.
Demucs model wasn't as successful, as probably was more meant for more stems, and MDX-Net gave better results for 2 stems.
Training details covered in this interview can be found at the top of Traning models guide section of the doc
The biggest issue in terms of archs and the source of muddiness, is phase. Currently, in audio separation there's not a great way to calculate phase in a model like the phase spectrogram as it's not as obvious as the magnitude spectrogram.
You take the vocal out of a heavy rock track, but the process is not perfect, so it will take some part of the instrumental with it. Even if you don't hear instrumental in vocals, there's still instrumental data in there in the phase of that vocal track.
In the end of the day, source separation is prediction. It's predicting where it thinks it is, but there will be always some imperfections, e.g. whenever you hear muddier sound in a track which has more noise like metal tracks.
Anjok emphasizes on (currently) lack of correlation between SDR and the fact that bigger SDR metric doesn’t necessarily mean better. He tried some top of the SDR chart result before, and wasn’t quite happy about them.
Because phase is a big part of the issue, now the new upcoming -
Demudder
A UVR feature incoming (it was also explained before on the server by Anjok - if something is not clear, try to find his messages there)
It uses lots of phasing tricks. It processes the track twice. The first takes instrumental from the first go around and compares it against the original mixture. It chops the mixture into 3 second chunks and ?inerts over that lists of chunks and for each segment, it cuts out where that segment is in the instrumental, and it finds similar events that aren't at the exact same place. It takes those chunks, and it analyses them against the instrumental that was generated, and it tries to find the most similar events it can from the instrumental, that aren't at the same place from that segment, and it finds similar events, and then it phases it, it does a phase invert of that instrumental
(56:30) If the volume or DB threshold isn't past the certain point because it's too loud then it means it does not cancel out and doesn't make phase invert, if it reaches a certain threshold like if it is below certain threshold it'll phase that, and then it will basically stitch together a new mixture that is kind of phased from that original instrumental output, and it reprocesses that new stitch together, mixture with the phase with the instrumental phase changed, and it processes that through the second pass, and then it takes that vocal and then phase inverts it with the mixture, with the original mixture and then what you end up having is some of the parts that are similar from the other parts of the track, you end up having those fill in the spectral holes.
Sam remarks find some similarity with probably how Izotope Imager works.
Anjok says: I'm trying to get a similar part, but also try to take it and phase it with that segment. Because it's not the exact same part of the segment, it's not gonna be a perfect phase, because it would be an original vocal output.
So it's kind of still finding the bit of instrumental that is still in the vocal.
Sam remarks about frequent situations where you perform separation, and it can lead to decrease of e.g. hi hat volume levels in instrumental, referring to what information separated vocal stem can wear. It's part of the muddiness Anjok tried to address with the feature.
Anjok didn't want to compromise vocal quality, and in some cases it makes the vocals better too, but it also depends on how the track was mixed originally. If it's an analog track recorded in one session or even a live track, it won't work so well. The problem is with e.g. 10 minutes track, when demudder won't find phase similarities so effectively. It will work the best on music made with samples. If the track is digital, it is more likely to work better.
Anjok currently works on it to make it work for all tracks.
The more he works on it, the more breakthroughs are made, but due to his day job, he had less time to work on it lately.
Anjok gives his appreciation to the group of very talented developers who made MDX-Net arch in the University of Korea. It's his favourite network. He's a big fan of Woosung Choi's work.
_____
Later, Aufr33 invented his own:
Simpler demudder
Published for paid users of x-minus.pro (when you pick Roformer model for instrumentals, buttons with methods appear; it is only applied for instrumentals, not vocals)
In his own words:
“
1. Separate the song into vocals and music
2. Invert the phase of the vocal and mix it with the music
3. Now separate this mix
4. Mix the vocals with the input song
It actually works more complicated than that. I added a high pass filter since the demudder is not needed at low frequencies.”
Probably something from the 100-250 Hz range.
Actually, Aufr33 used following ffmpeg command:
“ -filter_complex "[0:a]highpass=f=900[hp1];[0:a][hp1]amerge,pan=stereo|c0=c0-c2|c1=c1-c3[lp];[1:a]highpass=f=900[hp2];[lp][hp2]amix=inputs=2:duration=longest:normalize=0[out]"”
Rephrased by becruily
“use Roformer on a song
phase invert the vocal file and combine it with the instrumental
separate again using the same model
combine the original song and vocals (no inversion or anything) and you will get demudded inst
this is for instrumental, if you want demudded vocals just switch the two words (acapella and instrumental)”
Video how to apply demudder method
Notes
- For HQ 4 and at least denoise model enabled, the method seems to produce more vocal residues, so it might be feasible more for Roformers (it’s used optionally for Kim Mel-Roformer on x-minus).
- "xminus demudder is more pleasing to the ears" isling
- Some people might still prefer max_mag ensemble on x-minus or mel-roformer + bs-roformer ensemble in UVR
Phase fixer on x-minus for unwa inst v1 model copies phase from Kim Mel-Roformer model.
UVR Demudders released in the beta Roformer patch
(in Anjok’s words)
- Phase Rotate:
The fullest sounding, but can leave a lot of artifacts with certain models. I only recommend that method for the muddiest models. Otherwise, Combined Methods is the best”
- First, a filtered instrumental is created, and the left and right channels are swapped.
- The phase is shifted by 90 degrees.
- This modified filtered instrumental is then inverted with the original mixture, and another inference pass is performed on the resulting mixture.
- Finally, the vocal from the second pass is phase-inverted and combined with the original mixture, creating a cleaner instrumental.
- Phase Remix (similar to X-Minus; also available in this Colab):
I don't recommend using phase remix on the Instrumental v1e model. I recommend combined methods or phase rotate for models producing fuller instrumentals.
- The mixture is first separated into stems.
- The phase of the vocal stem is inverted and mixed with the filtered instrumental to produce a modified "mixture." Another inference pass is performed on this new mixture.
- The vocal stem extracted from the modified mixture is then reintroduced into the original mixture, creating a cleaner instrumental.
- This method is only recommended for models that produce very muddy instrumentals!
- Combine Methods:
- It's basically a weighted mix of the final instrumentals generated by "Phase Rotate", "Phase Remix," and the initial instrumental.
Demudder in UVR doesn’t work on 4GB VRAM Intel/AMD GPUs
(more demudder troubleshooting)
Phase fixer/swapper
(decreases vocal residues in instrumentals)
The method was invented by Aufr33 to fix noise in Roformers models trained with instrumental or other stem target. It copies phase from instrumental from a model trained with vocal stem target which usually gives muddy instrumentals but better bleedless metric (e.g. Kim’s or Becruily’s vocal) and copies it into the instrumental model result. Initially it was added only to x-minus/uvronline (iirc for premium users), but later Becruily wrote his own script doing the same (both torch and librosa implementations if you fail to use one of them).
Later Anjok implemented it into one of the UVR Roformer beta patches (Tools>Phase Swapper), although there it only allows changing high and low cutoff, but no high frequency weight, and santilli_ (Michael) found out that increasing it from 0.8 to 2 is beneficial for phase swapping from Becruily vocal to instrumental model, and that it’s much better than manipulating with high and low cutoff.
The common mistake people make when phase fixing manually is that, you still need instrumental result of a vocal model as reference, not vocal result, and instrumental from instrumental model as target, ofc.
Also, don't use separated stems in various formats (lossy and lossless will start in different places), and rather also from different sources - e.g. on MVSEP only 32-bit float for premium is not normalized - rather refrain from use it with results from e.g. Colab, which are not normalized. Even the normalization option in UVR might work a bit differently, but feel free to ensure.
Use manual phase fix Colab (by A5/Squid) - fork of Michael's Colab, but without inference.
(OG santilli_/Michael Colab here was fixed (it allowed inferencing and batch separation with phase fixing automatically), although since Google's changes to the env, the Colab got broken and the fix started giving weird results both with FLAC and FLOAT, while the local scripts give only slightly less muddy results than the manual Colab - dca).
Optionally you can also check the phase fixer in SESA Colab and MVSepless Colab.
Or MVSEP preset called “phase corrected instrumental”.
This specific preset is currently only for premium because by default it uses HyperACE V2 inst with (currently paywalled) BS-Roformer 124 bands.
But you can just click “Editor” option in the presets menu, and pick “phase corrected instrumental” preset in the editor on a free account, and on the displayed graph pick whatever models you want for phase fixing instead (e.g. becruily vocals [even instead of BS-Roformer 2025.07], which might work better for phase fixing than default BS-Roformer 124 bands, and is free model), and maybe run the job from there. Solely presets functionality and the editor is not paid, but such a modified preset on your main MVSep page will be paywalled. “If 2+ algorithm nodes, it is premium only”.
Currently you can only save your preset on your account and sharing is possible only by screenshotting or copying the graph code manually and sharing on your own.
- You can use and edit the original phase swap Python scripts for previously separated files here, and use it as a reference (iirc - be aware it has hardcoded file names into script to work, so Kim as voc model, so while using any other model, you still need the same file name).
The result of phase fixer - less noise in the instrumental, but more muddiness - it's not necessary in all cases (instructions of usage of the script is described in the link above).
Cutoffs
Optionally to the default settings in the Phase Fixer Colab you could set:
- 420 for low and 4200 for high or
- 500/500 and e.g. Mel-Kim model for source;
and bleed suppressor (by unwa/97chris) to alleviate the noise further (e.g. phase fixer on its own works better with v1 model to alleviate the residues).
- Besides the default UVR default 500/5000 and
- Colab default 500/9000 values, you could potentially “even try like
- 200/1000 or even below for 2nd value.” “I would say that the more noisy the input is, the lower you have to set the frequency for the phase fixer.” - jarredou
(“200/1000 is probably the most aggressive setting”, “for v1e 200/1000 def works better, v1e is very noisy”)
- 200/10000 is the least aggressive
Phase fixer models suggestions
Usually, I post suggested models in the descriptions of instrumental models along with values for phase fixer/swapper here, but you have some suggestions gathered in one place also in phase fixer Colab.
E.g.
a) becruily voc/becruily inst (very bleedless, but muddy)
b) becruily voc/flowersv10 (not as muddy)
c) FT2 Bleedless/flowersv10 (can be more effective, but FT3 might cut fewer instruments)
d) BS-Roformer 2025.07 (exclusive to MVSEP)/(any other inst model)
e) Mel-Roformer Bas Curtiz Edition on MVSEP/(any other inst model) - less bleeding/vocal shells than while using becruily voc, and might be even less than 2025.07 (bigger vocal bleedless metric)
Other models occasionally used for phase fixer with certain models: bigbeta5e, Unwa beta 6 (or experiment some newer models with potentially high bleedless metric)
Scale factor:
“Personally, I think leaving the scale factor as 0.8 is fine, increasing it to like 1.4 as in the notebook creates "holes" where the vocals are originally heard, so I don't think that's good.
Higher than 0.8 makes it muddy, v1e is great at being the fullest model so I would not touch scale factor if I were you (if you use v1e)”
Phase-fixed output used in UVR’s Manual ensemble
(method explained by objectbed, corrected by Ari/arxynr)
On example of dca’s ensemble in the doc:
"0) Unwa BS Roformer Resurrection Inst (BS 2025.07 as a reference for phase fix) + MVSEP BS Roformer 2025.07 (Max Spec)
—-> the least vocal crossbleeding.
Alternatively, you can use becruily vocal model instead of 2025.07 for the ensemble -
“Becruily vocal correctly recognize instruments far better than the instrumental one” - dca100fb8"
“This would equate to the following steps:
[guide corrected; didn't give correct results before]
1. Separate your mixture using the Unwa BS Roformer Resurrection Inst model.
• Output: inst_unwa.wav (instrumental) + optional vocal.
2. Separate your mixture using the MVSEP BS Roformer 2025.07 model.
• Output: inst_mvsep.wav + vox_mvsep.wav.
[Note: Set WAV output. Then MVSEP won't turn down the volume to accommodate for clipping, and in such cases it should use 32-bit float instead of 16-bit automatically, despite 16-bit being set for free user]
3. Build the ensemble with UVR’s Manual Ensemble mode:
4a. Inputs =
• inst_unwa.wav (from Step 1)
• inst_mvsep.wav (from Step 2)
4b. Set Algorithm = “Max Spec.”
4c. Click Start Processing
4. Phase fix with UVR’s Phase Swapper:
3a. Target Audio = inst_unwa+inst_mvsep (ensembled).wav (from Step 3).
3b. Reference Audio = inst_mvsep.wav (from Step 2).
3c. Click Start Processing.
3d. Note the resulting new file (~~~_phaseswapped.wav or similar).
The resulting file = your final instrumental stem (the one referenced in the Google Doc instructions as having the least cross-bleed).”
FAQ
Q: Question about MVSEP BS Roformer 2025.07. Is this a model I can download, or do I have to do it online on their site? I can't find it.
A: It's mvsep site only. No download. So you want to use this model, must go mvsep.com
Q: What exactly are we doing with a Phase Fixer/Swapper? As an audio engineer, I understand phase as how it relates to frequency over time. When a vocal stem is used as a "reference" for the target instrumental stem, what's actually happening?
A: “There is a phase in the waveform domain, like audio engineers experience it every day, and there is a phase in the STFT domain. In STFT domain phase means how each STFT bins content is organized with other bins.
The math concept is similar to phase in the waveform domain, but instead of having 1 phase value for 1 waveform, you have 1 value for each STFT bin and for each STFT frame, so it's way more complex... (to make it really short).
The phase fixer/swapper thing is operating in the STFT domain, so for each STFT bins, it will use the magnitude data of 1 model, and it will use the phase data from another model, and hopefully this can improve the final result”. - jarredou
“In short, what the script does is blend the stft phase of the "donor" file into the target, it uses different blending scales for different frequencies so that it'll only affect the parts that are directly related to the perceived noise” - santilli_ / Michael
Q: Apparently the original Resurrection model only has two models (BS-Roformer 1296/1297 by viperx and BS Large V1 by unwa) that work for phase fixer, but the Gabox one says to use 2025.7 for phase fixer (which works almost perfectly) How does that happen? How did the working models change for Gabox's fine tune?
A: ML models are black boxes. You never know what you'll get. Like in Forrest Gump. It depends on the specific instrumental model, which vocal model as the source for the phase fixer will work the best.
__
Pitch shifting/stretching algorithms' comparison
https://www.youtube.com/watch?v=gaSFt0tT2u4
https://www.youtube.com/watch?v=s-5g4I30_eY
https://www.youtube.com/watch?v=WH8KDQALYQY
“I've had much better results with Izotope RX than Studio One for example for stretching.”
Also, Bitwig can be good.
You can also try out paid Lossless Pitch AI on dango.ai (tuanziai.com/en-US).
or paulstretch
“if you want to retain the original pitch this is the first tool that comes to mind”
Research: https://discord.com/channels/708579735583588363/911050124661227542/1303058610934382675
Restoring hi-end in pitched-down tracks - click
____
What does changing batch_size from 1 to 2
(it wasn’t used in Rofo beta UVR for 9 Jan 2025, but maybe it got changed)
“if your input is batch time sequence, it looks like this:
[ batch1->[time->[sequence],time2->[sequence]..], batch2->[time->[sequence],time2->[sequence]..] ]
As you increase batch_size you increase the amount of data the model gets to churn through.
So higher batch_size allows the model to see more data before you do a thing called backward prop which calculates another thing called gradients,
which are used to improve the model by tuning loads of little values inside the neurons so that the next pass through is more accurate” frazer
Q: They told me that increasing batch size to 2 makes it process faster
A: “So when that user says you can increase the batch_size what they mean is you can use more than one song to process - i.e. instead of running a single song at batch_size = 1 you can run 2 at the same time (batch_size = 2)
Q: Ah so batch_size param is used for the amount of chunks of the input, so if I set [batch_size] to 4 my audio is chunked into 4”
A: “No, the chunks are split based on defined chunk_size in config (which is more related to STFT settings), and then the script is stacking 'batch_size' number of chunks in same tensor to process them at same time (for inference).” jarredou
A: “Increasing the batch size increases the number of chunks that can be processed at one time, which may speed up processing, but also increases memory usage.
It will probably not affect quality.” - unwa
Inference Colab by jarredou forces batch_size=1. Iirc the clicking issue with such value was fixed in MSST repo later, and you can stick to it. Probably in UVR too, since latest patches where newer inference code from MSST was implemented.
Someone was once telling that a value not bigger than 2 takes no more than 4GB of VRAM, but it will rather differ from AMD/Intel when the VRAM usage is higher due to lack of garbage collector present in CUDA.
TL;DR
models list shortened
04.08.26
There's no single model always for everything. Usually it's a matter of testing a few models or even ensembling till you get the best results (also, be aware that sometimes your taste or listening environment determines how sensitive you're to certain artefacts of separation models and how passable the results are for you).
ACAPELLA (vocals)
- MVSEP BS-Roformer 124 bands becruily (fine-tune of the below, good mix of fullness and bleedless, premium only)
- MVSEP BS-Roformer 124 bands (premium only, the best SDR, more bleedless)
MVSep BS-PolarFormer 124 bands (the 2nd best)
- MVSep BS-Roformer 2025.07 (bleedless, and accuracy, muddy)
- Becruily Deux (fullness, but still a bit similar to the PolarFormer),
BS_RoFormer_mag (fullness with more bleedless; very noisy at quiet parts),
BS_Roformer_mag_v2 (worse SDR, bigger fullness, you might like it more),
- unwa BS-Roformer-Leap Xe (best metrics for public model, e.g. the non Xe more highs, less mids vs Deux)
- HyperACE voc v2 (bleedless, with more fullness),
- MVSEP BS-Roformer 124 bands becruily (good mix of fullness and bleedless, premium only, duality model with also dedicated instrumental stem, so not an inversion)
Deux (former perfect compromise of bleedless and fullness, also duality model)
bs_roformer_inst_hyperacev2 (fuller; doesn’t work with UVR)
(older but still might work)
BS-Roformer Resurrection inst (sometimes noisier than Deux)
V1e+ (the above tend to be better)
Inst_GaboxFv8 (sometimes can be more destructive the top)
BS-Roformer-Leap Xe (fine-tuned Deux, sometimes flaws in BVs)
inst_gaboxFlowersV10 (can be noisy)
(bleedless models)
Fv7z
Inst_FV8
Rifforge
Phase fixed becruily inst with becruily voc
Vocal models for instrumentals (e.g. Big Beta 5e FT/Mel Kim FT by Unwa/MVSEP Polarformer/BS-Roformers)
(vocal models for instrumentals
- sometimes more bleedless)
E.g. Big Beta 5, FT2 Bleedless, Mel Kim, or some MVSEP-exclusive mentioned in the vocals section above
(ensembles)
Mel deux + BS 2025.07 (Max FFT/Spec) (“the best Ensemble for vocals for now”)
MVSep Polarformer, Deux and MVSep 2025.07 (Max FFT)
deux + big beta 7 (Max FFT) (“another good ensemble”)
Lead Vocals (KARAOKE/LVs)
small_karaoke_gaboxauf
BACKING VOCALS (BVs)
anvuew’s Karaoke BS-Roformer
Ensemble of becruily frazer karaoke + anvuew karaoke (Max Spec)
becruily & frazer BS-Roformer Karaoke
Stereo width feature for uvr bve v2
MVSep SATB Choir
MVSep Choir
ADLIBS
(probably some models for BVs)
MULTISINGER (two singers/duets)
MVSep SATB Choir
becruily & frazer BS Karaoke
Becruily Mel Karaoke
Dry Paint Dealer Undr MelBand Roformer Duet
MedleyVox
MVSEP Multispeaker model
MULTICHANNEL
MVSEP Multichannel
PHANTOM CENTER (center/similarity/mid extraction)
CenterWide model MDX23c dual fullness a.k.a. v2e
Ensemble Avg/Avg of SCNet V2 by gilliaan + gilliaan_mdx23c_centerwide_v2e
4-6 STEMS (drums, bass, other, vocals / piano, guitar)
MVSep Ensemble (4 stems) (2025.06.30) - for premium
BS-Roformer SW (6 stems - /w piano & guitar)
MDX23 fork
SCNet XL IHF by ZFTurbo
Demucs_ft
DRUMSEP (expects drums as input; hihat/cymbals/ride/crash, snare, toms, kick)
MVSEP 8 stems ensemble
MVSEP SCNet 4 stem
MVSEP SCNet 5 stem
MVSEP SCNet 6 stem model
jarredou MDX23C 5 stem model
jarredou MDX23C 6 stem model
UPSCALING (music)
AudioSR
Apollo Universal
UniverSR
A2SB
RE-USE
UPSCALING (voice/vocals)
AP-BWE
Clearer voice studio's clear voice
emastered.com (paid; or free for downloaded preview mp3 320kbps with browser’s devtools)
Matchering (for reference song master matching)
Masterknecht (-||-)
See also isling’s short separation guide
(at the moment it has some older models)
_____________________________________________________________________
Section with miscellanies -
Random stuff or quotations from below waiting to be added to relevant sections above with added authors and corrected. Possible duplicates. A mess, but potentially useful. Some stuff might be outdated. Random notes from our Discord server. Not working hyperlinks yet.
(added from the newest info to the oldest, currently 143 pages)
It won’t fit the GDoc limit, so it was moved here.
_______
Done by deton24, 2021-2026
Special thanks to Audio Separation Discord,
and all the people mentioned in the credits (Hall of Fame) section above.
Without you, it wouldn't happen.
_____________________
For help and discussion, visit our Audio Separation Discord: https://discord.gg/ZPtAU5R6rP | Download UVR or MSST-GUI
For inst/voc separation in cloud, try out free Colabs: BS/Mel-Roformer | MDX23 (2-4 stems) | MDX-Net | VR | Demucs 4 (2-6)