Ensemble algorithm explanations

Ensemble - a way to use multiple models to potentially get better results.

 

Rules to be broken here, but:

Max Spec is generally for vocals
(is maximum result of each stem, e.g. in a vocal you'll get the heaviest weighted vocal from each model, and the same goes for instrumental, giving a bit cleaner results, but more artefacts)

Min Spec for instrumentals in most cases
(it leaves the similarity from the models)

Avg Spec is something in between
(gets the average of vocals/instrumentals)

 

E.g. following the above, we get the following setting:

“Max Spec / Min Spec”

Left side = about the Vocal stem/output

Right side  = about the Instrumental stem/output

"Max takes the highest values between each separation to create the new one (fuller sounding, more bleed).

Min takes the lowest values between each separation to create the new one (filtered sounding, less bleed).

Avg is the average of each separation."

More

 

For ensemble, avg/avg got the highest SDR, then worse results for respectively max/max, min/max and min/min.

 

For single MDX model, min spec was the safest for instrumental models and gave the most consistent results with less vocal residues than others.

Max spec - is the cleanest - but can leave some artifacts (if you don't have them in your file, then Max Spec for your instrumental like now might be a good solution).

Avg - the best of the both worlds and the only possible to test SDR e.g. at least for ensembles, maybe even to this day if it wasn't patched

 

“Max Spec/Min Spec” option

For at least a single instrumental model, it's the safest approach for instrumentals and universal for vocals. E.g. Min Mag/Spec in kae Colab using the old codebase for MDX models gives me the only acceptable results with hip-hop. I usually separate using a single model, but I cannot guarantee that Min Spec in UVR and manual ensemble will necessarily work exactly like Min Mag in Colab for a single model. But the explanation remains the same. The best option might even depend on a song.

 

TL;DR

For vocals bleeding in instrumentals

 

You can use Spectral Inversion for alleviating problems with bleeding in instrumentals.

Max Spec/Min Spec is also useful in such scenario.

 

You want less bleed of Vocal in Instrumental stem?

Use Max-Min

 

For bleeding instruments in vocals

 

Phase Inversion enabled helps to get rid of transients of the kick which might be still hearable in vocals in some cases.

 

Set Ensemble Algorithm: Min/Avg when you still hear bleeding.

 

If still the same, try Min/Max instead of Avg/Avg when doing an ensemble with Vocals/Instrumental output.

 

Also, you can resign from ensemble setting, and simply use only one clean model on the models list if the result is still not satisfactory.

 

 

Further explanations

Why not always go for Min-Max when you want the best acapella?

Why not always go for Max-Min when you want the best Instrumental?

 

So far, I hear Max-Min on Instrumental sounds more 'muddy/muffled' compared to Avg-Avg.

I bet this will be the same for acapella, but it's less noticeable (I don't hear it).

 

Hence, I think the best approach would be always going with Avg-Avg.

Then based on the outcome - after reviewing, tweak it based on your desired outcome,

and process again with either Min-Max or Max-Min.”

 

Min = less bleeding of the other side/stem (into this side/stem), but could get sound muddy/muffled

Max = more full sound, but potential it will have more bleeding

Avg = average, so a bit of all models combined

 

Average/Average is currently the best for ensemble (the best SDR - compared with Min/Max, Max/Min, Max/Max).

“Ensemble is not the same as chopping/cutting off and stitching, it blends/removes frequencies. If song 1 has high vocals in the chorus, and song 2 has deep vocals in the chorus, max will mash them together, so the final song will have both high and deep vocals

while min will remove both vocals”

"If I ensembled with max, it would add a lot of noise and hiss, if I ensemble with min it would make the overall sound muted gsep."

 

Technical explanation on min/avg/max

 

Max - keeps the frequencies that are the same and adds the different ones

“Max spec tends to give more artifacts as it's always selecting the loudest spectrogram frequency bins in each stft frames. So if one of the inputs have artifacts when it should be silent, and even if all other inputs are silent at the same instant, max spec will select the artifacts, as it's the max loud part of spectrogram here.” jarredou

 

Min - keeps the frequencies that are the same and removes any different ones

 

"if the phases of the frequencies are not similar enough min spec and max spec algorithms for ensembles will create noisy artifacts (IDK how to explain them, it just kinda sounds washy), so it's often safer to go with average"

 

by Vinctekan

"Min = Detects the common frequencies between outputs, and deletes the different ones, keeps the same ones.

 

Max = Detects the common frequencies between outputs, and adds the difference to them.

 

Now you would think that Max-Spec would be perfect since it should combine the all of the strengths of every model, therefore it's probably the best option

 

That would be the case if it wasn't for the fact that the algorithms that are used are not perfect, and I posted multiples tests to confirm this.

 

However, it still gives probably the cleanest results, however, there are a few issues with said Max_Spec:

 

1. Lot of instrumentals are going to be left within the output

2. If you are looking to measure quality by SDR, don't expect it to be better than avg/avg

 

The average algorithm, basically, combine all the outputs and averages them. Like the average function in Excel.

The reason why it works best is that it does not destroy the sound of any of the present outputs compared to Max_Spec and Min_Spec

 

The 2 algorithms still have potential for testing, though."

 

 

More on how the ensemble in UVR works

 

"Max takes the highest values between each separation to create the new one (fuller sounding, more bleed).

Min takes the lowest values between each separation to create the new one (filtered sounding, less bleed).

Avg is the average of each separation."

 

“[E.g.] HQ 1 would be better if the ensemble algorithm worked how I thought it did.

It was explained to me that [ensemble algorithm] tries to find common frequencies across all the outputs and combines them into the result, which to me doesn't actually seem to happen when HQ1 manages to bring vocals to the mix in an 8 model ensemble, how is it not like "okay A those are vocals, and B you're the only model bringing those frequencies to me trying to imply that they are not vocals" and discard them. I mean I am running max/max, but I swear all avg/avg and min/min do is lower the volumes [see enemble in DAW], It's hard to know without days of testing”

“If u try avg/avg it will get quite muddy on instr result than max/max. But some song if you put kim vocal 1 will get vocal residue on the result”.