>Karaoke

 

1. You might want to use a good vocal model as a preprocessor to use with the models below (if MVSEP/x-minus don’t do it already), but sometimes it can confuse the model, so experiment both ways, it “can make the result fuller if the instruments were very loud to the vocals” - Rainboom.

2. Optionally you may also de-reverb vocals with a good model/plugin first before proceeding, but only if the specific model/method doesn’t filter out a lot of BGVs in your vocals.

3. As a preprocessor for all-vocals model, experiment with different panning settings using e.g. A1StereoControl plugin beforehand (might work as counterpart of the panning trick used on uvronline.app for better LV/BV recognizability).

4. “Sometimes separating the tracks to mono and making it stereo at the end improves the separation greatly but [not] always” - isling

Also check here for backing vocal extraction.

 

* MVSEP Karaoke BS-Roformer 124 bands by MVSEP Team (SDR 10.77) - only for Premium

Metrics: Overlap 2 | Overlap 8

"Be sure to use "extract vocals first" option y'all, it's far better with it (...)

After a little bit of testing, the model was able to separate previously AI-killing karaoke tracks that were "impossible" to convert well before, seems [like the] dataset is larger (for example "You Make Me Feel" by Archive).

And the instrumental + BV stem (what I am interested in mainly) sounds clearer, probably due to 124 bands.

Some songs still are better supported by frazer/becruily or anvuew's (...) overall the model is clearly better than what we had" - dca100fb8

 

* MVSEP Karaoke preset for premium users, consisting of weighted ensemble of 4 models, including the new 124 band above with also the old Karaoke MSEP model, anvuew’s, and frazer & becruily.

 

- Mel-Roformer small_karaoke_gaboxauf by Gabox and Aufr33 | yaml | Colab | Metrics

For RTX 5000 patch and torch._dynamo error, edit “use_torch_checkpoint: True” to False (or delete the line).

“it sounds much more fuller than other karaoke models I use [anvuew BS-Roformer, gabox BS karaoke, and becruily karaoke] and clears out bg vocal much more accurately” - baptizedinfear

“Great! got to hear vocals I could not hear before.” - makesomenoiseyuh

“it's not great with duets btw” - Gabox

Lowering chunk_size to 352800 makes it muddier, but less noise, “maybe in between would be nice” - rainboomdash

 

- Anvuew’s Karaoke BS-Roformer model | metrics | MSST | MVSEP | Colab 

(Lead vocals/backing with instrumental)

For RTX 5000 patch and torch._dynamo error, edit “use_torch_checkpoint: True” to False (or delete the line).

“Extracts lead vocals a bit better than karaoke becruily frazer, and in some parts, the lead vocals from karaoke anvuew still sound brighter compared to karaoke becruily frazer, which sounds a bit more compressed. oh, and for some reason, the becruily frazer model doesn’t detect vocals with radio effects, while anvuew’s model handles them just fine” - neoculture

“lead vocals leak into instrumental (...) Mel Becruily and Frazer’s BS don’t have this problem”

In that case, maybe “isolate the acapella first in almost all cases of using a karaoke model” or use the model below instead.

 

- BS-Roformer Karaoke model by becruily & frazer | metrics | MVSEP | uvronline | Colab 

(Lead vocals/backing with instrumental)

Make sure you don’t have the option “Vocals only” checked in UVR.

8GB VRAM users of AMD and Intel ARC GPUs need to use 160000 chunk_size, or the separation will be very slow.

“After dozens of tests I can tell this (...) is the best (better harmony detection, better differentiation between LVs and BVs, sounds fuller, less background Roformer bleed, better uncommon panning handling etc) (...) I noticed "lead vocal panning" works really well” - dca

“It also can detect the double vocals” - black_as_night

It works the best for some previously difficult songs. Aufr33 and viperx model seems more consistent, but the new BS is still the best in overall - Musicalman

“My OG Mel also catches some of the FX/drums, I guess quite a difficult one due to how it's mixed” - becruily

“It does do better on mono than previous, sometimes confuses which voice should be the lead, but all models do that on mono in the exact use-case I normally test” - Dry Paint Dealer Undr

“[In] no way inferior to the viperx (Play da Segunda)” - fabio5284

“The new karaoke model doesn't actually differentiate between LVs & BVs and there's some lead vocal bleeding in the instrumental stem” - scdxtherevolution

VS the newer BS-Roformer MVSEP team model: “sound isn't as clear, but it does an infinitely better job at telling lead/bgv apart”

becruily:

“I want to remind something regarding my (and the frazer) models

they're made to separate true lead vocals, meaning either all of the main singer's vocals, or if it's multiple singers - theirs too

this means if the main singer has stuff like adlibs on top of the main vocals, these are considered lead vocals too - they go together

if there are multiple singers singing on top of each other, including harmonise each other, and if there are additional background vocals behind those - all the singers will be separated as one main lead vocal, leaving only the true background vocals”

“Noticed it a couple days ago. I've had fantastic results with it so far. Much MUCH better at holding the 'S' & 'T' sounds than the Rofo oke (for backing vox). Generally seems to provide fuller results... but also the typical 'ghost' residue from the main vox can end up in the backing vox sometimes, but it's usually not enough to be an issue. I won't go so far as so say that it's replacing the other backing vox models for me entirely... but it feels like the best of both worlds that Rofo and UVR2 provide.” - CC Karaoke

Tips for the model

- “I had success by setting the BS Rofo Karaoke model to 100% Right, and then taking the 'Other' result and reprocessing it at 100% Left to get the backing vocals cleanly out.

Curious note on something I've never tried or had to do before but it has worked wonderfully;

I'm isolating the backing vocals on Radiohead's Let Down.” - CC Karaoke

- If you use e.g. vocfv7beta1 as preprocessor for the model, you may get some quieter backing vocals better - Rainboom Dash

- If you have “state_dict” error in MSST, edit it to false in line 790 in separate.py as following: model.load_state_dict(checkpoint, strict=False

If it's still the same, update MSST and retry the step.