Various speakers' isolation (from e.g. podcast or movie)

- MVSEP Male/Female SCNet model

- MVSEP Male/Female MelRoformer model

- Aufr33 BS-Roformer Male/Female beta model | config | Colab (based on the model below)

- Male/female BS-Roformer model by Sucial | config for UVR | tensor match error fix
(if they sing at intervals [one by one] they cannot be separated)

- Multispeaker model on MVSEP

- BS-Roformer becruily & frazer Karaoke model

- bs_karaoke_3stem_giantailab (model | yaml | for use with this bs_roformer.py file for MSST | mvsepless: HF (and possibly) Colab

The model worked for people talking over vocals where karaoke models failed and only extracted backing and lead vocals where the speech landed in the latter. E.g. it can separate phone-recorded (or stylized like that) speech, mixed with loop repeating one phrase harmonized, and put both into different stems, and instrumental in the third stem. E.g. on mvsepless if you check the option to extract instrumental with inversion during inference, you'll get only a stem with quiet residues, unperceivable after mixing with instrumental and one of those vocal stems, so the result will sound fuller. Then you can use a limiter to avoid clipping if you don't want to turn down the volume (then use 32-bit float output, it's an option called high quality near the end of mvsepless inference page).

 

 

 

- Guide and script for WhisperX (separating people in recording)

 

- https://github.com/alexlnkp/Easy-Audio-Diarisation

 

- Spectralayers 11’s unmix multiple voices option

 

(for further research) - some of these tools might get useful:

https://github.com/dmlguq456/SepReformer (SOTA for 2 speakers)

https://paperswithcode.com/task/speaker-separation/latest

https://arxiv.org/abs/2301.13341

https://paperswithcode.com/task/multi-speaker-source-separation/latest

 

____________________________________________________________________

---