___Best quality from YouTube and Soundcloud - how to squeeze out the most from the music taken from YT for separation___
Sometimes a better source just doesn’t exist, and only YouTube audio can be used for separation in some cases.
TL;DR
Use IntroC’s AAC/Opus combiner Python script (ref) / Colab
(if you find the results muddier than OG Opus, convert from 32-bit float to 24-bit).
The output is 44kHz (so Opus is downsampled).
Matches “QAAC -TVBR 86 quantization noise cancellation with the source audio”
Introduction to YT codecs
Audio on YT in most cases is available in two formats:
1) AAC (m4a) and Opus. As I mentioned, the latter appears for older or popular uploads, or videos uploaded in QHD or 4K. Most videos will have both formats available already. Currently only browsers without Opus support plau that audio stream (iirc Safari)
AAC on YT is @128kbps with 16kHz cutoff and 44kHz (that’s not artificial cutoff - that’s how the codec normally behaves when such bitrate is set).
2) Opus on YT is 96/128/152kbps with 20kHz cutoff (spectrum up to 24kHz for videos uploaded before ~2020+, but only with some aliasing above 20kHz, probably as a result of applied resampler) always 48kHz (44kHz audio is always upsampled with built-in resampler in Opus - that’s how the Opus works - it has always 48Khz output).
1) and 2) can be downloaded, e.g. via JDownloader 2 (once you downloaded one file, you must delete the previously shown entry in link grabber and add the link once more, and now pick the Opus (m4a is default) for download).
You can also use it online too:
https://ytdlp.online/ - create a free acount, otherwise downloading will be very slow; YouTube-dl web interface allowing downloading Opus when all others fail, just use:
-f 251 https://www.youtube.com/watch?vXXXXXXX
Afterwards, the download link will be shown at the end of the command line or in your user folder - RBM on it and save the link.
Note: sometimes youtube-dl might fail with error code/forbidden if you don’t add "&pp=" at the end of the link format above. It’s added to the link when you use search to find some videos for tracking purposes.
Be aware that sometimes Opus/format 251 is just not available for some videos - check it in stats for nerds when you open it on YT, see the all YT formats list)
These sites are frequently unreliable: https://cobalt.tools/ - (stopped supporting YT), probably it was just GUI for yt-dlp.
https://yt-dl-web.vercel.app/ - doesn’t always work
https://savefrom.net/ - (the one redirecting from ssyoutube.com amended links) - works when all the above fails, but sometimes doesn’t allow obtaining opus/webm or above 720p
(then downr.org will be able to download in better quality, but audio can be still in M4A).
Opus files downloaded from JDownloader are different from Opus in webm files seen by spectrum, but I can’t compare it with Cobalt as Spek doesn’t cooperate with its webm files in at least progressive mode which is “direct vimeo stream”. yt-dlp with -x argument might be free of the issue, but I haven’t checked yet.
Don't download as Opus from JDownloader 2. The quality will be affected.
Download always as webm in any quality - all qualities will contain the same Opus audio stream in the same bitrate.
Be aware that sometimes JDownloader wrongly reports bitrate as 96kbps, while when you demux the webm file with MKVToolnix-GUI and then with MKVExtractGUI2 (compatible with MKVToolnix v 20), the result Opus file (add extension manually afterwards) will have average bitrate of not much below 128kbps (that’s how VBR works).
Don’t download in OGG from any service like previously Cobalt allowed. It's a recompression from webm/Opus. OGG file is not on the variants list in JDownloader (and probably the same would be in CML tools like yt-dlp, so it’s simply not on YT).
However, it will have some additional information below 16kHz compared to Opus downloaded from JDownloader, probably because it was sourced from webm, and not JDownloader's Opus, but that’s it. Recompression here will add some ringing issues and compression artefacts. Details and spectrograms here.
Although, there might be some cases when other codecs are stored just in the OGG container, but confirming which case we deal with might be troublesome.
Sometimes it happens that m4a (AAC) sounds better than Opus. It all depends on the audio. It is more likely to happen if both have the same cutoff in spectrogram due to how it was uploaded on YT.
DIY
(potential alternatives to IntroC’s script)
#1 Joining frequencies with EQ method
1) Download both M4A and Opus audio from YT (if Opus is available for your video)
2) Upsample M4A to 48kHz (or else you won’t align the two files perfectly) with e.g. Resampler (PPHS) in Ultra mode in Foobar 1.3.20>Convert>...
3) To have frequencies above 16kHz from Opus and better sounding frequencies up to 16kHz from AAC, we will combine the best of the both worlds by:
a) applying resonant highpass on Opus file at 15750Hz in e.g. Ozone 8/9 EQ
b) aligning the track to M4A audio file (converted to 48kHz WAV 32), so added as separate track in free DAWs like Audacity, Cakewalk, Ableton Lite, or Pro Tools Intro (or eventually Reaper with its infinite trial).
Export the mixdown as WAV24. It should be more than enough.
Using brickwall highpass instead will result in a hole in frequency in the result spectrogram (check it in Spek afterwards, and also whether there are no overlap frequencies in the crossover - consider checking also linear phase in e.g. free QRange EQ).
#2 Manual ensemble in UVR
Files ensemble with Max Spec in UVR
Instead of EQ, you can use ensemble after manual upsampling of M4A file. You can have your files aligned in UVR.
Be aware that this method is not fully transparent, and produce files a little bit brighter, and still with cutoff, but not brickwall like in M4A.
Without upsampling step, you can use Max Spec method with great results also for Soundcloud which provides 64kbit/s opus and 128 kbp/s mp3 and 256kbp/s AAC.
You only need to amplify the mp3 file by 3dB. Align step is also necessary here, but it can be performed in UVR.
(fixed in UVR 5.6) Be aware that a bug in manual ensemble exists which forces 16 bit output despite choosing e.g. 32-bit float. To fix it, you need to execute regular separation of a song with any AI model with a 32 bit set, and then you need to return to manual ensemble without changing any settings now, so from now on it will retain 32-bit float in manual ensemble.
You can fix this by changing the 510th line of lib_v5/spec_utils.py to:
sf.write(save_path, normalize(output.T, is_normalization), samplerate, subtype='FLOAT')
then restart the program (you may not find that file if your UVR is not taken from source).
TBH, I didn’t compare directly the first EQ vs the latter Max Spec method, but the latter sounds brighter for sure than opus, and m4a.
“while it helps to make trebles more defined, it's a bit flawed, due ensembling 3 different compression methods, so 3 different compression flaws/errors and noises”.
PS. For YT I also tried downsampling Opus to 44 and to leave M4A intact, but it gave worse results (probably because of more frequencies affected by resampler in this case).
Why codec combining?
Audio file sizes and bitrate are the same for both formats. Knowing that the cutoff in AAC is not artificial, but codec without a doubt efficiently compresses only audio up to 16kHz, leaving everything higher blank and untouched, we can came to the conclusion that the frequencies up to 16kHz in AAC may sound better than in Opus, since the size and bitrate of both files is the same, and most likely bitrate in AAC is not used to frequencies above 16kHz, so full 128kbps bitrate is used only for frequencies up to 16kHz in AAC codec while in Opus for the whole spectrum up to 20 or even 24kHz in some old videos till around 2020, while keeping the same size, so that might be more harmful for frequencies up to 16kHz than in AAC. Later IntroC expanded on that idea and introduced more specific crossover points based on more experimentation.
PS. Later after some time, I received an explanation/reassurance on the purpose of this process here, saying it’s generally justified and Opus is actually better than AAC even above 9600Hz, so one more additional cutoff in AAC will be needed. Also, it might be worthwhile to use phase linear EQ to get rid of some coloration of the result file.
Experimenting with it, make sure that you don’t run into overlapping frequencies in area of bypassing (e.g. you can see it here as slightly brighter area above 9.6kHz up to 12kHz) to avoid it in e.g. in RX editor, one filtered signal needs to be 10Hz away from another one. I.e. if the lowpass is 12000 Hz, then the highpass is 12010 Hz. “But there is a catch with iZotope RX. The 10Hz away I described is only applied to the Copy operation (when you basically select the frequency range, and just CTRL+C by copying it). But there is also the Silence operation (when you select freq. range and press Delete, it eliminates the freq. in this range), and it is another way around: you need to get the other signal 10Hz inward, so they overlap. I.e.: 12000 Hz lowpass, 11990 Hz highpass. Here is the video demo: https://youtu.be/h5yE5cpqqMU” (it was written before the final Python script/Colab at the top)
#3 Bash script to automate the AAC/Opus quality combining from YT audio
introC later wrote his (older) bash script (before the Python one) which makes an alignment (so trimming 1600 samples from m4a), performs cutoffs and joins frequencies of both files for you - without an overlap issue (tested with white noise). The script works for multiple m4a and webm files with the same name. Probably, MSYS2 (or cygwin) is required to run this script on Windows or for W10/11 use WSL (read).
He also took a more conservative approach here and changed the cutoff frequency from 9600Hz to 1400Hz since AAC didn’t perform better in one song, but below 1400Hz it will be rather in every case. What cutoff is actually the best might be sometimes depending on a song. The script is subject to change.
#4 Method for better quality of instrumental leaks on YT by theamogusguy
“I did something really odd. (...) since you can only rip max 128kbps I did something really odd to get a higher quality instrumental:
I inverted the 128kbps AAC YouTube rip; into the original to get the acapella
I took the subtracted acapella and ran it through AI (Mel-Roformer 2024.10) to reduce the compression artifacts
I then inverted the isolated acapella and mixed it with the lossless to get an... unusual lossless instrumental file?
also the OPUS stream goes up to 20khz but I feel like the sample rate difference is gonna cause issues, so I ended up ripping AAC (OPUS is 48khz while most music is 44.1khz)”
#5 Resample downloaded Opus/webm file back to 44kHz with ARDFTSRC.
At least for playback, it can enhance audio quality giving stronger phantom center and somehow a bit less of subjective compression artefacts (e.g. in “Ariana Grande - God is a woman (Official Video)”). Download the following and execute it in CMD:
https://github.com/rorgoroth/mingw-cmake-env/releases/tag/latest
ffmpeg -i "C:\opus48.opus" -af asf2sf=dblp,ardftsrc=44100:quality=61656210:bandwidth=0.9941249 -c:a pcm_f32le C:\output44.wav
It will give you a 32-bit float file after downsampling, so you could turn down the volume before separation (probably you could even settle on some volume attenuation in the command itself). ARDFTSRC with these parameters was #1 on the resamplers chart on hydrogenaudio at the time. It requires 24GB of RAM to process faster (if you have, turn on two SSDs for pagefile if you have 16GB RAM or less). Be aware that the script doesn't track the progress, you need to just wait after executing tipo it finishes. In a limited RAM scenario, it can take even 10 minutes or more for a 5-minute song. ARDFTSRC can be also used in the latest Foobar2000 versions since 2.26 (it’s installed already, iirc it has a bit less resource consuming and a bit less SOTA settings, but settings higher than default genuinely take more time before playing any file).