So the the longest time I had been struggling with popping and crackle when using HDMI audio on my GPU out to my TV. I've tried different cables, different graphics card drivers, new Windows installs, not using the nvidia HDMI audio drivers, uninstalling nvidia experience, even a new GPU, lots of things. I think the cause was a latency problem of some kind. I downloaded a latency checker program and indeed, when the audio was crackling there was a latency spike. I tried disabling various drivers, which helped, but never fully solved the problem.
But then, after I did a fresh Windows install, I found that the problem had expanded to affect even my primary audio. In addition, I was getting an abysmal frame rate even in older games that I had played before. Out of desperation, I reinstalled Windows again, but this time, I let Windows automatically install all my drivers, save for my GPU. Lo and behold! Perfect, crystal clear HDMI audio and I was getting more frames than I ever had out of my GPU! I was being a good little PC builder and making sure to download all the latest drivers from my motherboard manufacturer's site, but I was punished!
So yeah, from now on, I'm only going to install drivers if Windows doesn't install them or if it doesn't install the latest one (i.e GPU drivers, after I found RTX was disabled when using the Windows drivers). It seems times have changed and not only are the default Windows drivers perfectly fine, they seem to cause less issues.