The noise problem is pretty easy to mitigate by choosing 2U servers instead of 1U. The latter are forced by the form factor to use smaller, higher speed fans.
A bigger issue for enterprise hardware is that it's optimized for performance per watt under load, not idle power consumption. Running a mostly-idle rack server 24/7 can result in a pretty sizable electric bill. This also depends heavily on the model. Some will idle at ~50 watts, others at ~300, but both of these are significantly higher than a Raspberry Pi or an old laptop which for personal use will generally do the job.
Business class desktops are also a good alternative here. Many models have pretty reasonable idle power consumption (check this for yourself, I've seen 6W but also 60W) and then you get a couple of drive bays and PCIe slots and expandable RAM which you don't get from a Raspberry Pi.
These days, pretty much the only thing that makes sense is a mini PC. AMD laptop chips generally trade blows with Apple stuff on power efficiency when you thrash them, and you get a surprisingly capable machine for not very much money.
It's really not worth it to run old hardware 24/7 unless it's making money. Buying a new machine of equivalent capability is (normally) pretty cheap, and it doesn't take very long for the power savings to pay for themselves.
They can be had with fairly respectable specs too. Certainly enough to play around with small local models.
"When you thrash them" is kind of the issue. There are ten year old business desktops with a <10W idle power consumption. If your use for it is to have something to rsync files to and host your personal website and the like, even old hardware is going to average 99% idle. There is no meaningful power savings from newer hardware unless you're consistently putting it under significant load.
Some of the newer hardware is actually worse because the idle power consumption of PCs since around 2010 is determined in significant part by the low-load efficiency of the power supply. Brand new machines with the wrong power supply can use several times as much power at idle as ten year old machines with the right power supply. Annoyingly, power supply efficiency at idle is rarely documented so the only thing to do is measure it.