In my experience every single (AIO) water cooler I've tried has been significantly worse in terms of noise than a decent air cooler.

Specifically the pumps. Those things have an obnoxious high-pitched whine that I personally find unbearable, especially during low/idle workloads.

It's possible that the actal dB level is lower, but the frequency and sound characteristics matters. A lot.