While I agree with you, I find that sometimes the “experience” can improve.
The most common “artifact” of AV1 is to make things slightly more blurry for example. A common H.265 artifact is “blockiness”. I have re-encoded H.265 to AV1 and not only gotten smaller files that playback better on low-end hardware but also display less blockiness while still looking high-resolution and great colour overall.
I always encode 10 bit colour and fast-decode for re-encoding to AV1, even if coming from an 8 bit original.
But then you look at flashback scenes and wonder where the noise has gone.
A lot of movies have purposeful noise, blurriness, snow, and fake artifacts to represent flashback scenes. One level of compression often keeps them okay-ish (like you can tell side by side that it's different, but only when you know what to look for). But these are the scenes that get especially ruined by two layers of compression.