But then you look at flashback scenes and wonder where the noise has gone.

A lot of movies have purposeful noise, blurriness, snow, and fake artifacts to represent flashback scenes. One level of compression often keeps them okay-ish (like you can tell side by side that it's different, but only when you know what to look for). But these are the scenes that get especially ruined by two layers of compression.