I don't think it's bad enough to think of them as an adversary: I think letting the compiler do a little of the boring stuff is ok, and for the most part I am rarely surprised by this stuff. I don't -O3 and -funroll-loops and just forget-about-it either - I choose each of the -f options individually based on a combination of my understanding of what kinds of transformations they do, testing, and whether I am prepared to continuously keep those things in mind when programming.
What are you thinking of?