Well floating point operations never throw an exception, which I kind of like, personally. I would rather go in the opposite direction and change integer division by zero to return MAX / MIN / 0.

But NaN could be defined to be smaller or higher than any other value.

Well, there are multiple NaN. And NaN isn't actually the only weirdness; there's also -0, and we have -0 == 0. I think equality for floating point is anyway weird, so then why not just define -0 < 0.