Or a 3rd option is it should not be allowed similar to dividing by 0.

What do you mean "not allowed"? Throwing a compile or runtime error? Many languages allow division by zero, and x/0 typically gives inf unless x is 0, then x/0 gives nan. If x is negative then x/0 gives -inf. Of course this all can get tricky with floats, but mathematically it makes sense to divide by zero (interpreted as a limit).

For NaNs, maybe in some domains it could make sense, but eg I would find it impractical when wanting to select rows based on values in a column and stuff like that.