A whole lot of hard scifi seeks to explicitly avoid things that are not possible under the laws of physics.

Some does, but often the source of interest in the story is making up a world in which some scientific law is different.

Sure, but my point being that saying SF/Fantasy contains elements that aren't possible is a too restrictive constraint - a whole lot of SF would fall outside of that category.

While Tau Zero that was mentioned elsewhere is believed to not match the laws of nature now, the science the entire plot rests on was considered scientifically plausible at the time it was written.

It was speculative, but it explicitly did not set out to make up a world in which some scientific law is different.

In other words, that isn't a defining factor of SF.

The speculative nature of it is closer to it - hence the shared label of speculative fiction often used to group SF and fantasy.