I know what you mean, but the boundary can get blurry in some cases.

A Canticle for Liebowitz for example mostly feels like some kind of fantasy but for the fact that the reader knows it's set in a post-apocalyptic nuclear wasteland.

Big chunks of the Peter F. Hamilton Void series is basically more or less set in a slightly magical Early Modern Venice.

The Laundry Files is strongly and deliberately in the middle ground of technology and magic, despite being ostensibly set in the present-day.

Stone Spring is an alternate history set in the Stone Age, but is not substantially more ahistorical than a non-fantasy historical novel about a person who didn't exist in reality doing things that never actually happened. Perhaps there's more focus on the engineering rather than fighting, romance, politics, murder and whatever else historical novels often revolve around, but building is as valid a human thing to do as plotting a regicide, say.

Generally, the concepts in both are the same: construct an "unreal" world and set a story in there, often with a projection of real-world issues onto the hypothetical substrate. Often the only real difference is if the unreal element is driven by magic, technology or a small change in a historical event. Sometimes it's a mixture. Sometimes the technology is treated as magic because the users don't understand it. Sometimes the magic is treated as a technology. Sometimes the historical divergence was thousands of years ago, sometimes it has only just happened in the story.

It would probably be more accurate to lump the whole lot under something generic like "speculative fiction" but that's not really a well-known term and has a slightly different meaning that blends into things like historical settings which may not be generally considered fantasy.