They probably could be taxed but not a lot of countries do this and those that do have exemptions.
There's many arguments against taxation of primary residents but the strongest is probably that it punishes and hinders the ability for people to make sideways movements from similar properties to other similar properties, so it hinders housing liquidity, labour movement etc.