In practice a LVT is usually applied statistically. You run a regression on the features of the buildings across all parcels, determine a coefficient of value for various structures and usages (eg. multifamily is worth $X in cash flow for a certain size apartment, extra bedroom in a SFH is worth $Y, pool is worth $Z), subtract out the computed value of the structure, and then smooth across all properties in a given local area to compute the value of the land alone. That's what you're taxed on.

"Tax the land as if was being used for best use" is an oversimplification. The government can't realistically know what the best use of land is, or what value that would command. That's for the market to decide. But the point of an LVT is that you're taxed on the base value of the land (as measured by how much profit your neighbors are making off it), and so only usages of the land that are better than average remain profitable. If none of your neighbors can turn a profit on it either, tax will be negligible.