The OED says that the "house or building..." use of "premise" actually comes from an earlier legal meaning ("The subject of a conveyance or bequest..."). Even for those who (inaccurately) think etymology determines "correctness", this isn't an incorrect use of the word.