The way I phrase this to others is: Language models produce linguistically valid sentences, not factually correct sentences.