I think the solution to your hang-up, which doesn't exist yet (AFAIK) is an LLM that is fine-tuned specifically for strict narrative context. The NPCs would then be trained on its own lore, and prompted accordingly. Hallucinations would for sure completely ruin the immersion, break the fourth wall, etc, so this thing would need to have some level of deterministic control. I'm hopeful for the immediate future though, seems like something that is totally doable even if it isn't 100% perfect at first.