From my experience there's definitely context beyond the current set of LLM state. It's how they're able to regurgitate facts or speak at all.
From my experience there's definitely context beyond the current set of LLM state. It's how they're able to regurgitate facts or speak at all.
> regurgitate facts or speak at all.
Most of that is encoded into weights during training, though external function call interfaces and RAG are broadening this.