I'm not the above poster, but my experiences generally line up with theirs.

The first 20% of context is usually very solid. Agent stays on task, is responsive and focused on the original goal.

The last 20% of context is usually pretty bad. Agent tends to loop, repeat, drift away into sub-goals (or non-goals).

The middle 60% is a bit of a grab-bag. Sometimes it's fine, sometimes it's not. It varies by task, prompt, and luck.

The above 25% and 50% limits seem like reasonable and simple rules to keep agents in the productive zones of usage.

It's generally not worth doing extensive testing to try to find your exact threshold unless you're running exactly the same prompt at scale, on similar data.

If you are in that spot, I won't actually recommend langchain tooling (I've been somewhat frustrated with it) but I think the concepts are worth understanding.

You can do something like https://docs.langchain.com/oss/python/langchain/test/evals. Trajectory evals in particular seem to suffer as context length grows (https://docs.langchain.com/oss/python/langchain/test/evals#t...).

I've used this for testing an agent that handles provisioning and permissions for developers with SaaS tooling (ex - if a developer asks, go provision an account [saml/oauth/etc] with the 3rd party, configure access, and let them know how to use it).

I'm about 6 months out of date at this point, so I won't speak for the latest models, but what was available early this year absolutely suffers from extended context lengths.