LLMs aren't architected to handle filter-style comprehensive search without setting them up with additional tools.
Asking an LLM for a list of every county in the USA for example, or every county with a population of more than 100,000 people.
Even if those county names and their populations are mixed up in their weights, the nature of next-token-prediction does not lend them to effectively answering comprehensive, detailed questions like that.
An agent system build on top of an LLM can do it, if it has access to tools which can help access eg a table of counties and then filter them with SQL or Pandas or similar.
Yes, I was assuming they'd use external tools. Using only the raw LLM doesn't sound like a good strategy.
Considering that agents are not a new concept, why isn't this a solved problem by now?
Agents are a very new concept.
We've got the early LLM-based AI agents in 2023, and it only became a popular, mainstream thing in 2025 - with Claude Code.