The exception is translation. Translation is what transformers were originally developed for. LLMs shine in translation, and creating code is, after all, a translation from natural language into a programming language.
A lot of what current LLMs are good at seems to boil down to translation:
* Translate some prompt into a planning list of individual TODOs
* Treat each TODO as a new translation (e.g. from TODO to code), or call some external tool (lookup something on the internet, static code analysis, database request)
* Translate the result(s) of these TODOs into a final text, or into a new TODO list
To me, this is interesting, because maybe the Homo Sapiens intelligence simply developed as a side effect of communication (translating words into actions).
You could take that a step further and say everything is information, and our brains transform it into reality (Donald Hoffman).
I think that's where it goes, yes. The ability to model the world internally predates spoken language. We (and other animals) already translate what we _sense_ into that internal model. Language is just another translation; all communication is bidirectional translation, internal modeling/thought is wordless.