> Thought process
Given that the language of the thought process can be different from the language of conversation, it’s interesting to consider, along the lines of Sapir–Whorf, whether having LLMs think in a different language than English could yield considerably different results, irrespective of conversation language.
(Of course, there is the problem that the training material is predominantly English.)
I’ve wondered about this more generally (ie, simply prompting in different languages).
For example, if I ask for a pasta recipe in Italian, will I get a more authentic recipe than in English?
I’m curious if anyone has done much experimenting with this concept.
Edit: I looked up Sapir-Whorf after writing. That’s not exactly where my theory started. I’m thinking more about vector embedding. I.e., the same content in different languages will end up with slightly different positions in vector space. How significantly might that influence the generated response?
The answer is yes, LLMs have different behavior and factual retrieval in different languages.
I had some papers about this open earlier today but closed them so now I can't link them ;(
That "native language" could be arbitrary embeddings.