I think your comment about inventing new words is an interesting one. One of the things that I believe limits our ability to discover new ideas is our ability to describe related concepts. For example, the reason we still can't have clear discussions on consciousness is probably partly due to the fact that the necessary concepts haven't been cemented in language. We need new language before we can describe consciousness.

I would guess LLMs are limited in their ability to be genuinely novel because they are trained on a fixed language. It makes research into the internal languages developed by LLMs during training all the more interesting.