> Everything that is physical is nothing but remixes and recombinations of a very small set of tokens.
We're not talking about "physical" with LLMs, we're talking about knowledge and creativity and reasoning, which are metaphysical.
The sum total of human knowledge cannot possibly be purely composed of remixes and recombinations, there has to be some baseline that humans invented for there to even be something to remix!
All of that is rooted in physics though.
Lnowledge and creativity absolutely are physical things. It's clear from brain injury studies that there are very localized and specific functions to this creativity.
Drugs also clearly have a very physical affect on these attributes.