What explains the emergent abilities of generative pre-trained transformers at massive-scale? Abilities that the smaller GTP’s don’t possess.
Simple programs can give rise to very complex behaviour. Conway’s game of live is Turing Complete and has four rules.
Conway’s Game of Live can simulate a Turing machine, can therefore implant a GTP.
Does that mean Conway’s Game of Life is conscious? I don’t think so.
Does it rule out Conway’s Game of life from implementing a system that has consciousness as an emergent ability?
I’m not convinced I know the answer.
> What explains the emergent abilities of generative pre-trained transformers at massive-scale?
I don't see why the abilities couldn't be an encoded modelling of enough of the world to produce those abilities. It seems like a simple enough explanation. Less data, less room to build a model of how things work. More data, sufficient room to build a model.
Conway's Game of Life is then not conscious in and of itself, because there's not enough in its encoded data to result in emergent behaviour beyond what we see.
If we expand it to also include a vast amount of data such as a Turing machine running an LLM then we can reasonably say we are closer to saying that that configuration of it is conscious.
It's not the firing-of-neurons mechanism and its relevant complexity or simplicity that make us conscious or not.
It's not the GoL algorithm that would make the machine conscious either.
It's the emergent behaviour of a sufficiently complex system.
The system _including_ its data.