> transforming an input into an output based on what inputs that part of the brain has previously been exposed to
You changed the definition there, for it to be like an LLM it should be:
> transforming an input into an output trying to mimic inputs that part of the brain has previously been exposed to
Anyone can see how that limits you a lot, and why that makes it so much harder for LLM to learn things properly than it is for humans.
Well no, you changed the definition. LLMs don't just mimic what they've been trained on. They mimic what they're rewarded for. Like a brain!
Pre-training is just direct mimicry. A pre-trained LLM is very stupid and mostly useless. To become useful they are post-trained with a reward function.
Right, the CoT training step does work a bit more like normal training. But those do produce some very weird results, if you look at the "thoughts" CoT training leads to, I wouldn't say that CoT adds general intelligence, it teaches it to solve specific problems just like how any regular neural net training works.
The only general part of this training is the text prediction. All the other steps add specific skills to it and try to not hurt the generality as they do so, but we have no way to add general skill in the second step. So the general part is still the text predictor part. We know that training a neural net on specific tasks doesn't produce generality, we have tried that before, the text predictor is what let us get a glimpse of generality.
So then the text predictor try to solve problems by moving into the different specific skills there.
But yeah, you are right, without the CoT the text predictor wouldn't have any reliable skills to map to, so its still an important part of it. But the CoT doesn't add to the generality of the model, it just adds special skills it can utilize.
But, I agree what I am arguing now is a weaker statement. But I still believe its true, but I probably wont convince you of it.