Probabilistic and deterministic are not opposites.
The brain is deterministic at the level of specific interactions, which process inputs in a highly chaotic (but still deterministic) manner into probabilistic outcomes.
The opposite of deterministic is random, i.e. in the quantum sense of truly no relationship between input and output.
There are probably some quantum effects in the brain here and there, but the vast majority of it is just traditional deterministic interactions networked together in such a complex system that the resulting behavior is much, much easier to predict in probabilistic terms than otherwise.
Ok right, but then I don't see the original point. Or like, you don't seem to be appealing to a hard science/soft science distinction here if the question revolves around what conceptual model is most "useful" to us. Useful for what? For predicting future actions? That makes sense, but does actually that speak to op's (possibly still fraught) metaphysical/semantic point here? It just feels like you are speaking over it, right? They could just as easily counter that a probabilistic model of the physical brain is not a very useful description of human intelligence, so why use it? You can't appeal to materialism here because you are already practicing some implied pragmatism in your commitments. You pretty much have as much ground to stand on as they do!
I am saying that there is no categorical distinction to be made on functional grounds between artificial intelligence and brains.
You cannot say an AI model cannot be intelligent because it's a probability machine, when all available evidence points toward natural intelligence also being generated by probability machines (much more complex ones, called brains).
But brains do much more than just predict tokens based on previously seen tokens. I think all the other things brains do are probably important for our intelligence.
So, LLM are just next token predictors, brains are next token predictors + many other things in addition, and that makes people still feel LLM are dumb even when they solve a lot of problems using tokens.
Of the vast uncertainties and philosophical exercises that we must face to bridge the chasm between where we are now, and where we will be when we understand intelligence, I can take comfort in claiming, with 100% accuracy, that our biology is not based on technology invented by Google in 2017.
> But brains do much more than just predict tokens based on previously seen tokens. I think all the other things brains do are probably important for our intelligence.
Like what?
What specific biological structure in the brain could be doing anything other than producing output as a function of 1) current electrical/chemical/thermal inputs and 2) previous electrical/chemical/thermal inputs?
No, brains doesn't just try to mimic pasts signals, LLMs do that but brains doesn't.
The way they make LLM solve problems is by adding a lot of logical jumps into its data, or break down different problems etc, and then as it predicts the text it predicts these logical jumps and then solves the problem. That is very different from how humans learn to solve problems, you don't feed them a billion different state transitions they have to encode to be able to navigate math, they learn to become proficient at math from a few hundred to a few thousand examples, that is fundamentally different from how LLM can learn.
That LLM are so slow learners that requires massive amount of data is a big reason its hard to make them smarter, and its caused by them being next token predictors. And the reason humans can learn with so little data is because we are not just next token predictors.
Okay, then what part of the brain is doing anything other than transforming an input into an output based on what inputs that part of the brain has previously been exposed to?
Well no, you changed the definition. LLMs don't just mimic what they've been trained on. They mimic what they're rewarded for. Like a brain!
Pre-training is just direct mimicry. A pre-trained LLM is very stupid and mostly useless. To become useful they are post-trained with a reward function.
Right, the CoT training step does work a bit more like normal training. But those do produce some very weird results, if you look at the "thoughts" CoT training leads to, I wouldn't say that CoT adds general intelligence, it teaches it to solve specific problems just like how any regular neural net training works.
The only general part of this training is the text prediction. All the other steps add specific skills to it and try to not hurt the generality as they do so, but we have no way to add general skill in the second step. So the general part is still the text predictor part. We know that training a neural net on specific tasks doesn't produce generality, we have tried that before, the text predictor is what let us get a glimpse of generality.
So then the text predictor try to solve problems by moving into the different specific skills there.
But yeah, you are right, without the CoT the text predictor wouldn't have any reliable skills to map to, so its still an important part of it. But the CoT doesn't add to the generality of the model, it just adds special skills it can utilize.
But, I agree what I am arguing now is a weaker statement. But I still believe its true, but I probably wont convince you of it.
This seems fine but things being the same "on functional grounds" is just one way things can be the same. Why should we privilege this framing as something definitive versus another conceivable one?
On functional grounds my bike has not a lot of distinction from a horse, but just, like, saying that doesn't tell me much about either. Or at least, it seems to leave out a lot of otherwise crucial details and differences..
What does it mean to you, this point of view? Are you truly coming from like a 20th century pragmatism point of view? Where what is most useful is what is right? Or are your trying to make a larger claim about nature? I think being clear about that would help focus your critique here.
Probabilistic and deterministic are not opposites.
The brain is deterministic at the level of specific interactions, which process inputs in a highly chaotic (but still deterministic) manner into probabilistic outcomes.
The opposite of deterministic is random, i.e. in the quantum sense of truly no relationship between input and output.
There are probably some quantum effects in the brain here and there, but the vast majority of it is just traditional deterministic interactions networked together in such a complex system that the resulting behavior is much, much easier to predict in probabilistic terms than otherwise.
Ok right, but then I don't see the original point. Or like, you don't seem to be appealing to a hard science/soft science distinction here if the question revolves around what conceptual model is most "useful" to us. Useful for what? For predicting future actions? That makes sense, but does actually that speak to op's (possibly still fraught) metaphysical/semantic point here? It just feels like you are speaking over it, right? They could just as easily counter that a probabilistic model of the physical brain is not a very useful description of human intelligence, so why use it? You can't appeal to materialism here because you are already practicing some implied pragmatism in your commitments. You pretty much have as much ground to stand on as they do!
I am saying that there is no categorical distinction to be made on functional grounds between artificial intelligence and brains.
You cannot say an AI model cannot be intelligent because it's a probability machine, when all available evidence points toward natural intelligence also being generated by probability machines (much more complex ones, called brains).
But brains do much more than just predict tokens based on previously seen tokens. I think all the other things brains do are probably important for our intelligence.
So, LLM are just next token predictors, brains are next token predictors + many other things in addition, and that makes people still feel LLM are dumb even when they solve a lot of problems using tokens.
> brains are next token predictors
Of the vast uncertainties and philosophical exercises that we must face to bridge the chasm between where we are now, and where we will be when we understand intelligence, I can take comfort in claiming, with 100% accuracy, that our biology is not based on technology invented by Google in 2017.
> But brains do much more than just predict tokens based on previously seen tokens. I think all the other things brains do are probably important for our intelligence.
Like what?
What specific biological structure in the brain could be doing anything other than producing output as a function of 1) current electrical/chemical/thermal inputs and 2) previous electrical/chemical/thermal inputs?
LLM doesn't just produce output as a function, they are much more specific: they predict text based on text they have been trained on.
Yes that is the item (2) parameter described above.
No, brains doesn't just try to mimic pasts signals, LLMs do that but brains doesn't.
The way they make LLM solve problems is by adding a lot of logical jumps into its data, or break down different problems etc, and then as it predicts the text it predicts these logical jumps and then solves the problem. That is very different from how humans learn to solve problems, you don't feed them a billion different state transitions they have to encode to be able to navigate math, they learn to become proficient at math from a few hundred to a few thousand examples, that is fundamentally different from how LLM can learn.
That LLM are so slow learners that requires massive amount of data is a big reason its hard to make them smarter, and its caused by them being next token predictors. And the reason humans can learn with so little data is because we are not just next token predictors.
Okay, then what part of the brain is doing anything other than transforming an input into an output based on what inputs that part of the brain has previously been exposed to?
> transforming an input into an output based on what inputs that part of the brain has previously been exposed to
You changed the definition there, for it to be like an LLM it should be:
> transforming an input into an output trying to mimic inputs that part of the brain has previously been exposed to
Anyone can see how that limits you a lot, and why that makes it so much harder for LLM to learn things properly than it is for humans.
Well no, you changed the definition. LLMs don't just mimic what they've been trained on. They mimic what they're rewarded for. Like a brain!
Pre-training is just direct mimicry. A pre-trained LLM is very stupid and mostly useless. To become useful they are post-trained with a reward function.
Right, the CoT training step does work a bit more like normal training. But those do produce some very weird results, if you look at the "thoughts" CoT training leads to, I wouldn't say that CoT adds general intelligence, it teaches it to solve specific problems just like how any regular neural net training works.
The only general part of this training is the text prediction. All the other steps add specific skills to it and try to not hurt the generality as they do so, but we have no way to add general skill in the second step. So the general part is still the text predictor part. We know that training a neural net on specific tasks doesn't produce generality, we have tried that before, the text predictor is what let us get a glimpse of generality.
So then the text predictor try to solve problems by moving into the different specific skills there.
But yeah, you are right, without the CoT the text predictor wouldn't have any reliable skills to map to, so its still an important part of it. But the CoT doesn't add to the generality of the model, it just adds special skills it can utilize.
But, I agree what I am arguing now is a weaker statement. But I still believe its true, but I probably wont convince you of it.
This seems fine but things being the same "on functional grounds" is just one way things can be the same. Why should we privilege this framing as something definitive versus another conceivable one?
On functional grounds my bike has not a lot of distinction from a horse, but just, like, saying that doesn't tell me much about either. Or at least, it seems to leave out a lot of otherwise crucial details and differences..
What does it mean to you, this point of view? Are you truly coming from like a 20th century pragmatism point of view? Where what is most useful is what is right? Or are your trying to make a larger claim about nature? I think being clear about that would help focus your critique here.