Because it's based on physics, which is based on mathematics. Alternately, even if we one day learn that physics is not reducible to mathematics, both humans and computers are still based on the same physics.
"emergent superintelligent AI" is as much superstition as believing in imaterial souls. One company literally used the term "people spirits" to refer to how LLMs behave in their official communications.
It's a cult. Like many cults, it tries to latch on science to give itself legitimacy. In this case, mathematics. It has happened before many times.
You're trying to say that, because it's computers and stuff, it's science and therefore based on reason. Well, it's not. It's just a bunch of non sequitur.
You're mistaking the thing for the tool we use to describe the thing.
Physics gives us a way to answer questions about nature, but it is not nature itself. It is also, so far (and probably forever), incomplete.
Math doesn't need to agree with nature, we can take it as far as we want, as long as it doesn't break its own rules. Physics uses it, but is not based on it.
I will answer under the metaphysical assumption that there is no immaterial "soul", and that the entirety of the human experience arises from material things governed by the laws of physics. If you disagree with this assumption, there is no conversation to be had.
The laws of physics can, as far as I can tell, be described using mathematics. That doesn't mean that we have a perfect mathematical model of the laws of physics yet, but I see no reason to believe that such a mathematical model shouldn't be possible. Existing models are already extremely good, and the only parts which we don't yet have essentially perfect mathematical models for yet are in areas which we don't yet have the equipment necessary to measure how the universe behaves. At no point have we encountered a sign that the universe is governed by laws which can't be expressed mathematically.
This necessarily means that everything in the universe can also be described mathematically. Since the human experience is entirely made up of material stuff governed by these mathematical laws (as per the assumption in the first paragraph), human intelligence can be described mathematically.
Now there's one possible counter to this: even if we can perfectly describe the universe using mathematics, we can't perfectly simulate those laws. Real simulations have limitations on precision, while the universe doesn't seem to. You could argue that intelligence somehow requires the universe's seemingly infinite precision, and that no finite-precision simulation could possibly give rise to intelligence. I would find that extremely weird, but I can't rule it out a priori.
I'm not a physicist, and I don't study machine intelligence, nor organic intelligence, so I may be missing something here, but this is my current view.
I wonder if we could ever compute which exact atom in nuclear fission will split at a very specific time. If that is impossible, then our math and understanding of physics is so far short of what is needed that I don’t feel comfortable with your starting assumption.
Quantum mechanics doesn't work like that. It doesn't describe when something will happen, but the evolution of branching paths and their probabilities.
I'm just saying you're mistaking the thing for the the tool we use to describe the thing.
I'm also not talking about simulations.
Epistemologically, I'm talking about unknown unknowns. There are things we don't know, and we still don't know we don't know yet. Math and physics deal with known unknowns (we know we don't know) and known knowns (we know we know) only. Math and physics do not address unknown unknowns up until they become known unknowns (we did not tackle quantum up until we discover quantum).
We don't know how humans think. It is a known unknown, tackled by many sciences, but so far, incomplete in its description. We think we have a good description, but we don't know how good it is.
If a human body is intelligent, and we could in principle set up a computer-simulated universe which has a human body in it and simulate it forward with sufficient accuracy to make the body operate as a real-world human body has, we would have an artificial general intelligence simulated by a computer (i.e using mathematics).
If you think there are potential flaws in this line of reasoning other than the ones I already covered, I'm interested to hear.
We currently can't simulate the universe. Not only in capability, but also knowledge. For example, we don't know where or when life started. Can't "simulate forward" from an event we don't understand.
Also, a simulation is not the thing. It's a simulation of the thing. See? The same issue. You're mistaking the thing for the tool we use to simulate the thing.
You could argue that the universe _is_ a simulation, or computational in nature. But that's speculation, not very different epistemologically from saying that a magic wizard made everything.
Of course we can't simulate the universe (or, well, a slice of a universe which obeys the same laws as ours) right now, but we're discussing whether it's possible in principle or not.
I don't understand what fundamental difference you see between a thing governed by a set of mathematical laws and an implementation of a simulation which follows the same mathematical laws. Why would intelligence be possible in the former but fundamentally impossible in the latter, aside from precision limitations?
FWIW, nothing I've said assumes that the universe is a simulation, and I don't personally believe it is.
Again, you're mistaking the thing for the tool we use to describe the thing.
> aside from precision limitations
It's not only about precision. There are things we don't know.
--
I think the universe always obeys rules for everything, but it's an educated guess. There could be rules we don't yet understand and are outside of what mathematics and physics can know. Again, there are many things we don't know. "We'll get there" is only good enough when we get there.
The difference is subtle. I require proof, you seem to be ok with not having it.
Because it's based on physics, which is based on mathematics. Alternately, even if we one day learn that physics is not reducible to mathematics, both humans and computers are still based on the same physics.
And the soul?
So far, we have found no need for this hypothesis.
(Aside from "explaining" why AI couldn't ever possibly be "really intelligent" for those who find this notion existentially offensive.)
"emergent superintelligent AI" is as much superstition as believing in imaterial souls. One company literally used the term "people spirits" to refer to how LLMs behave in their official communications.
It's a cult. Like many cults, it tries to latch on science to give itself legitimacy. In this case, mathematics. It has happened before many times.
You're trying to say that, because it's computers and stuff, it's science and therefore based on reason. Well, it's not. It's just a bunch of non sequitur.
You're mistaking the thing for the tool we use to describe the thing.
Physics gives us a way to answer questions about nature, but it is not nature itself. It is also, so far (and probably forever), incomplete.
Math doesn't need to agree with nature, we can take it as far as we want, as long as it doesn't break its own rules. Physics uses it, but is not based on it.
I will answer under the metaphysical assumption that there is no immaterial "soul", and that the entirety of the human experience arises from material things governed by the laws of physics. If you disagree with this assumption, there is no conversation to be had.
The laws of physics can, as far as I can tell, be described using mathematics. That doesn't mean that we have a perfect mathematical model of the laws of physics yet, but I see no reason to believe that such a mathematical model shouldn't be possible. Existing models are already extremely good, and the only parts which we don't yet have essentially perfect mathematical models for yet are in areas which we don't yet have the equipment necessary to measure how the universe behaves. At no point have we encountered a sign that the universe is governed by laws which can't be expressed mathematically.
This necessarily means that everything in the universe can also be described mathematically. Since the human experience is entirely made up of material stuff governed by these mathematical laws (as per the assumption in the first paragraph), human intelligence can be described mathematically.
Now there's one possible counter to this: even if we can perfectly describe the universe using mathematics, we can't perfectly simulate those laws. Real simulations have limitations on precision, while the universe doesn't seem to. You could argue that intelligence somehow requires the universe's seemingly infinite precision, and that no finite-precision simulation could possibly give rise to intelligence. I would find that extremely weird, but I can't rule it out a priori.
I'm not a physicist, and I don't study machine intelligence, nor organic intelligence, so I may be missing something here, but this is my current view.
I wonder if we could ever compute which exact atom in nuclear fission will split at a very specific time. If that is impossible, then our math and understanding of physics is so far short of what is needed that I don’t feel comfortable with your starting assumption.
Quantum mechanics doesn't work like that. It doesn't describe when something will happen, but the evolution of branching paths and their probabilities.
I'm not talking about soul.
I'm just saying you're mistaking the thing for the the tool we use to describe the thing.
I'm also not talking about simulations.
Epistemologically, I'm talking about unknown unknowns. There are things we don't know, and we still don't know we don't know yet. Math and physics deal with known unknowns (we know we don't know) and known knowns (we know we know) only. Math and physics do not address unknown unknowns up until they become known unknowns (we did not tackle quantum up until we discover quantum).
We don't know how humans think. It is a known unknown, tackled by many sciences, but so far, incomplete in its description. We think we have a good description, but we don't know how good it is.
If a human body is intelligent, and we could in principle set up a computer-simulated universe which has a human body in it and simulate it forward with sufficient accuracy to make the body operate as a real-world human body has, we would have an artificial general intelligence simulated by a computer (i.e using mathematics).
If you think there are potential flaws in this line of reasoning other than the ones I already covered, I'm interested to hear.
We currently can't simulate the universe. Not only in capability, but also knowledge. For example, we don't know where or when life started. Can't "simulate forward" from an event we don't understand.
Also, a simulation is not the thing. It's a simulation of the thing. See? The same issue. You're mistaking the thing for the tool we use to simulate the thing.
You could argue that the universe _is_ a simulation, or computational in nature. But that's speculation, not very different epistemologically from saying that a magic wizard made everything.
Of course we can't simulate the universe (or, well, a slice of a universe which obeys the same laws as ours) right now, but we're discussing whether it's possible in principle or not.
I don't understand what fundamental difference you see between a thing governed by a set of mathematical laws and an implementation of a simulation which follows the same mathematical laws. Why would intelligence be possible in the former but fundamentally impossible in the latter, aside from precision limitations?
FWIW, nothing I've said assumes that the universe is a simulation, and I don't personally believe it is.
> a thing governed by a set of mathematical laws
Again, you're mistaking the thing for the tool we use to describe the thing.
> aside from precision limitations
It's not only about precision. There are things we don't know.
--
I think the universe always obeys rules for everything, but it's an educated guess. There could be rules we don't yet understand and are outside of what mathematics and physics can know. Again, there are many things we don't know. "We'll get there" is only good enough when we get there.
The difference is subtle. I require proof, you seem to be ok with not having it.