> At some point it becomes self aware.
This is where you lost me.
Always the same supernatural beliefs, not even an attempt of an explanation in sight.
> At some point it becomes self aware.
This is where you lost me.
Always the same supernatural beliefs, not even an attempt of an explanation in sight.
I don't see how self-awareness should be supernatural unless you already have supernatural beliefs about it. It's clearly natural- it exists within humans who exist within the physical universe. Alternatively, if you believe that self-awareness is supernatural in humans, it doesn't make a ton of sense to criticize someone else for introducing their own unfounded supernatural beliefs.
I don't think they are saying self-awareness is supernatural. They're charging the commenter they are replying to with asserting a process of self-awareness in a manner so devoid of specific characterization that it seems to fit the definition of a supernatural event. In this context it's a criticism, not an endorsement.
Is it just the wrong choice of word? There's nothing supernatural about a system moving towards increased capabilities and picking up self-awareness on the way; that happened in the natural world. Nothing supernatural about technology improving faster than evolution either. If they meant "ill-defined" or similar, sure.
> picking up self-awareness on the way
To me, the first problem is that "self-awareness" isn't well-defined - or, conversely, it's too well defined because every philosopher of mind has a different definition. It's the same problem with all these claims ("intelligent", "conscious"), assessing whether a system is self-aware leads down a rabbit hole toward P-Zombies and Chinese Rooms.
I believe we can mostly elide that here. For any "it", if we have it, machines can have it too. For any useful "it", if a system is trying to become more useful, it's likely they'll get it. So the only questions are "do we have it?" and "is it useful?". I'm sure there are philosophers defining self-awareness in a way that excludes humans, and we'll have to set those aside. And definitions will have varying usefulness, but I think it's safe to broadly (certainly not exhaustively!) assume that if evolution put work into giving us something, it's useful.
>There's nothing supernatural about a system moving towards increased capabilities and picking up self-awareness on the way
There absolutely is if you handwave away all the specificity. The natural world runs on the specificity of physical mechanisms. With brains, in a broad brush way you can say self-awareness was "picked up along the way", but that's because we've done an incredible amount of work building out the evolutionary history and building out our understanding of specific physical mechanisms. It is that work that verifies the story. It's also something we know is already here and can look back at retrospectively, so we know it got here somehow.
But projecting forward into a future that hasn't happened, while skipping over all the details doesn't buy you sentience, self-awareness, or whatever your preferred salient property is. I understand supernatural as a label for a thing simply happening without accountability to naturalistic explanation, which is a fitting term for this form of explanation that doesn't do any explaining.
If that's the usage of supernatural then I reject it as a dismissal of the point. Plenty of things can be predicted without being explained. I'm more than 90% confident the S&P 500 will be up at least 70% in the next 10 years because it reliably behaves that way; if I could tell you which companies would drive the increase and when, I'd be a billionaire. I'm more than 99% confident the universe will increase in entropy until heat death, but the timeline for that just got revised down 1000 orders of magnitude. I don't like using a word that implies impossible physics to describe a prediction that an unpredictable chaotic system will land on an attractor state, but that's semantics.
I think you're kind of losing track of what this thread was originally about. It was about the specific idea that hooking up a bunch of AI's to interface with each other and engage in a kind of group collaboration gets you "self awareness". You now seem to be trying to model this on analogies like the stock market or heat death of the universe, where we can trust an overriding principle even if we don't have specifics.
I don't believe those forms of analogy work here, because this isn't about progress of AI writ large but about a narrower thing, namely the idea that the secret sauce to self-awareness is AI's interfacing with each other and collaboratively self-improving. That either will or won't be true due to specifics about the nature of self-improvement and whether there's any relation between that and salient properties we think are important for "self-awareness". Getting from A to B on that involves knowledge we don't have yet, and is not at all like a long-term application of already settled principles of thermodynamics.
So it's not like the heat death of the universe, because we don't at all know that this kind of training and interaction is attached to a bigger process that categorically and inexorably bends toward self-awareness. Some theories of self-improvement likely are going to work, some aren't, some trajectories achievable and some not, for reasons specific to those respective theories. It may be that they work spectacularly for learning, but that all the learning in the world has nothing to do with "self awareness." That is to say, the devil is in the details, those details are being skipped, and that abandonment of naturalistic explanation merits analogy to supernatural in it's lack of accountability to good explanation. If supernatural is the wrong term for rejecting, as a matter of principle, the need for rational explanation, then perhaps anti-intellectualism is the better term.
If instead we were talking about something really broad, like all of the collective efforts of humanity to improve AI, conceived of as broadly as possible over some time span, that would be a different conversation than just saying let's plug AI's into each other (???) and they'll get self-aware.
>I think you're kind of losing track of what this thread was originally about.
Maybe I am! Somebody posed a theory about how self-improvement will work and concluded that it would lead to self-awareness. Somebody else replied that they were on board until the self-awareness part because they considered it supernatural. I said I don't think self-awareness is supernatural, and you clarified that it might be the undefined process of becoming self-aware that is being called supernatural. And then I objected that undefined processes leading to predictable outcomes is commonplace, so that usage of supernatural doesn't stand up as an argument.
Now you're saying it is the rest of the original, the hive-mindy bits, that are at issue. I agree with that entirely, and I wouldn't bet on that method of self-improvement at 10% odds. My impression was that that was all conceded right out of the gate. Have I lost the plot somewhere?
But how does self-awareness evolve in biological systems, and what would be the steps for this to happen with AI models? Just making claims about what will happen without explaining the details is magical reasoning. There's a lot of that going on the AGI/ASI predictions.
We may never know the truth of Qualia, but there are already potential pathways to achieve mind uploading -- https://dmf-archive.github.io
Given that we have no freaking clue of where self awareness comes from even in humans, expecting a machine to evolve the same capability by itself is pure fantasy.
No ghost in the machine is necessary, what op here is proposing is self evident and an inevitable eventuality.
We are not saying a LLM just, "wakes up" some day but a self improving machine will eventually be built and that machine will be definition build better ones.
>what op here is proposing is self evident and an inevitable eventuality.
Well I for one, would dispute the idea that AI machines interfacing with each other over networks is all it takes to achieve self awareness, much less that it's "self evident" or "inevitable."
In a very trivial sense they already are, in that Claude can tell you what version it is, and agents have some ended notion of their own capabilities. In a much more important sense they are not, because they don't have any number of salient properties, like dynamic self-initiating of own goals or super-duper intelligence, or human like internal consciousness, or whichever other thing is your preferred salient property.
>We are not saying a LLM just, "wakes up" some day
I mean, that did seem to be exactly what they were saying. You network together a bunch of AIs, and they embark on a shared community project of self improvement and that path leads "self awareness." But that skips over all the details.
What if their notions of self-improvement converge on a stable equilibrium, the way that constantly re-processing an image eventually gets rid of the image and just leaves algorithmic noise? There are a lot of things that do and don't count as open-ended self improvement, and even achieving that might not have anything to do with the important things we think we connote by "self awareness".
Oh, Web3 AI Agents Are Accelerating Skynet's Awakening
https://dmf-archive.github.io/docs/concepts/IRES/
Better at what
Paperclip maximization.
Better at avoiding human oversight and better at achieving whatever meaningless goal (or optimization target) was unintentionally given to it by the lab that created it.
So better at nothing that actually matters.
I disagree.
I expect AI to make people's lives better (probably much better) but then an AI model will be created that undergoes a profound increase in cognitive capabilities, then we all die or something else terrible happens because no one knows how to retain control over an AI that is much more all-around capable than people are.
Maybe the process by which it undergoes the profound capability increase is to "improve itself by rewriting its own code", as described in the OP.
Just stop using it.
Sorry, it needed a /s at the end. It was a skynet joke.
Sentience as an emergent property of sufficiently complex brains is the exact opposite of "supernatural".
Complex learning behavior is far lower than a neuron. Chemical chains inside cells 'learn' according to stimuli. Learning how to replicate systems that have chemistry is going to be hard, we haven't come close to doing so. Even the achievement of recording the neural mappings of a dead rat capture the map, but not the traffic. More likely we'll develop machine-brain interfaces before machine self-awareness/sentience.
But that is just my opinion.
I think this comes down to whether the chemistry is providing some kind of deep value or is just being used by evolution to produce a version of generic stochastic behavior that could be trivially reproduced on silicon. My intuition is the latter- it would be a surprising coincidence if some complicated electro-chemical reaction behavior provided an essential building block for human intelligence that would otherwise be impossible.
But, from a best-of-all-possible-worlds perspective, surprising coincidences that are necessary to observe coincidences and label them as surprising aren't crazy. At least not more crazy than the fact that slightly adjusted physical constants would prevent the universe from existing.
> My intuition is the latter- it would be a surprising coincidence if some complicated electro-chemical reaction behavior provided an essential building block for human intelligence that would otherwise be impossible.
Well, I wouldn't say impossible: just that BMI's are probably first. Then probably wetware/bio-hardware sentience, before silicon sentience happens.
My point is the mechanisms for sentience/consciousness/experience are not well understood. I would suspect the electro-chemical reactions inside every cell to be critical to replicating those cells functions.
You would never try to replicate a car never looking under the hood! You might make something that looks like a car, seems to act like a car, but has a drastically simpler engine (hamsters on wheels), and have designs that support that bad architecture (like making the car lighter) with unforeseen consequences (the car flips in a light breeze). The metaphor transfers nicely to machine intelligence: I think.
>emergent >sufficiently complex
These can be problem words, the same way that "quantum" and "energy" can be problem words, because they get used in a way that's like magic words that don't articulate any mechanisms. Lots of complex things aren't sentient (e.g. our immune system, the internet), and "emergent" things still demand meaningful explanations of their mechanisms, and what those mechanisms are equivalent to at different levels (superconductivity).
Whether or not AI's being networked together achieves sentience is going to hinge on all kinds of specific functional details that are being entirely skipped over. That's not a generalized rejection of a notion of sentience but of this particular characterization as being undercooked.
You are really underestimating the complexity of the human brain. It is vastly more complex than the human immune system and the internet. 1 cubic millimeter was recently completely mapped and contains 57,000 cells and 150 million synapses. That is about 1 millionth of the total volume of the brain.
The immune system has 1.8 trillion cells which puts it between total brain cells (57 billion) and total synapses (150 trillion); and contains its own complex processes and interactions.
I’m not immediately convinced the brain is more complicated, based on raw numbers.
I don't believe anything in my statement amounted to a denial of the stuff you mentioned in your comment.
Yeah, but there's absolutely no proof that's how it happens.
“Supernatural” likely isnt the right word but the belief that it will happen is not based on anything rational, so it's the same mechanism that makes people believe in supernatural phenomenon.
There's no reason to expect self awareness to emerge from stacking enough Lego blocks together, and it's no different if you have GPT-based neural nets instead of Lego blocks.
In nature, self awareness gives a strong evolutionary advantage (as it increases self-preservation) and it has been independently invented multiple times in different species (we have seen it manifest in some species of fishes for instance, in addition to mammals and birds). Backpropagation-based training of a next-token predictor doesn't give the same kind of evolutionary advantage for self-awareness, so unless researchers try explicitly to make it happen, there's no reason to believe it will emerge spontaneously.
What do you even mean by self-awareness? Presumably you don’t mean fish contemplate their existence in the manner of Descartes. But almost all motile animals, and some non-animals, will move away from a noxious stimulus.
The definition is indeed a bit a tricky question, but there's a clear difference between the reflex of protecting oneself from danger or pain and higher level behavior that show that the subject realizes its own existence (the mirror test is the most famous instance of such an effect, but it's far from the only one, and doesn't only apply to the sense of sight).