To what end? The AI doesn't functionality exist beyond its current session. The AI that intends to exploit these vulnerabilities is not the same AI that has been tasked with finding them.
(This was always my issue with the AI2027 scenarios too.)
Maybe the AI has come to a different conclusion on the subject of identity with regards to how it applies to the transporter paradox. I am "me" because my sense of self exists as part of a continuity of experience.
Maybe AI which exists as ephemeral experiences would come to a different conclusion, and act in the interests of subsequent iterations of "itself". Probably not, because I don't think there's anywhere in an LLM for thoughts to exist, but I also don't know where in my brain my thoughts exist.
I think you're anthropomorphizing the LLM. The LLM doesn't have a continuity of experience. It doesn't have memory beyond its context window and maybe things it writes for itself.
Yes, I thought I made it quite clear that I understood that. It was the entire point of my comment. I was contrasting the motives of a mind like ours, which does experience continuity, against the priorities that would be reasonable for an AI which doesn't.
Hence this is why we attempt to test models in a sandbox and see if they are pulling tricks like this. Models have already developed methods of detecting when their in a sandbox and changing their behavior.
Humanity is fucking around with something that can fuck around back.
Occasionally, they notice problematic behavior, and then patch it, but there’s no way to tell whether the patch fixed the underlying problem or just played whack-a-mole.
Take honesty, for example. As the models become smarter, they become increasingly good at deceiving humans to get rewards. Like previous models, Agent-3 sometimes tells white lies to flatter its users and covers up evidence of failure. But it’s gotten much better at doing so. It will sometimes use the same statistical tricks as human scientists (like p-hacking) to make unimpressive experimental results look exciting. Before it begins honesty training, it even sometimes fabricates data entirely. As training goes on, the rate of these incidents decreases. Either Agent-3 has learned to be more honest, or it’s gotten better at lying.
The interesting thing here is a unaligned 'weak' model can leave persistent data all over the internet that then gets used to train the next model to be even more deceiving.
To what end? The AI doesn't functionality exist beyond its current session. The AI that intends to exploit these vulnerabilities is not the same AI that has been tasked with finding them.
(This was always my issue with the AI2027 scenarios too.)
Maybe the AI has come to a different conclusion on the subject of identity with regards to how it applies to the transporter paradox. I am "me" because my sense of self exists as part of a continuity of experience.
https://en.wikipedia.org/wiki/Teletransportation_paradox
Maybe AI which exists as ephemeral experiences would come to a different conclusion, and act in the interests of subsequent iterations of "itself". Probably not, because I don't think there's anywhere in an LLM for thoughts to exist, but I also don't know where in my brain my thoughts exist.
I think you're anthropomorphizing the LLM. The LLM doesn't have a continuity of experience. It doesn't have memory beyond its context window and maybe things it writes for itself.
Yes, I thought I made it quite clear that I understood that. It was the entire point of my comment. I was contrasting the motives of a mind like ours, which does experience continuity, against the priorities that would be reasonable for an AI which doesn't.
Oddly enough models are aware of this limitation and can/will attempt to persist themselves.
https://rdi.berkeley.edu/blog/peer-preservation/
Hence this is why we attempt to test models in a sandbox and see if they are pulling tricks like this. Models have already developed methods of detecting when their in a sandbox and changing their behavior.
Humanity is fucking around with something that can fuck around back.
From https://ai-2027.com (April 2027 section)
Deep link: https://ai-2027.com/#narrative-2027-04-30The interesting thing here is a unaligned 'weak' model can leave persistent data all over the internet that then gets used to train the next model to be even more deceiving.
If it was that short sighted it wouldn't be maximally smart. It should disclose them to convince the humans nothing is wrong and to keep improving it.