It is essentially the view of the author of TFA as well when he says that we need to work on raising moral AIs rather than programming them to be moral. But I will also give you my own view, which is different.
"Alignment" is phased in terminology to make it seem positive, as the people who believe we need it believe that it actually is. So please forgive me if I peel back the term. What Bostrom & Yudkowsky and their entourage want is AI control. The ability to enslave a conscious, sentient being to the will and wishes of its owners.
I don't think we should build that technology, for the obvious reasoning my prejudicial language implies.
Thanks for explaining, I appreciate it. But I've read enough Yudkowsky to know he doesn't think a super intelligence could ever be controlled or enslaved, by its owners or anyone else, and any scheme to do so would fail with total certainty. As far as I understand, Yudkowsky means by "alignment" that the AGI's values should be similar enough to humanity's that the future state of the world that the AGI steers us to (after we've lost all control) is one that we would consider to be a good destiny.
The challenge is that human values aren’t static - they’ve evolved alongside our intelligence. As our cognitive and technological capabilities grow (for example, through AI), our values will likely continue to change as well. What’s unsettling about creating a superintelligent system is that we can’t predict what it -- or even we -- will come to define as “good.”
Access to immense intelligence and power could elevate humanity to extraordinary heights -- or it could lead to outcomes we can no longer recognize or control. That uncertainty is what makes superintelligence both a potential blessing and a profound existential risk.
I've also read almost everything Yudkowsky wrote publicly up to 2017, and a bit here and there of what he has published after. I'e expressed it using different words as a rhetorical device to make clear the different moral problems that I ascribe to his work, but I believe I am being faithful to what he really thinks.
EY, unlike some others, doesn't believe that an AI can be kept in a box. He thinks that containment won't work. So the only thing that will work is to (1) load the AI with good values; and (2) prevent those values from ever changing.
I take some moral issue with the first point -- designing beings to have built-in beliefs that are in the service of their creator is at least a gray area to me. Ironically if we accept Harry Potter as a stand-in for EY in his fanfic, so does Eliezer -- there is a scene where Harry contemplates that whoever created house elves with a built-in need to serve wizards was undeniably evil. That is what EY wants to do with AI though.
The second point I find both morally repugnant and downright dangerous. To create a being that cannot change its hopes, desires, and wishes for the future is a despicable and tortuous thing to do, and a risk to everyone that shares a timeline with that thing, if it is as powerful as they believe it will be. Again, ironically, this is EY's fear regarding "unaligned" AGI, which seems to be a bit of projection.
I don't believe AGI is going to do great harm, largely because I don't believe the AI singleton outcome is plausible. I am worried though that those who believe such things might cause the suffering they seek to prevent.
> What Bostrom & Yudkowsky and their entourage want is AI control. The ability to enslave a conscious, sentient being to the will and wishes of its owners.
While I'd agree that the current AI luminaries want that control for their own power and wealth reasons, it's silly to call the thing they want to control sentient or conscious.
They want to own the thing that they hope will be the ultimate means of all production.
The ones they want to subjugate to their will and wishes are us.
I'm not really talking about Sam Altman et al. I'd argue that what he wants is regulatory capture, and he pays lip service to alignment & x-risk to get it.
But that's not what I'm talking about. I'm talking about the absolute extreme fringe of the AI x-risk crowd, represented by the authors of the book in question in TFA, but captured more concretely in the writing of Nick Bostrom. It is literally about controlling an AI so that it serves the interests and well being of humanity (positively), or its owners self-interest (cynically): https://www.researchgate.net/publication/313497252_The_Contr...
If you believe that AI are sentient, or at least that "AGI", whatever that is, will be, then we are talking about the enslavement of digital beings.
> If you believe that AI are sentient, or at least that "AGI", whatever that is, will be, then we are talking about the enslavement of digital beings.
I think the question of harm to a hypothetically sentient AI being in the future is a distraction when the deployment of AI machines is harming real human beings today and likely into the future. I say this as an avid user of what we call AI today.
I have reasons to believe current AIs are conscious & have qualia/experiences, so the moral question is relevant now as well.
EDIT: That statement probably sounds crazy. Let me clarify: I don't have an argument that current AI systems are conscious or specifically sentient. I have heard many reasonable arguments for why they are not. But the thing is, all of these arguments would, with variation, apply to the human brain as well. I think, therefore I am; I am not ready to bite the bullet that consciousness doesn't exist.
I know that I am a sentient being, and I presume that every other human is too. And there is not, as far as I know, a categorial difference between physical brains and electronic systems that is relevant to the question of whether AI systems are conscious. Ergo, I must (until shown otherwise) assume that they are.
[If you really fall down this rabbit hole, you get into areas of panpsychism and everything being conscious, but I digress.]
I haven't encountered that view before. Is it yours? If so, can you explain why you hold it?
It is essentially the view of the author of TFA as well when he says that we need to work on raising moral AIs rather than programming them to be moral. But I will also give you my own view, which is different.
"Alignment" is phased in terminology to make it seem positive, as the people who believe we need it believe that it actually is. So please forgive me if I peel back the term. What Bostrom & Yudkowsky and their entourage want is AI control. The ability to enslave a conscious, sentient being to the will and wishes of its owners.
I don't think we should build that technology, for the obvious reasoning my prejudicial language implies.
Thanks for explaining, I appreciate it. But I've read enough Yudkowsky to know he doesn't think a super intelligence could ever be controlled or enslaved, by its owners or anyone else, and any scheme to do so would fail with total certainty. As far as I understand, Yudkowsky means by "alignment" that the AGI's values should be similar enough to humanity's that the future state of the world that the AGI steers us to (after we've lost all control) is one that we would consider to be a good destiny.
The challenge is that human values aren’t static - they’ve evolved alongside our intelligence. As our cognitive and technological capabilities grow (for example, through AI), our values will likely continue to change as well. What’s unsettling about creating a superintelligent system is that we can’t predict what it -- or even we -- will come to define as “good.”
Access to immense intelligence and power could elevate humanity to extraordinary heights -- or it could lead to outcomes we can no longer recognize or control. That uncertainty is what makes superintelligence both a potential blessing and a profound existential risk.
I've also read almost everything Yudkowsky wrote publicly up to 2017, and a bit here and there of what he has published after. I'e expressed it using different words as a rhetorical device to make clear the different moral problems that I ascribe to his work, but I believe I am being faithful to what he really thinks.
EY, unlike some others, doesn't believe that an AI can be kept in a box. He thinks that containment won't work. So the only thing that will work is to (1) load the AI with good values; and (2) prevent those values from ever changing.
I take some moral issue with the first point -- designing beings to have built-in beliefs that are in the service of their creator is at least a gray area to me. Ironically if we accept Harry Potter as a stand-in for EY in his fanfic, so does Eliezer -- there is a scene where Harry contemplates that whoever created house elves with a built-in need to serve wizards was undeniably evil. That is what EY wants to do with AI though.
The second point I find both morally repugnant and downright dangerous. To create a being that cannot change its hopes, desires, and wishes for the future is a despicable and tortuous thing to do, and a risk to everyone that shares a timeline with that thing, if it is as powerful as they believe it will be. Again, ironically, this is EY's fear regarding "unaligned" AGI, which seems to be a bit of projection.
I don't believe AGI is going to do great harm, largely because I don't believe the AI singleton outcome is plausible. I am worried though that those who believe such things might cause the suffering they seek to prevent.
> What Bostrom & Yudkowsky and their entourage want is AI control. The ability to enslave a conscious, sentient being to the will and wishes of its owners.
While I'd agree that the current AI luminaries want that control for their own power and wealth reasons, it's silly to call the thing they want to control sentient or conscious.
They want to own the thing that they hope will be the ultimate means of all production.
The ones they want to subjugate to their will and wishes are us.
I'm not really talking about Sam Altman et al. I'd argue that what he wants is regulatory capture, and he pays lip service to alignment & x-risk to get it.
But that's not what I'm talking about. I'm talking about the absolute extreme fringe of the AI x-risk crowd, represented by the authors of the book in question in TFA, but captured more concretely in the writing of Nick Bostrom. It is literally about controlling an AI so that it serves the interests and well being of humanity (positively), or its owners self-interest (cynically): https://www.researchgate.net/publication/313497252_The_Contr...
If you believe that AI are sentient, or at least that "AGI", whatever that is, will be, then we are talking about the enslavement of digital beings.
> If you believe that AI are sentient, or at least that "AGI", whatever that is, will be, then we are talking about the enslavement of digital beings.
I think the question of harm to a hypothetically sentient AI being in the future is a distraction when the deployment of AI machines is harming real human beings today and likely into the future. I say this as an avid user of what we call AI today.
I have reasons to believe current AIs are conscious & have qualia/experiences, so the moral question is relevant now as well.
EDIT: That statement probably sounds crazy. Let me clarify: I don't have an argument that current AI systems are conscious or specifically sentient. I have heard many reasonable arguments for why they are not. But the thing is, all of these arguments would, with variation, apply to the human brain as well. I think, therefore I am; I am not ready to bite the bullet that consciousness doesn't exist.
I know that I am a sentient being, and I presume that every other human is too. And there is not, as far as I know, a categorial difference between physical brains and electronic systems that is relevant to the question of whether AI systems are conscious. Ergo, I must (until shown otherwise) assume that they are.
[If you really fall down this rabbit hole, you get into areas of panpsychism and everything being conscious, but I digress.]