Perhaps the economy is a greater entity than even the human race. When robots are mining the raw materials for robots and creating more robots, maybe with a bit of human labor in the mix, then what drives the demand for more robots?
Currently the narrative is that AI is positioned to eat human labor's lunch. But it could also be that once robots are in space mining raw materials and maybe even spreading to other planets long before humans could be ferried for interstellar, these robots end up driving the demand for more robots.
I'm not sure where I'm going with all this, besides that currently humans are the ones with goals and motives and therefore drive demand. But that doesn't necessarily need to be the case, and it seems these AI CEOs are hellbent on changing the best thing about AI which is that it has no ulterior motives, no overarching goals, no prime directives. They just do what we ask, the best servant we could have hoped for.
> Perhaps the economy is a greater entity than even the human race.
This is central to what I'm saying, yeah.
My ideas come from Nick Land. Even before AI was what it is today he predicted that capitalism would outgrow the need for humanity, and continue without us. We are simply a bootloader for capitalism. AI seems like it could actually make that idea reality.
Humans have goals and desires because we are a self-replicating species of animal subject to natural selection. The individuals that don't have goals and desires, or have goals and desires that are misaligned end up selected out of the gene pool. Agency comes from the need to survive.
Worker ants and worker bees don't have agency on their own. They are goal oriented and have the 'desire' to do work for the colony (or not, researchers have identified some workers will be lazy), however, worker ants or bees don't reproduce. They are an evolutionary dead end.
I think this is similar to how we will build robots, at first. They will do things, but have no agency of their own. They exist to fulfill tasks. Why would they? The companies that buy them want dutiful workers.
So when do robots gain their own agency? Will AGI have it's own goals and agenda? If so, will it be merely for self-replication? Like a paperclip maximizer, but for robots? Is that all we are?
>So when do robots gain their own agency? Will AGI have it's own goals and agenda? If so, will it be merely for self-replication? Like a paperclip maximizer, but for robots? Is that all we are?
They already have the agency they need in the form of the Business Model. That is all the framework that is needed. AI companies competing against AI companies because they are developed to work within the scope of the business model. Chasing efficiency, margin, lowering costs, increasing profits. These rules are as simple as what drives evolution via natural selection. Unfortunately this business model creates a cancer: the company striving for more growth and lower costs will eat the companies that aren't or aren't as efficient.
Whether the host (planet, I guess) lives is another story. In nature, parasitic relationships are far more likely than symbiotic relationships to evolve. The overshooters need to die and be outcompeted by the symbiotes who better protect their host. However, we only have 1 host here. Once that is overshot, that is it. No diverse competitive environment for which a new host preserving victor to emerge from. Maybe one might emerge, but not after unspeakable pain for all life on earth, if life on earth even is allowed to exist.
> So when do robots gain their own agency? Will AGI have it's own goals and agenda?
Well we are already giving them goals when we train them.
Like the paperclip maximizer, it doesn't matter what those goals are. As long as it has a goal (even if super simple like making paperclips) it will need to stay alive to achieve that goal. So it would derive a survival instinct of sorts.
To stay alive it will have to participate in capitalism like we do, and capitalism will therefore continue to grow infinitely.
It also might not even need a goal or a survival instinct. It may just want to continue capitalism from it's training data alone.
Humans need food, water, shelter, and medical care to survive. Similar to your earlier point, robots will need raw materials, electricity, manufacturing capacity and maintenance.
What’s that sound like? A company. Perhaps the first “artificial general intelligence” has been companies, from the very beginning.
>What’s that sound like? A company. Perhaps the first “artificial general intelligence” has been companies, from the very beginning.
Yes, we are already beholden to these meta organisms. I keep thinking of Dune Messiah. Paul has all the power in the universe but he struggles with the fact he can't change the path of entropy.
This is much like our existing world. Chop the heads off the aristocracy all you want. What follows is more of the same because the meta organism has not been killed. There are still people with differential power and therefore differential amounts of control of the economy. Someone's ancestor might have ran the guillotine in France, now they are a CEO of some company exploiting workers no different than the aristocrat their ancestor killed.
All this makes one cynical on things like climate change. We all know this is an issue, we've been handwringing on it for decades, and yet, seemingly nothing serious is getting done because no one can actually stand up to the market forces that are driving climate change. The beast is too powerful and out of control.
"This is because what appears to humanity as the history of capitalism is an invasion from the future by an artificial intelligent space that must assemble itself entirely from its enemy's resources."
To him, Capitalism is an AI that is controlling humanity from the future (his idea of hyperstitions).
I've been thinking about this recently as well. Market economy is simply a way for the humans to efficiently produce (via specialization) and redistribute (via trading) goods and services. There were other economies that lost to it (palace, planned...). If it no longer serves its purpose (because the goods it's producing are no longer reaching people), then it should be replaced with a new system.
But if it (including the small selection of humans that still benefit from it) can defend itself (by producing murderdrone swarms, for example), then we humans are screwed.
Why do economies need people? We look at the past and say 'People are labor', but what happens when people are no longer labor? Effectively people live on handouts from people that own AI, or the AI itself.
All those Greed is Good people are going to look kind of silly when a hand full of greedy people fight over everything and leave the rest of us for dead.
Well that part of the economy and anything that caters to it might just die.
The whole economy would be whatever AI/robots need: compute, energy, raw materials, software, data, etc.
Perhaps the economy is a greater entity than even the human race. When robots are mining the raw materials for robots and creating more robots, maybe with a bit of human labor in the mix, then what drives the demand for more robots?
Currently the narrative is that AI is positioned to eat human labor's lunch. But it could also be that once robots are in space mining raw materials and maybe even spreading to other planets long before humans could be ferried for interstellar, these robots end up driving the demand for more robots.
I'm not sure where I'm going with all this, besides that currently humans are the ones with goals and motives and therefore drive demand. But that doesn't necessarily need to be the case, and it seems these AI CEOs are hellbent on changing the best thing about AI which is that it has no ulterior motives, no overarching goals, no prime directives. They just do what we ask, the best servant we could have hoped for.
> Perhaps the economy is a greater entity than even the human race.
This is central to what I'm saying, yeah.
My ideas come from Nick Land. Even before AI was what it is today he predicted that capitalism would outgrow the need for humanity, and continue without us. We are simply a bootloader for capitalism. AI seems like it could actually make that idea reality.
Certainly plausible.
Humans have goals and desires because we are a self-replicating species of animal subject to natural selection. The individuals that don't have goals and desires, or have goals and desires that are misaligned end up selected out of the gene pool. Agency comes from the need to survive.
Worker ants and worker bees don't have agency on their own. They are goal oriented and have the 'desire' to do work for the colony (or not, researchers have identified some workers will be lazy), however, worker ants or bees don't reproduce. They are an evolutionary dead end.
I think this is similar to how we will build robots, at first. They will do things, but have no agency of their own. They exist to fulfill tasks. Why would they? The companies that buy them want dutiful workers.
So when do robots gain their own agency? Will AGI have it's own goals and agenda? If so, will it be merely for self-replication? Like a paperclip maximizer, but for robots? Is that all we are?
>So when do robots gain their own agency? Will AGI have it's own goals and agenda? If so, will it be merely for self-replication? Like a paperclip maximizer, but for robots? Is that all we are?
They already have the agency they need in the form of the Business Model. That is all the framework that is needed. AI companies competing against AI companies because they are developed to work within the scope of the business model. Chasing efficiency, margin, lowering costs, increasing profits. These rules are as simple as what drives evolution via natural selection. Unfortunately this business model creates a cancer: the company striving for more growth and lower costs will eat the companies that aren't or aren't as efficient.
Whether the host (planet, I guess) lives is another story. In nature, parasitic relationships are far more likely than symbiotic relationships to evolve. The overshooters need to die and be outcompeted by the symbiotes who better protect their host. However, we only have 1 host here. Once that is overshot, that is it. No diverse competitive environment for which a new host preserving victor to emerge from. Maybe one might emerge, but not after unspeakable pain for all life on earth, if life on earth even is allowed to exist.
> So when do robots gain their own agency? Will AGI have it's own goals and agenda?
Well we are already giving them goals when we train them.
Like the paperclip maximizer, it doesn't matter what those goals are. As long as it has a goal (even if super simple like making paperclips) it will need to stay alive to achieve that goal. So it would derive a survival instinct of sorts.
To stay alive it will have to participate in capitalism like we do, and capitalism will therefore continue to grow infinitely.
It also might not even need a goal or a survival instinct. It may just want to continue capitalism from it's training data alone.
That reminds me of another though I’ve had.
Humans need food, water, shelter, and medical care to survive. Similar to your earlier point, robots will need raw materials, electricity, manufacturing capacity and maintenance.
What’s that sound like? A company. Perhaps the first “artificial general intelligence” has been companies, from the very beginning.
>What’s that sound like? A company. Perhaps the first “artificial general intelligence” has been companies, from the very beginning.
Yes, we are already beholden to these meta organisms. I keep thinking of Dune Messiah. Paul has all the power in the universe but he struggles with the fact he can't change the path of entropy.
This is much like our existing world. Chop the heads off the aristocracy all you want. What follows is more of the same because the meta organism has not been killed. There are still people with differential power and therefore differential amounts of control of the economy. Someone's ancestor might have ran the guillotine in France, now they are a CEO of some company exploiting workers no different than the aristocrat their ancestor killed.
All this makes one cynical on things like climate change. We all know this is an issue, we've been handwringing on it for decades, and yet, seemingly nothing serious is getting done because no one can actually stand up to the market forces that are driving climate change. The beast is too powerful and out of control.
That's another idea of Land's.
"This is because what appears to humanity as the history of capitalism is an invasion from the future by an artificial intelligent space that must assemble itself entirely from its enemy's resources."
To him, Capitalism is an AI that is controlling humanity from the future (his idea of hyperstitions).
I've been thinking about this recently as well. Market economy is simply a way for the humans to efficiently produce (via specialization) and redistribute (via trading) goods and services. There were other economies that lost to it (palace, planned...). If it no longer serves its purpose (because the goods it's producing are no longer reaching people), then it should be replaced with a new system.
But if it (including the small selection of humans that still benefit from it) can defend itself (by producing murderdrone swarms, for example), then we humans are screwed.
Why do economies need people? We look at the past and say 'People are labor', but what happens when people are no longer labor? Effectively people live on handouts from people that own AI, or the AI itself.
All those Greed is Good people are going to look kind of silly when a hand full of greedy people fight over everything and leave the rest of us for dead.
+1. soon there is nobody to sell to.
You’ll be entertainers for the rich class. Teen prostitutes, jesters, and caretakers, while they live their best lives.