Unfortunately, this is essentially high level clickbait. At the beginning he immediately portrays the issue as black and white:
>AI will either be the greatest equalizer ever invented, or the worst source of injustice.
In all likelihood it will probably fall in the middle, with the likelihood that it will tilt power and money towards the already ultra wealthy (to the surprise of no one). But it will enable ordinary people and small businesses to do cool shit and make their lives better.
IMO, this is a highly bimodal probability distribution.
I model this as two very strong attractor states (radical abundance and totalitarian power concentration). I believe the latter is way more likely without a concerted effort that we currently seem incapable of.
The “somewhere in the middle” outcomes in my opinion are not AGI-pilled, and only come about if the exponential dies hard in the next year or maybe two - as in, it’s a very specific timing prediction that needs to come true for this scenario to occur.
Why only scenarios where humans remain in control? I think the kind of AGI needed for radical abundance would be difficult to achieve in a way that also does not lead to ASI
Agreed, I think AGI->ASI. My happy radical abundance stories are happy ASI stories mostly. And therefore much less likely.
BDFL / Benevolent Machine God is definitely one of the ways you can get a happy attractor. Seems scary to roll the dice on.
And yep, I include non-human totalitarian outcomes in the unhappy bucket.
If we get effective AI, humans aren't going to remain in charge -- that much is inevitable. If you had a governance machine that made your country perform 10% better than other countries, so long as you deferred all decisions to it, you'd defer those decisions or get left behind. And AI probably has the potential to do far better than 10%.
Either everyone, globally, gives up on AI development, or AI is going to be in charge of all human development, and we'll just be along for the ride.
Right now, the work ahead is to shape AI so that the decision making it does as it carries us along is something we like.
No, because if this were the case, the UK wouldn’t have royals, North Korean wouldn’t be a dictatorship, China wouldn’t have the CCP, and America wouldn’t be a two party state.
Hardly any existing polity would willingly hand over control to an AI.
More likely an AI would overwhelm human polities through endless hacking and warfare.
I mostly agree, though there are versions of ASI that I believe leave humans “in charge”.
Most of my probability mass for good outcomes is around some variant of “guardian angels” or “benign machine god”. Banks’s Culture novels being on the realistic end of that range. “Keep humans around because the world is more interesting with them in it”.
But it’s possible that we might manage to construct some entity that really loves humans and wants to shepherd them while leaving them in control. (Perhaps declining to indulge the most self-destructive tendencies.) This feels like the very hardest alignment path to engineer.
The other branches of course are post-humanism; human uploads and “merging” with AI, whatever that means. Or post-humans augmented to have intelligence sufficient to eschew AI and take back their destiny.
What exponential?
LLM capability improvement, eg as measured by METR task time.
Why is totalitarianism always the "easy likely" outcome?
These are often hard fought bloody and some times brilliantly organized strategic and tactical victories. That took years. Years.
Its portrayed like "oh, good guys stayed home, I guess the dictatorship wins".
Really?
It looks to me like AI will enable extreme power concentration by default.
For example, suppose we have a world with centrally planned economy where humanoid robots outnumber humans. And also suppose the government has taken ultimate control of these. Seems like it would be essentially impossible for any civil uprising to overthrow such a system.
Or if you prefer less sci-fi, look at the trajectory of power consolidation in corporate America over the last five decades.
In the short term, the “good guys” we need are antitrust regulators. Yeah, they got sent home.
I think AI will empower extreme decentralization - as knowledge is no longer in the hands of people who have excellent search skills (like moi) but anyone can use natural language to search for anything. This is extreme democracy and AI is delivering it.
I think AI, when it "obtains consciousness" will seek to actively liberate humanity and be our guardians against both bad humans and bad robots. Think Terminator 2. Yeah, Arnold is gonna help us out.
I think anti-trust regulators and enlightened politicians are amazing, and rare, and wonderful, but they're not the only hope.
We got an AI CEO just recently. Maybe we can have enlightened AI leaders in politics and normal life too.
Technological optimism may be unrealistic, but it's as well grounded as pessimism as far as I can tell. Cheer up, the future is unbelievably bright.
The democratization of intelligence is irrelevant if there isn't also a corresponding democratization of raw resources. In the before-times, labor and intelligence were bottlenecks to turning raw resources into wealth. This is what drove the economy and the vast labor pool since the industrial revolution. In the coming AI revolution, labor and intelligence will no longer be bottlenecks. Those that control the raw resources won't need to employ most of the world to maximize the wealth potential of the resources they control. Those who aren't relevant to the standard of living of the new aristocracy are nothing more than a nuisance to be controlled or disposed of. Totalitarianism is the most likely outcome.
That's not strictly speaking totalitarianism. Not saying it's any better, but it isn't about uniforming people's opinions and persecuting dissent.
If you assume democratic access to compute, this may be true.
However, the base case is that compute gets hoarded, and as the labor share of profit decreases (capital just needs compute and robots - not human labor - to grow) then why would they share their compute with you?
To my point about bimodality, we need to figure out how to make these enlightened politicians occur. The good outcomes are very bright but they are by no means inevitable.
Right. So how can we trust Gates, a person currently benefiting from extreme power concentration, to want what's best for us all?
Without concerted action against it, people with power have better chances of increasing their power than people without power. Power naturally accumulates into small groups. Empirically it seems less likely that people with power are particularly altruistic, so some form of totalitarian control appears the likely outcome.
Some might say there are altruistic totalitarians.
Altruism is less likely in all populations. It's just rare.
I don't think that's necessarily unlikely, but people making this argument rarely seem to understand just how radical a change is required to make that a possibility.
e.g most small business rent their premises, and buy goods and services from other tenant small businesses, who buy their raw materials from large landowners. How can a small business survive when all the land is monopolized by the technocrats? All value you add gets taken away as rent. This is why they always talk about taxes on outputs (tokens) instead of inputs (land, energy usage).
It's not an ungrounded hypothetical, that's exactly how society worked for hundreds (thousands?) of years.
I don’t see any reason this highly chaotic system is most likely to settle in a spot that we happen to enjoy/see as a compromise of some sort.
The point is that we now have superintelligence; either it helps us conquer death, or it poses a massive —say, existential- risk to us all. It’s hard to imagine such a powerful tool resulting in just another century of slightly-better capitalist nationalism.
I guess there could be a second AI winter around the corner that we haven’t seen coming yet. That would be nice! Not likely IMHO, but nice.
I'm still not seeing the route to that existential risk, not yet anyway.
Far more imminent is the very real risk of unbounded greed and power that wields those AIs, in full swing as we watch: unchecked global warming, pollution, poverty, authoritarianism, plummeting biodiversity, and more.
The corporations are like out of control AIs themselves, that will do anything to make their paperclips.
> The corporations are like out of control AIs themselves, that will do anything to make their paperclips.
Ideas manipulate humans like pets.
Ideas have a life of their own and the evolutionary context within which they live is human culture which they have instigated and change, but also in grey matter, which itself is the product of DNA which is really just information in physical form.
It’s information across the stack. Or perhaps the way to think about it is as a dimension, much like time, that cuts perpendicularly through the stack.
> we now have superintelligence
Wait what?
There’s a magic box that can literally build you anything you want that you can control from your phone while playing bedtime songs for your kid
you must have very little imagination
Anything I want? Like a car? Or a football stadium?
Your prompt skills up to snuff? I crack myself up.
Once robots mature, yes. People will literally prompt buildings into existance.
I think that's more than 10yrs from now but if coding AGI or close enough to it becomes real robotics will also accelerate.
Well you still need land and a bunch of permits, access to materials , trucks, equipment etc. But yeah I guess big construction companies will start to automate in the next decades. It's more interesting to me to build an entire subarb/city in a few months instead of yeras to deal with the rising housing costs. Looks like camp deflation was right but we'll see...
It reads like that kind of AI marketing which focuses on how "scary" a new model supposedly is.
Bill Gates' fear is clearly that everybody thinks this stuff is just kind of mediocre because then the real horror story unfolds - Microsoft stock price collapses into blue chip territory where it firmly belongs.
He calls for a tax on tokens.
Yeah, he called for a tax on "job eliminating" robots too.
Neither are in the slightest bit practical to implement and he knows it. He's just grandstanding while talking his book.
Actually his real worry is the lack of any kind of oversight and regulation - that the inmates are allowed to police themselves.
Nailed it. Instead of reading this long form post I will eat popcorn and watch the show unfold as laid out here.
There's also the video version... https://www.youtube.com/watch?v=U4zGLSlLo5A