Doesn't this seem to be where all domains are headed?
I get kinda freaked out when I feel like all the AI "utopianists" haven't taken the next logical step of thinking about what society looks like when humans are subpar in every domain (and you may argue this won't happen, though I'm becoming more and more a believer that it will, but my point is the utopianists believe that this absolutely will happen, and that it's also a wonderful thing). How motivated do you think folks will be to do the hard cognitive work to focus on things like math problems when there is a good chance AI will do it better?
Doesn't the premise that there's something inferior about these AI solutions, imply that there is something superior about human intelligence and that there will continue to be some kind of useful work for humans to do?
> Doesn't the premise that there's something inferior about these AI solutions, imply that there is something superior about human intelligence and that there will continue to be some kind of useful work for humans to do?
Not necessarily. The "something superior about human intelligence" may have dependencies that "these AI solutions" are able to eliminate, such as the motivation to refine intellectual talent to a high level. Basically, AI could kick the ladder out from under human intelligence but be incapable of actually surpassing it in important ways, enabling a burst of advancement that's also a dead end. Sort of like https://en.wikipedia.org/wiki/The_Road_Not_Taken_(short_stor....
So the AI could be inferior but there's still no useful work for humans, because the environment doesn't allow them to work up to that level anymore.
This is kinda feeling a bit like SBF's coin flip bet: https://www.businessinsider.com/sam-bankman-fried-coin-flip-.... Achieve human-superior AGI this generation or humanity stagnates.
I don't think humans necessarily become subpar when AI can do most of the work. Taking my own personal example, I have far more intellectual curiosity and improved my skills in programming far more with Claude Code than for 15 years of programming without AI simply because I was bogged down by boilerplate and grunt work. Now that AI handles most of the boilerplate and grunt work and can handle harder and harder problems, I have the time and space to work on unexplored frontier problems.
So, no, my skills have not become subpar, but have only become stronger because of the presence of AI.
So when AI is also better at coming up with the tasks and already solving the frontier problems? 1M AI agents running full time thinking at 10M times faster than humans, with 100 times the intelligence in each agent? Any human ideas regarding "frontier problems" or "what the AI should focus on" are irrelevant. We have automated our own thinking and human cognition will be about as important as being "the best at abacus". Its an interesting party trick, but in no way useful.
Before AI, did you find that the existence of human expertise in the world far surpassing yours 'automated your thinking and cognition' or limited your intellectual scope to inventing party-tricks? I've always found greater intelligences than mine are like runways for curiosity. It's hard to believe anyone in their presence would fall into a stupor instead of being highly stimulated by them - and I can't see why it would be any different with ASI. Actually, this "AI undoes your cerebral strapping" argument seems exactly like the kind of slovenly thinking it wants to blame on AI.
yes to a degree. When I was 16, my idea about "what cutting edge thing to work on" was pointless, I couldn't make a meaningful contribution. Most developer's work is mundane and wrote. Things they think are "cutting edge" are actually just ideas they haven't considered, or markets they don't realize are saturated with really good players already. It takes time, effort, and knowledge to make a meaningful contribution. But if those fields are saturated by players that are beyond human ability, then its a bit like trying to become a competitive chess player when the bar for entry is stockfish -- no human will ever again pass this bar. At a certain point, the bar for being paid for intellectual contribution will surpass human abilities. The things humans can understand and ideas they contribute will not be worth much.
To make an unrelated anology, the presence of Lebron James definitely dampens my ability to contribute meaningfully to basketball. Even with all my practice, I cannot achieve his level. But if he didn't exist, and the best 90% of basketball players were eliminated, then I might be able to meaningfully contribute. Of course, I can still play basketball as a hobby, but you were talking about making meaningful contributions on hard novel problems at the frontier of knowledge and ability. That frontier is moving away from humans and will soon be the domain of electronic experts.
I think the main point is the ubiquity of AI intelligence. If there is a human far surpassing me, but unavailable to me, then I have to work through it myself. If the answer is a prompt away, then why bother.
> what society looks like when humans are subpar in every domain
> when AI can do most of the work
You're not really responding to the core hypothetical of his comment
To me it reads as that: for utopians, you may benefit from LLMs now, but they'll still surpass you later, what then ?
Thank you for putting it so succinctly. I find that lots of people get "stuck" in the current state of things - they think that AI will continue to improve, but they inherently believe (perhaps subconsciously) that humans will always have the edge in things like "ingenuity", "inventiveness" or "curiosity".
I don't blame them, I also believed that until very recently. I thought there was something unique and special about sentience and consciousness, but my thinking has changed in that I don't think machines need to develop the sentience we experience to still surpass us cognitively along all dimensions.
But iff you really examine what grounds your beliefs, and then consider that AI very well could surpass humans even in domains like curiosity and real creativity (at several orders of magnitude faster), it starts to look like a very, very uncomfortable future for humans IMO.
I saw this quote on Twitter:
> Best outcome: we become pets
> Worst outcome: we become cows
I don't get this take at all. Do you feel inferior that bears are stronger than you? Or that a machine can lift more weight than you? Or that a calculator can calculate faster than you?
These AI things are machines which are completely under our control. Even if they become more intelligent than us, that doesn't mean that it has authority over us. There are plenty of very intelligent humans which work under the authority of much less intelligent humans. Many engineers are smarter than their CEOs.
What makes you think that just because an AI is more intelligent that it will not be completely subject to our authority? It's just a tool in our hands.
> What makes you think that just because an AI is more intelligent that it will not be completely subject to our authority? It's just a tool in our hands.
Because it would be like 100 chimps trying to make a million humans subject to their authority.
Geoffrey Hinton, considered "The Godfather of AI" for his pioneering work on neural networks, and who now says he somewhat regrets his life work due to the existential risks, has said similar things, e.g. "How many examples do you know of a more intelligent thing being controlled by a less intelligent thing? There are very few examples"
So, what if they surpass me? It's still a tool under my control. Many engineers work under the authority of a CEO, even though they're smarter than them. Just because AI might be more intelligent doesn't mean it's still not a machine that is completely under my control.
> I have the time and space to work on unexplored frontier problems.
Like what?
Mostly low-level performance optimization and new programming language design.
Obviously it is a ludicrous example, but there are books for this: The Culture novels by Iain. M. Banks.
I'd be quite happy as a citizen of The Culture.
But, on a more serious note, or more realistic, I find it vaguely insulting that you think that so many of us are just so shallow we'll not figure out how to live meaningful lives in an AI dominated future. The world is already meaningless. Some people live in the US, get tech jobs and live lives of almost indescribable luxury compared to someone born in South Sudan. There are beggars on the streets of most African countries, who are deformed of malnutrition and suffering in pain. There are children who will never know true happiness, and die in misery.
It's all fucking senseless and nasty, and as someone who originated from such a poor country it is infuriating to see spoiled developed world complaints sometimes.
We live, suffer, die. Some suffer less, some a lot more. We find out own meaning somehow, in this brief weird experience that we perceive as conscious life.
If an AI can bring a few billion 3rd world people out of misery, I don't think I'm going to shed tears if that means you have to somehow "find meaning" outside of a high paying tech job.
The worry “how will people find meaning with super-abundance?” has also me as tone deaf ennui of the already super-privileged. That we should be so lucky to have this worry.
> Some people live in the US, get tech jobs and live lives of almost indescribable luxury compared to someone born in South Sudan. There are beggars on the streets of most African countries, who are deformed of malnutrition and suffering in pain. There are children who will never know true happiness, and die in misery.
It's a bit baffling to me that folks think that the reason so many live in poverty now is a resource problem, and that an abundance of resources from AI would solve this problem. It is certainly a resource allocation problem, so I don't understand why folks believe that AI or our AI overlords will decide that everyone gets 40 acres and a mule.
I don't think most people care if it does bring a few billion people out of poverty, that's the point. They don't care either way, they just want their "meaning" (well paid US tech jobs) preserved.
> infuriating to see spoiled developed world complaints sometimes
Maybe glance back at your comment history and find you mostly are complaining about very developed world things and investigate why there might be a disconnect between these two aspects of you?
> If an AI can bring a few billion 3rd world people out of misery, I don't think I'm going to shed tears if that means you have to somehow "find meaning" outside of a high paying tech job.
Well, this is begging the question (how awful are people for worrying about the future when the future is going to be so great?), but you're also missing that there's more to humanity than surviving or not, and there always has been. There's no reason to reduce this to a dichotomy.