I think it's a weird take because it implies that the 6 billion people he's talking about actually have some interest in knowing about the capabilities of LLMs to solve frontier math? This is simply not something they care about or can evaluate. There are maybe several thousand people in the world who have some (abstract and barely-monetizable) use for this information, plus probably another 100,000 who don't understand any of the math, but like to cheer on.
We have already reached "peak LLM" in terms of what normal people realistically need it to know or reason about. In fact, I'd say we reached that point about 1.5-2 years ago. There are two other barriers that remain unsolved:
1. They're less dependable than humans and can't be meaningfully punished or forced to make up for mistakes, so you can't really replace humans with them, not without having a human babysit.
2. Most people don't really have a special need for an LLM in their life. They may like that it answers questions or helps you polish a resume or, I guess the labs' favorite, helps you make restaurant reservations. But this sure isn't worth $200/mo for most people. Probably not even worth $5/mo.
It'd be kinda funny if we create superhuman AGI and then no one has any real use for it, perhaps except for military murder-bots. There's always market for that.
I know I’m bordering on repetitive and strongly negative, but the combination of the two barriers you mentioned has all but halted all of my interaction with agents.
Even ones that cannot be easily dismissed (like the agent at the top of Google search) are often subtly wrong in a way that requires significantly more effort on my end to parse the loquacious output to determine where the inconsistencies are.
Look, can they be useful to generate the html or jscript for humanity’s 9 billionth iteration of a web form? Sure. But man, I wish sanity had reigned and people had applied ML models in general to more substantive and beneficial projects.
(Before anyone chimes in, I’m familiar with implementations of ML applied to esoteric domains; but by their very nature these don’t get all the news cycles, or hiring, or any of the other absolute insanity that the domain seems to contain)
This is by the weirdest possible opinion on llms as a whole. Not the AI booster club, not the AI doomer, not the undecided, but somehow "I can't think of anything to use it for." It is like watching the beginning of the internet in 90s and saying there is nothing to browse. The whole concept is just weird to me. At least the first 3, I can conceptually understand. It is admittedly hard for me to understand someone saying 'i tried this and best it can do is summarize emails'; I am not saying it is not true ( my boss said that exact thing ), but it is hard for me to reconcile with my view of the world.
To be fair, this was a commonly expressed opinion when the internet was in its infancy.
Paul Krugman (1998)
• "The growth of the Internet will slow drastically, as the flaw in 'Metcalfe's law'—which states that the number of potential connections in a network is proportional to the square of the number of participants—becomes apparent: most people have nothing to say to each other! By 2005 or so, it will become clear that the Internet's impact on the economy has been no greater than the fax machine's."
Clifford Stoll (1995)
• "We'll soon buy books and newspapers straight over the Internet. Uh, sure."
• "The truth is no online database will replace your daily newspaper, no CD-ROM can take the place of a competent teacher and no computer network will change the way government works."
• "Lacking editors, reviewers or critics, the Internet has become a wasteland of unfiltered data."
There's a massive tendency amongst even intelligent people to completely dismiss the potential of new technology based on that technology's current observed capability.
Also seems weird to me. Claude cowork could just do my monthly tax receipt collection and compare with my account better than the bookkeeper I pay for it. That is a very cross sectional problem
Most people rarely have these kind of administrative tasks in the first place, and when they do they often run into GP's barrier 1. I can't recommend that my extended family have Claude do their taxes, even though I work with it on my own taxes, because they might ask for something that produces bad results and can't evaluate the output to double check. ("Hey Claude, I want a higher refund, can you tell me what to put on the forms so I get at least $3000?")