It's worth remembering that this is all happening because of video games !
It is highly unlikely that the hardware which makes LLMs possible would have been developed otherwise.
Isn't that amazing ?
Just like internet grew because of p*rn, AI grew because of video games. Of course, that's just a funny angle.
The way I see it, AI isn't accidental. Its inception has been in the first chips, the Internet, Open Source, Github, ... AI is not just the neural networks - it's also the data used to train it, the OSes, APIs, the Cloud computing, the data centers, the scalable architectures.. everything we've been working on over the last decades was inevitably leading us to this. And even before the chips, it was the maths, the physics ..
Singularity it seems, is inevitable and it was inevitable for longer than we can remember.
I think it's a bit hard to say that this is definitively true: people have always been interested in running linear algebra on computers. In the absence of NVIDIA some other company would likely have found a different industry and sold linear algebra processing hardware to them!
Almost certainly not at the scale of the consumer gaming industry, however!
Google is making millions of TPUs per year. Nvidia ships more gaming GPUs, but it's not like multiple orders of magnitude off.
I'm willing to bet TPUs wouldn't be nearly as successful or sophisticated without the decades of GPU design and manufacturing that came before them.
Current manufacturing numbers are a small part of the story of the overall lineage.
You are missing his point. They very likely don't start building TPUs if there were no GPUs.
Google DeepMind can trace part of it's evolution back to a playtester for the video game Syndicate who saw an opportunity to improve the AI of game NPCs.
what a load of utter tripe