Something people forget is that a training cluster with tens of thousands of GPUs is a general purpose supercomputer also! They can be used for all sorts of numerical modelling codes, not just AI. Protein folding, topology optimisation, route planning, satellite image processing, etc…
We bought a lot of shovels. Even if we don’t find more gold, we can dig holes for industry elsewhere.
I think there is an LLM bubble for sure, but I’m very bullish on the ease with which one can generate new specialized models for various tasks that are not LLM.
For example, there’s a ton of room for developing all kinds of low latency, highly reliable, embedded classifiers in a number of domains.
It’s not as gee-whiz/sci-fi as an LLM demo, but I think potentially much bigger impact over time.
Agreed! One thing I noticed is that the LLM craze seems to have triggered some other developments in only vaguely related fields.
My favourite example is the astonishing pace with which reverse-rendering technology has progressed. It started with a paper by NVIDIA showing projections of 2D photos being "fitted" into a 3D volume of differentiable hashtables, and then the whole thing exploded when Guassian Splats were invented. I full expect this niche all by itself to generate a huge variety of practical applications. Computer games and movie special effects, obviously, but also AR/VR, industrial uses, mapping, drone navigation, etc...
Spot on.