Well the railroads aren't the seat of power that they used to be---the progress ebbs and flows in mysterious ways. Even their arguable replacement, air travel, is not the paradigm of economic dominance.

Everybody is now excited about the top end: more parameters, larger context window. Justly so, but I think that it's more important in the long term what will happen at the low end, both in the software and hardware direction. Software obviously has started (c.f. DeepSeek). As for hardware, Google TPU ASIC is six years old, and has presumably 4096 ALUs and uses few watts of power to perform 4 TOPs. Several years later, I would hope that it's reachable goal for those 4 watts to do as many TOPs as say H20 (400 or so).

Once the models stabilize, I could see a low power inference peripheral design with huge TPU matrix with an embedded model parameter Flash EPROM feeding it. This could go into anything from white goods to computers. Imagine a thermometer that you speak symptoms to and ask for diagnosis; is it a good idea? I don't know... but is it possible? I could see that it is.