Is "faster" really what we are talking about right now? It could be a lot faster to take a helicopter to work everyday too, versus riding a bike.
Also, why are people moving mountains to make huge, power obliterating datacenters if actually "its fine, its not that much"?
Speed highly correlates with power efficiency. I believe my hardware maxes out somewhere around 150W. 15 seconds of that isn't much at all.
> Also, why are people moving mountains to make huge, power obliterating datacenters if actually "its fine, its not that much"?
I presume that's mostly training, not inference. But in general anything that serves millions of requests in a small footprint is going to look pretty big.
There's a billion users. Why do we make massive cities and factories and fields if humans only need 2000 calories a day
idk, why?
Because there's a lot of people
> It could be a lot faster to take a helicopter to work everyday too, versus riding a bike.
Great analogy.
It's not a good analogy at all, because of what they said about mundane hardware. They're specifically not talking about any kind of ridiculous wattage situation, they're talking about single GPUs that need fewer watts than a human in an office to make text faster than a human, or that need 2-10x the watts to make video a thousand times faster.
It's a Framework Desktop motherboard. I believe the CPU on that maxes out somewhere around 150W.