The A100 came out 5.5 years ago and is still the staple for many AI/ML workloads. Even AI hardware just doesn’t depreciate that quickly.
The A100 came out 5.5 years ago and is still the staple for many AI/ML workloads. Even AI hardware just doesn’t depreciate that quickly.
Users are waiting for Blackwell. Then Rubin. CRWV depreciates GPUs over 6 years. Rails last a lot longer.
Don't they degrade physically from being run at full blast 24/7 for so many years?
Not nearly as badly if you keep them cool.
Unless you care about FLOP/Watt, which big players definitely do.
No more than 5-10% of the hourly cost of an A100 will be in actual power. It’s not a big factor.
This. There’s even a market for them being built (DRW).