The only reason anyone uses a TPU is because they couldn't get the best GPUs.

Okay? I'm not sure where you're going with this.

Google's TPUs have obvious advantages for inference and are competitive for training.