>Google is already doing this and has an experimental project on top of already having TPUs and cramming their quantized flash onto individual TPUs for inference.

Google is no longer a serious player in frontier AI. I doubt they will ever hit a SOTA model again.