I think it's a bit hard to say that this is definitively true: people have always been interested in running linear algebra on computers. In the absence of NVIDIA some other company would likely have found a different industry and sold linear algebra processing hardware to them!
Almost certainly not at the scale of the consumer gaming industry, however!
Google is making millions of TPUs per year. Nvidia ships more gaming GPUs, but it's not like multiple orders of magnitude off.
I'm willing to bet TPUs wouldn't be nearly as successful or sophisticated without the decades of GPU design and manufacturing that came before them.
Current manufacturing numbers are a small part of the story of the overall lineage.
You are missing his point. They very likely don't start building TPUs if there were no GPUs.