Yes you do, you get PyTorch or whatever else, built on top of those vendor-specific libraries.
That is the thing with Mojo, when it arrives as 1.0, the LLM progress and the investment that is being done in GPU JITs for Python, make it largely irrelevant for large scale adoption.
Sure some customers might stay around, and keep Modular going, the gold question is how many.
Pytorch is built on an amalgamation of these different frameworks, not on one of them used to target different vendors.