yea, "virtualizing" an NVIDIA GPU is not truly feasible right now from what I see.

I do know that AMD has done more in this space. When I worked at Google there were (other) people on the Stadia team doing this. And some open source bits out there that I've seen, including from AMD.

Unfortunately, my real world work... and most real inference etc work others are doing... is on CUDA/NVIDIA.