Typical desktop GPU ram does not support being write-back cached by the CPU. With PCIe resizable BAR, you could map the area into ram, so you could technically fit 32GB to memory, but it would have to be uncached (or write-combine cached), which would make it really, really slow.
There are a bunch of datacenter GPUs that support full cache coherency, but if you used them like that the VRAM would be very high latency from the CPU. So it would only be really slow.
I assume on Linux you could use something like daxctl to tell the kernel to treat the vRAM as normal RAM, but I think this would be Intel/AMD only.
I don't think it would help. It's not just a software issue that can be fixed in the kernel, the hardware fundamentally isn't part of the cache coherency system of the CPU.
This is correct, look at IBM's CAPI for an example of the needed hardware
[flagged]