I am also considering to buy 3-4x RTX 6000 Pro 96GB plus some Ryzen workstation with a grant.
Is this the best general-purpose choice as of 2026 with $50k for training, fine-tuning and running large open models?
I am also considering to buy 3-4x RTX 6000 Pro 96GB plus some Ryzen workstation with a grant.
Is this the best general-purpose choice as of 2026 with $50k for training, fine-tuning and running large open models?
Foe multi GPU make sure you have enough PCIE lanes! That rules out consumer grade sockets like AM5, you would need Threadripper or EPYC.
Why are these sockets "ruled out"? Pipeline/layer parallelism doesn't need high bandwidth between nodes, and tensor parallelism has middling performance unless you have very fast networking and very slow compute. It all depends on what you're doing.
You are correct that bandwidth requirements depends a lot on the exact workload. And that in specific cases, it might be doable to have AM5 for multiple RTX6000Pro. The parent mentioned workloads that are general, and broader than inference-only. In that case I would consider spending a bit extra on the motherboard to ensure that PCIE bandwidth is not an issue.
[flagged]