I can't imagine spending $48K on a home GPU server, but I did just splurge and buy a PC with an RTX 5090, specifically to hold the largest models you can fit in 32GB. It's a top of the line PC with water cooled high end CPUs, 64GB RAM, RTX 5090 for $5K. To me the jury is still out whether this was a worthwhile investment, but I do expect to use this machine for a decade. I don't run it at 100% power (it's mostly idle, except for times when I'm training or doing batch inference). It has the nice property of being blackwell generation, similar to the machines we use at work.
It just scares me to own a box that is $48K in my house, especially if it breaks, or gets stolen.
Last fall, seeing the writing on the wall, I pieced together an "AI" rig, 96GB ram, 2x RTX 3090, 9950X - not exactly top of the line, but it came in around CDN$3000 all in all, with most parts second hand. I don't think I could build that for CDN$10000 today.
I've been using it pretty steadily for a variety of personal projects, and the only improvement (aside from the obvious "more VRAM") I feel pressed to make is a portable AC unit / some kind of a focused cooling solution. The rig raises the ambient temperature in the office by 4C at least.
Now with the murmurs of even the large players reconsidering their AI spend, and usage-based pricing shifts, having a self-contained, owned, and independently administered compute resource is looking better and better.
Well a lot of people have that in their garage, even "worst" it's on wheels so even easier to steal.
I'm not saying it's worth it just that it's not such a crazy amount in comparison.
>> To me the jury is still out whether this was a worthwhile investment, but I do expect to use this machine for a decade.
The high cost and power consumption are both signs of the death of Moore's law, so you are probably correct that this system will be near state of the art for some time.
I was looking at Ultras for sale, and had same worry, so didn't end up getting one. I have some peace of mind comfort about applecare and technical repair, but i couldn't find insurance that would cover theft (or rather, i did, but it was too expensive)
Yes! It scared me too. I tried to insure it under my renter's insurance policy, but they not surprisingly refused. I had to get business insurance to cover it
You showed this setup to a business insurance underwriter and they gave you a policy? Can I ask how much the premium is? Or is this just theft insurance?
Not even a single mention of gaming.
No wonder gamers hate AI bros.
I have a second computer with an RTX 4090 for gaming (running Windows). I also used the new RTX 5090 running Linux to evaluate whether Proton/Wine allow me to run Windows games on linux (yes, it works, but the compatibility and frame rate issues make me stick to native Windows for now).
If you want a GPU that has comparable performance on Linux to Windows- you want AMD. NVIDIA drivers are notoriously bad. Many of my games run better on Linux with the open source AMd drivers. (CachyOS rolling rolling rolling).
Sadly if you want a GPU with good AI performance you gotta go with NVIDIA. It might sound crazy but as a 7900XTX owner.. My 12GB 3060 on my linux server outperformed the 7900XTX by 40%. The 3060 only has half the vram of the AMD card. Proprietary drivers under Arch Linux.
On top of the significantly worse software on AMD's side (literally didn't work on windows in particular - so the "performs as good on both systems" is a nonstarter, some GGUF library dependency just doesn't work/exist under AMD on windows). Had me running the AMD card on windows under WSL (not a problem with nvidia though, that ran just fine on windows-side directly).
Aaaand also the other AMD bugs, such as the pink squares display corruption that has been an active issue for my GPU in particular (7900XTX) for over a year, maybe approaching two at this point, with no fix in sight from the AMD team (barely and ack at all - not on a single patch notes, just a bunch of reddit discussion). Really regret spending so much on an AMD gpu.
I have no interest in moving to AMD for video cards right now- the network effect of NVIDIA is just too high, and their peak performance is insane. I also haven't noticed any major issues with nvidia drivers, unless you mean specifically running Windows games on Linux machines with nvidia cards, where I have zero experience.
Network effect for graphics cards? Literally what? Your friends don’t care what GPU you run my guy and there is not much benefit of having brand loyalty to a company like Nvidia that gives absolutely zero fucks about people that aren’t their enterprise customers buying GPUs by the thousands. If there’s any “network effect” for gaming GPUs on Linux it’s in favor of AMD because of the immense amount of work Valve has been putting in to make it work well for their steam* hardware.
Nvidia’s drivers are trash for gaming on Linux and the majority of your “compatibility and framerate issues” are because you’re using a sub-par product for the job.
> Nvidia that gives absolutely zero fucks about people that aren’t their enterprise customers buying GPUs by the thousands.
for the context, nvidia gives zero fucks about people who are buying their GPUs by thousands as well.
I am also an enterprise customer that buys GPUs by the thousands, you can see a bit more about my work here: https://www.gene.com/media/press-releases/15010/2023-11-21/g... and https://blogs.nvidia.com/blog/roche-ai-factories-omniverse/ and have worked with nvidia since the mid-2000s on high performance computing for scientific research (in addition to having nvidia graphics cards since the Riva TNT, running both Windows and Linux). So having a blackwell graphics card I can evalute with linux and windows, both for ML training, inference, and gaming, is a huge network effect.
We’re talking about your gaming PC here. Nobody is forcing you to ONLY buy Nvidia graphics for your personal gaming rig when you ALSO have a purpose built AI rig. Nvidia just removed “gaming” as a segment from their financial reports. They give zero fucks. This absurd blind loyalty serves no purpose.
https://www.protondb.com/
I wonder what's going wrong there? Personally I found compatibility and performance on Linux to be extremely good. And just keeps getting better. And that's not even just me, that's all kinds of benchmarks out there. Sorry to hear that. : ' (
No idea. I agree that in principle I should have close to the same performance on Linux. I just didn't want to spend a bunch of time customizing configs and updating software so I could reach parity with Windows when I had two computers.
I have the same rig as you minus watercooling, and I assume you have AMD Ryzen 7 9800X3D? Anyway, it's my only PC now, I game, dev, run local models, edit photos, edit videos, all in Manjaro. I get ~70FPS in Cyberpunk at 4k, every setting at "Insane" or whatever goofy thing they call it, Ray tracing on path tracing off, with no framegen but with DLSS set to quality. Without DLSS I get around 40fps. Seems equivalent to what I see online with people with a similar build on Windows.
I run hyprland, seems to be the only wayland based keyboard-forward WM that has good nvidia support (and, allegedly, supports HDR, though I haven't got this working). I heard gnome was pretty good otherwise. I was running i3 before and it also worked fine, however once I got into wanting to get streaming working, there wasn't good compatibility between i3/xorg and tools like sunshine. I believe steam streaming worked fine on it though iirc.
The only thing I miss from windows: easy streaming with sunshine/moonlight. Steam streaming works (usually heh) but it took me a couple days of fiddling to get a stream to work at all through sunshine, and it is choppy. But for local gaming, I don't miss windows at all, I'm so glad to finally have all my drives converted from NTFS to ext4.
No, it's an Alienware R51 with Intel Core Ultra 9 285K 3.2GHz Processor; NVIDIA GeForce RTX 5090 32GB GDDR7; 64GB DDR5-6400 RAM; 4TB Solid State Drive; Microsoft Windows 11 Home; 2.5GbE LAN; 2x2 Intel Killer WiFi 7 BE1750+Bluetooth 5.4; Liquid Cooler
I don't see it on the Dell site anymore, only more expensive, lesser configurations (good timing on my part?).
Yeah, I really want to put in the time to try out various games, but realistically, the whole point of getting a second computer and installing Linux was to be able to train and serve models, and switching between serving a model (that people in my house want to use at random times) and gaming didn't seem like a great choice. If I did get good results, I'd seriously consider wiping Windows 11 from my older machine (an older Alienware with a 4090), but to be honest, I'm perfectly comfortable on Windows desktop.
Having built an almost identical rig earlier this year can promise at least one similarly-spec'd machine gets equal use between AI and gaming (Both on Linux). Stupid-excited for the Steam Frame to finally come out.
or crypto... what's old is new again.
I would probably hate someone if they were buying the same hardware as me but doing something actually useful with it. Any game worth playing doesn't require high specs anyway. There is such a large catalog of old games.
I specifically got the previous model so I could play AAA games with all the settings set to Ultra, at 4K. Cyberpunk 2077 struggled even with my 4090, so I had to disable ray tracing and enable DLSS. Since I've run out of new AAA games I've been playing older ones and it's crazy how fast they are.
I don't think you can dismiss gaming as not "actually useful".
> No wonder gamers hate AI bros.
Personally, playing with AI models is way more fun than getting sucked into a game loop. Game loops feel like busy work hooked to an engineered dopamine drip. AI models are new frontiers and are exciting to build with, modify, lobotomize, and hack around with.
And some of us are doing AI stuff all freaking day at day job and just want to play some Tekken when we get home for 30 minutes after the kid is in bed. But now Playstations are 1000$ and Ram and GPU prices are astronomical.
Not everyone is hustling 24/7 like some kind of lunatic.
[dead]
I remember playing Quake III which had user-programmable bots and thinking "wow, this is a really hard computer vision and reasoning problem". And then realizing "huh, that's a major research area, I should work on that". Later I learned that the bots were fairly simple and worked on far simpler world representations (nav meshes).
It looks like DM took a crack at it: https://deepmind.google/blog/capture-the-flag-the-emergence-...