unfortunately running anything locally for serious personal use makes no financial sense at all right now.
4x rtx 6000 pro is probably the minimum you need to have something reasonable for coding work.
unfortunately running anything locally for serious personal use makes no financial sense at all right now.
4x rtx 6000 pro is probably the minimum you need to have something reasonable for coding work.
That's the setup you want for serious work yes, so probably $60kish all-in(?). Which is a big chunk of money for an individual, but potentially quite reasonable for a company. Being able to get effectively _frontier-level local performance_ for that money was completely unthinkable so far. Correct me if I'm wrong, but I think Deepseek R1 hardware requirements were far costlier on release, and it had a much bigger gap to market lead than Kimi K2.5. If this trend continues the big 3 are absolutely finished when it comes to enterprise and they'll only have consumer left. Altman and Amodei will be praying to the gods that China doesn't keep this rate of performance/$ improvement up while also releasing all as open weights.
I'm not so sure on that... even if one $60k machine can handle the load of 5 developers at a time, you're still looking at 5 years of service to recoup $200/mo/dev and that doesn't even consider other improvements to hardware or the models service providers offer over that same period of time.
I'd probably rather save the capex, and use the rented service until something much more compelling comes along.
At this point in time, 100% agreed. But what matters is the trend line. Two years ago nothing came close, if you wanted frontier-level "private" hosting you'd need an enterprise contract with OpenAI for many $millions. Then R1 came, it was incredibly expensive and still quite off. Now it's $60k and basically frontier.
Of course... it's definitely interesting. I'm also thinking that there are times where you insource vs outsource to a SaaS that's going to do the job for you and you have one less thing to really worry about. Comparing cost to begin with was just a point I was curious about, so I ran the numbers. I can totally see a point where you have that power in a local developer workstation (power requirements notwithstanding), good luck getting that much power to an outlet in your home office. Let alone other issues.
Right now, I think we've probably got 3-5 years of manufacturing woes to work through and another 3-5 years beyond that to get power infrastructure where it needs to be to support it... and even then, I don't think all the resources we can reasonably throw at a combination of mostly nuclear and solar will get there as quickly as it's needed.
That also doesn't consider the bubble itself, or the level of poor to mediocre results altogether even at the frontier level. I mean for certain tasks, it's very close to human efforts in a really diminished timeframe, for others it isn't... and even then, people/review/qa/qc will become the bottleneck for most things in practice.
I've managed to get weeks of work done in a day with AI, but then still have to follow-up for a couple days of iteration on following features... still valuable, but it's mixed. I'm more bullish today than even a few months ago all the same.