> I personally dropped $20k on a high end desktop - 768G of RAM, 96 cores, 96 GB Blackwell GPU - last October, before RAM prices spiked […]
768GB of RAM is insane…
Meanwhile, I’ve been going back and forth for over a year about spending $10k on a MacBook Pro with 128GB. I can’t shake the feeling I’d never actually use that much, and that, long term, cloud compute is going to matter more than sinking money into a single, non-upgradable machine anyway.
Your battery is going to suffer because of the extra ram as well.
I don't know your workloads, but for me personally 64 GB is the ceiling buffer on RAM - I can run entire k8s cluster locally with that and the M5 Pro with top cores is same CPU as M5 Max. I don't need the GPU - the local AI story and OSS models are just a toy for my use-cases and I'm always going to shell out for the API/frontier capabilities. I'm even thinking of 48 config because they already have those on 8% discounts/shipped by Amazon and I never hit that even on my workstation with 64 GB.
> Your battery is going to suffer because of the extra ram as well.
No, it won't. The power drain of merely refreshing DRAM is negligible, it's no higher than the drain you'd see in S3 standby over the same time period.
Given the DRAM refresh is part of S3 standby, I'm afraid this is circular reasoning.
I suspect this is one of those "it depends" situations; does the 128gb vs 64gb sku have more chips or denser chips? If "more chips" probably it'll draw a tiny bit more power than the smaller version. If the "denser" chips, it may be "more power draw" but such a tiny difference that it's immaterial.
Similarly, having more cache may mean less SSD activity, which may mean less energy draw overall.
If I had a chip to put on the roulette table of this "what if" I'd put it on the "it won't make a difference in the real world in any meaningful way" square.
I thought my Z620 with 128GB of RAM was excessive! Actually, HP says they support up to 192GB of RAM, but for whatever reason the machine won't POST with more than 128GB (4Rx4) in it. Flawed motherboard?
Look at the way age gating is going in a global coordinated push. Can control of compute be far behind?
It wasn't my primary motivator but it hasn't made me regret my decision.
I hummed and hawed on it for a good few months myself.
Just look at ITAR and the various attempts at legislating 3D printing and CNC machining of firearms parts to see one justification point of that.
> Can control of compute be far behind?
How is this going to work? You need uncontrolled compute for developing software. Any country locking up that ability too much will lose to those who don't.
> How is this going to work? You need uncontrolled compute for developing software.
I've read about companies where all software developers have to RDP to the company's servers to develop software, either to save on costs (sharing a few powerful servers with plenty of RAM and CPU between several developers) or to protect against leaks (since the code and assets never leave the company's Citrix servers).
Even for tiny crews doing nothing of fatal significance, this is unironically superior to "throw it on GitHub"
>You need uncontrolled compute for developing software
Oh you sweet summer child :(
You think our best and brightest aren't already working on that problem?
In fact they've fucking aced it, as has been widely celebrated on this website for years at this point.
All that remains is getting the rest of the world to buy in, hahahaha.
But I laugh unfairly and bitterly; getting people to buy in is in fact easiest.
Just put 'em in the pincer of attention/surveillance economy (make desire mandatory again!).
And then offer their ravaged intellectual and emotional lives the barest semblance of meaning, of progress, of the self-evident truth of reason.
And magic happens.
---
To digress. What you said is not unlike "you need uncontrolled thought for (writing books/recording music/shooting movies/etc)".
That's a sweet sentiment, innit?
Except it's being disproved daily by several global slop-publishing industries that exist since before personal computing.
Making a blockbuster movie, recording a pop hit, or publishing the kind of book you can buy at an airport, all employ millions of people; including many who seem to do nothing particularly comprehensible besides knowing people who know people... It reminds me of the Chinese Brain experiment a great deal.
Incidentally, those industries taught you most of what you know about "how to human"; their products were also a staple in the lives of your parents; and your grandparents... if you're the average bougieprole, anyway.
---
Anyway, what do you think the purpose of LLMs even is?
What's the supposed endgame of this entire coordinated push to stop instructing the computer (with all the "superhuman" exactitude this requires); and instead begin to "build" software by asking nicely?
Btw, no matter how hard we ignore some things, what's happening does not pertain only to software; also affected are prose, sound, video, basically all electronic media... permit yourself your one unfounded generalization for the day, and tell me - do you begin to get where this is going?
Not "compute" (the industrial resource) but computing (the individual activity) is politically sensitive: programming is a hands-on course in epistemics; and epistemics, in turn, teaches fearless disobedience.
There's a lot of money riding on fearless disobedience remaining a niche hobby. And if there's more money riding on anything else in the world right now, I'd like an accredited source to tell me what the hell that would be.
Think for two fucking seconds and once you're done screaming come join the resistance.
> 768GB of RAM is insane.
Before this price spike, it used to be you could get a second-hand rack server with 1TB of DDR4 for about $1000-2000. People were massively underestimating the performance of reasonably priced server hardware.
You can still get that, of course, but it costs a lot more. The recycling company I know is now taking the RAM out of every server and selling it separately.
Apple hardware is incredibly overpriced.
My home server has 512GB RAM, 48 cores, my 4 desktops are 16 cores 128GB, 4060GPU each. Server is second hand and I paid around $2500 for it. Just below $3000 price for desktops when I built them. All prices are in Canadian Pesos
Canadian Pesos?
Jokes because the Canadian dollar’s value isn’t very high right now.
See a $1100 GPU on eBay, but it’s in the US? Actually a $1900 GPU.
A colleague were just talking about how well he timed the purchase of his $700 24GB 3090.
Please, it's actually Cambodian Dollhairs or Canuckistan Pesos.
It is sarcasm. Our dollar which used to be on par with US is no more.
> spending $10k on a MacBook Pro with 128GB.
As someone who just bought a completely maxed out 14" Macbook Pro with an M5 Max and 128GB of RAM and 8TB SSD, it was not $10k, it was only a bit over $7k. Where is this extra $3k going?
Tangential, I bought a nearly identically-spec'd (didn't spring for the 8 TB SSD - in retrospect, had I kept it, I would've been OK with the 4 TB) model, and returned it yesterday due to thermal throttling. I have an M4 Pro w/ 48 GB RAM, and since the M5 Max was touted as being quite a bit faster for various local LLM usages, I decided I'd try it.
Turns out the heatsink in the 14" isn't nearly enough to handle the Max with all cores pegged. I'd get about 30 seconds of full power before frequency would drop like a rock.
I haven't really had a problem with thermal throttling, but my highest compute activity is inferencing. The main performance fall-off I've observed is that the cache/context size to token output rate curve is way more aggressive than I expected given the memory bandwidth compared to GPU-based inferencing I've done on PC. But other than spinning up the fans during prompt processing, I'm able to stay peak CPU usage without clock speed reducing. Generally though this only maintains peak compute utilization for around 2-3 minutes.
I'm wondering if there was something wrong with your particular unit?
It could be a different country?
With the way legislation is going these days, self hosting is becoming ever more important. RAM for zfs + containers on k3s doesn't end up being that crazy if you assuming you need to do everything on your own. (at home I've got 1 1tb ram machine, 1 512gb, 3x 128gb all in a k3s cluster with some various gpus about about a half pb of storage before ~ last sept this wasn't _that_ expensive to do)