Potentially unpopular take: memory manufacturers have been operating on the margins of profitability for quite a while now. Their products are essentially an indistinguishable commodity. Memory from Samsung or Micron or another manufacturer may have slight differences in overclockability, but that matters little to folks who just want a stable system. Hopefully the shortage leads large purchasers to engage in long-term contracts with the memory manufacturers which give them the confidence to invest in new fabs and increased capacity. That would be great for everyone. Additionally, we're likely to see Chinese fab'd DRAM now, which they've been attempting since the '70s but never been competitive at. With these margins, any new manufacturer could gain a foothold.
If LLMs' utility continues to scale with size (which seems likely as we begin training embodied AI on a massive influx of robotic sensor data) then it will continue to gobble up memory for the near future. We may need both increased production capacity _and_ a period of more efficient software development techniques as was the case when a new 512kb upgrade cost $1,000.
> Hopefully the shortage leads large purchasers to engage in long-term contracts with the memory manufacturers which give them the confidence to invest in new fabs and increased capacity.
Most DRAM is already purchased through contracts with manufacturers.
Manufacturers don't actually want too many extremely long term contracts because it would limit their ability to respond to market price changes.
Like most commodities, the price you see on places like Newegg follows the "spot price", meaning the price to purchase DRAM for shipment immediately. The big players don't buy their RAM through these channels, they arrange contracts with manufacturers.
The contracts with manufacturers will see higher prices in the future, but they're playing the long game and will try to delay or smooth out purchasing to minimize exposure to this spike.
> Additionally, we're likely to see Chinese fab'd DRAM now, which they've been attempting since the '70s but never been competitive at.
Companies like Samsung and SK Hynix have DRAM fabs in China already. This has been true for decades. You may have Chinese fab'd DRAM in the computer you're using right now.
Are you referring to complete home-grown DRAM designs? That, too, was already in the works.
Except silicon, power, and water (and a tiny amount of plastic/paper for packaging), what else does a fab need that only produces DRAM? If true, then power is far and away the most variable input cost.
> Why would not the same apply advice to oil and gas contracts?
Because oil & gas suppliers only ever sell one product, and memory fabs can dynamically switch product mix in response to supply & demand to optimize profits. The same sand, power and water can make DDR4, HBM or DDR5
Oil and gas suppliers have several products: gas, diesel, jet a, propane, naptha, asphalt etc.
Aren't the proportions is those essentially static?
Depends a lot on the oil field, geology is random
> Except silicon, power, and water
Various chemicals too, https://haz-map.com/Processes/97
> Are you referring to complete home-grown DRAM designs? That, too, was already in the works.
Yes, via cxmt as discussed by Asianometry here: https://www.youtube.com/watch?v=mt-eDtFqKvk
As I mentioned, various groups within China have been working on China-native DRAM since the '70s. What's new are the margins and market demand to allow them to be profitable with DRAM which is still several years behind the competition.
Lots of low end android boxes use cxmt ddr4.
Well, what really prompted this crisis is AI, as well as Samsung shutting down some production (and I have to say I don't think they mind that the pricing has skyrocketed as a result!)
But yes we're going to need more fabs for sure
> Well, what really prompted this crisis is AI,
If the shortage of RAM is because of AI (so servers/data centers I presume?), wouldn't that mean the shortage should be localized to RDIMM rather than the much more common UDIMM that most gaming PCs use? But it seems to me like the pricing is going up more for UDIMM than RDIMM.
UDIMM and RDIMM use the same DRAM chips. And my understanding is that the fabs can switch between DDR5, LPDDR5, and maybe HBM as needed. This means high demand for one type can create a shortage of the others.
> This means high demand for one type can create a shortage of the others.
Wouldn't that mean that a shortage of DRAM chips should cause price difference in all of them? Not sure that'd explain why RDIMM prices aren't raising as sharply as UDIMM. That the fab and assembly lines have transitioned into making other stuff makes sense why'd there be a difference though, as bradfa mentioned in their reply.
It's a valid question if you're not familiar with the RAM market. Sorry you're getting downvoted for it.
The manufacturers make the individual chips, not the modules (DIMMs). (EDIT: Some companies that make chips may also have business units that sell DIMMS, to be pedantic.)
The R in RDIMM means register, aka buffer. It's a separate chip that buffers the signals between the memory chips and the controller.
Even ECC modules use regular memory chips, but with extra chips added for the ECC capacity.
It can be confusing. The key thing to remember is that the price is driven by the price of the chips. The companies that make DIMMs are buying chips in bulk and integrating them on to PCBs.
> Even ECC modules use regular memory chips, but with extra chips added for the ECC capacity.
Quite a few unbuffered designs in the past had a "missing chip". If you ever wondered why a chip was missing on your stick, it's missing ECC. Don't know if it's still the case with DDR5 though.
I have not seen that yet DDR5, I think the signal integrity requirements are too high now to even have unused pads open. Most sticks don’t appear to have many traces at all on the top/bottom sides, just big power/ground planes.
Also with DDR5 each stick is actually 2 channels so you get 2 extra dies.
There's some new half assed ECC type of RAM, not sure the name.
Was reading a series of displeased posts about it. Can't seem to find it now.
On die ECC for DDR5. Which corrects locally but does not signal the host or deal with data between the die and the CPU.
There's 9 bits in an ECC byte.
Because manufacturers transitioned fab and assembly lines from low margin dram to higher margin products like hbm, hence reducing dram supply. But the demand for consumer grade dram hasn’t changed much so prices for it go up.
The chips come from same factory. And difference betweeen those two is... a buffer chip. And extra ram die for ECC
Same chips in both, made in the same fabs. Any relative price difference is like the difference between regular and premium gas/petrol.
wait, are you saying that there's no difference between regular and premium gas?
The “regular” and “premium” label at the pump is misleading. The premium gas isn’t better. It’s just different. Unless your car specifically requires higher octane fuel, there is no benefit to paying for it. https://www.kbb.com/car-advice/gasoline-guide/
You get slightly better mpg on premium, just not enough to justify the cost.
Not unless you’re adjusting timing. Premium gas has lower energy per unit mass and per unit volume than standard gas.
> Not unless you’re adjusting timing.
Which, every modern ECU will do automatically based on output from the knock sensors.
This may surprise you, but gas stations typically only have two grades of fuel stored in tanks. Mid-grade gas is mixed at the pump from the other two.
no, but made in same place with mostly same ingredients, just different ratio to hit higher octane (and in some cases some extra additives).
Also vary a bit between winter a summer, basically in winter they can get away with putting a bit more volatile compounds coz it's colder
It's a sad trend for "the rest of us" and history in general. The economic boom of the 80's thru the 2010s has been a vast democratization of computation - hardware became more powerful and affordable, and algorithms (at least broadly if not individually) became more efficient. We all had supercomputers in our pockets. This AI movement seems to move things in the opposite direction, in that us plebeians have less and less access to RAM, computing power and food and...uh...GPUs to play Cyberpunk; and are dependent on Altermanic aristocracy to dribble compute onto us at their leisure and for a hefty tithe.
I am hoping some of that Clayton Christensen disruption the tech theocracy keep preaching about comes along with some O(N) decrease in transformer/cDNN complexity that disrupts the massive server farms required for this AI boom/bubble thing.
One can see it that way, granted. When I zoom all the way out, all of consumer computation has existed as sort of an addendum or ancillary organ to the big customers: government, large corporations, etc. All our beloved consumer tech started out as absurdly high priced niche stuff for them. We've been sold the overflow capacity and binned parts. And that seems to be a more-or-less natural consequence of large purchasers signing large checks and entering predictable contracts. Individual consumers are very price sensitive and fickle by comparison. From that perspective, anything that increases overall capacity should also increase the supply of binned parts and overflow. Which will eventually benefit consumers. Though the intervening market adjustment period may be painful (as we are seeing). Consumers have also benefited greatly from the shrinking of component sizes, as this has had the effect of increasing production capacity with fixed wafer volume.
> When I zoom all the way out, all of consumer computation has existed as sort of an addendum or ancillary organ to the big customers: government, large corporations, etc.
Perfectly stated. I think comments like the one above come from a mentality that the individual consumer should be the center of the computing universe and big purchasers should be forced to live with the leftovers.
What's really happening is the big companies are doing R&D at incredible rates and we're getting huge benefits by drafting along as consumers. We wouldn't have incredible GPUs in our gaming systems and even cell phones if the primary market for these things was retail entertainment purchases that people make every 5 years.
The iPhone wasn't designed or marketed to large corporations. 3dfx didn't invent the voodoo for B2B sales. IBM didn't branch out from international business machines to the personal computer for business sales. The compact disc wasn't invented for corporate storage.
Computing didn't take off until it shrank from the giant, unreliable beasts of machines owned by a small number of big corporations to the home computers of the 70s.
There's a lot more of us than them.
There's a gold rush market for GPUs and DRAM. It won't last forever, but while it does high volume sales at high margins will dominate supply. GPUs are still inflated from the crypto rush, too.
> The iPhone wasn't designed or marketed to large corporations.
The iPhone isn't exactly a consumer computation device. From that perspective, it does less work at a higher cost.
Advances in video cards and graphics tech were overwhelmingly driven by video games. John Carmack, for instance, was directly involved in these processes and 'back in the day' it wasn't uncommon for games, particularly from him, to be developed to run on tech that did not yet exist, in collaboration with the hardware guys. Your desktop was outdated after a year and obsolete after 2, so it was a very different time than modern times where you example is not only completely accurate, but really understating it - a good computer from 10 years ago can still do 99.9% of what people need, even things like high end gaming are perfectly viable and well dated cards.
yes. a good reason to upgrade was PCIe 4.0 for I/O. GPU and SSD needs caused PCIe 5.0 to follow soon after.
> We wouldn't have incredible GPUs in our gaming systems and even cell phones if the primary market for these things was retail entertainment purchases that people make every 5 years.
Arguably we don't. Most of the improvements these days seem to be on the GPGPU side with very little gains in raster performance this decade.
Gaming drove the development of GPUs which led to the current AI boom. Smartphones drove small process nodes for power efficiency.
SGI and 3Dfx made high-end simulators for aerospace in the beginning. Gaming grew out of that. Even Intel's first GPU (the i740) came from GE Aerospace.
Wolfenstein 3d was released before 3DFx existed, was purely CPU rendered, and generally considered the father of modern 3d shooters. Even without the scientific computing angle, GPUs would have been developed for gaming simply because it was a good idea that clearly had a big market.
Flight simulators just had more cash for more advanced chips, but arcade games like the Sega Model 1 (Virtua Racing) was via Virtua Fighter an inspiration for the Playstation, and before that there was crude games on both PC and Amiga.
Games were always going to go 3d sooner or later, the real pressure of the high volume competitive market got us more and more capable chips until they were capable enough for the kind of computation needed for neural networks faster than a slow moving specialty market could have.
> Flight simulators just had more cash for more advanced chips
Yes. That is my point. The customers willing to pay the high initial R+D costs opened up the potential for wider adoption. This is always the case.
Even the gaming GPUs which have grown in popularity with consumers are derivatives of larger designs intended for research clusters, datacenters, aerospace, and military applications.
No question that chip companies are happy to take consumers money. But I struggle to think of an example of a new technology which was invented and marketed to consumers first.
It's symbiotic, I suppose.
3dfx didnt. They had a subsidiary? spinoff? Quantum3D that reused 3dfx commodity chips to build cards for simulators.
100%. We’ve seen crazy swings in RAM prices before.
A colleague who worked with me about 10 years ago on a VDI project ran some numbers and showed that if a Time Machine were available, we could have brought like 4 loaded MacBook Pros back and replaced a $1M HP 3PAR ssd array :)
> This AI movement seems to move things in the opposite direction, in that us plebeians have less and less access to RAM, computing power and food and...uh...GPUs to play Cyberpunk; and are dependent on Altermanic aristocracy to dribble compute onto us at their leisure and for a hefty tithe.
Compute is cheaper than ever. The ceiling is just higher for what you can buy.
Yes, we have $2000 GPUs now. You don't have to buy it. You probably shouldn't buy it. Most people would be more than fine with the $200-400 models, honestly. Yet the fact that you could buy a $2000 GPU makes some people irrationally angry.
This is like the guy I know who complains that pickup trucks are unfairly priced because a Ford F-150 has an MSRP of $80,000. It doesn't matter how many times you point out that the $80K price tag only applies to the luxury flagship model, he anchors his idea of how much a pickup truck costs to the highest number he can see.
Computing is cheaper than ever. The power level is increasing rapidly, too. The massive AI investments and datacenter advancements are pulling hardware development forward at an incredible rate and we're winning across the board as consumers. You don't have to buy that top of the line GPU nor do you have to max out the RAM on your computer.
Some times I think people with this mentality would be happier if the top of the line GPU models were never released. If nVidia stopped at their mid-range cards and didn't offer anything more, the complaints would go away even though we're not actually better off with fewer options.
The thing about being annoyed about the top of the range prices, for me, it irritates as it feels like it drags the lower models prices upwardsz
But it’s not like the lower priced models are subsidizing the high-end models (probably the opposite; the high-end ones have greater margins).
> Some times I think people with this mentality would be happier if the top of the line GPU models were never released. If nVidia stopped at their mid-range cards and didn't offer anything more, the complaints would go away even though we're not actually better off with fewer options.
If the result was that games were made and optimised for mid-range cards, maybe regular folks actually would be better off.
I would take this argument more seriously if
-the whole reason why the GPU is $2000 is because of said AI bubble sucking up wafers at TSMC or elsewhere, with a soupçon of Jensen's perceived monopoly status...
-for a good part of the year, you could not actually buy said $2000 GPU (I assume you are referring to the 5090) also because of said AI bubble
(granted, while Jensen does not want to sell me his GPU, I would like to point out that Tim Cook has no problem taking my money).
on that point, I can go and buy a Ford F150 tomorrow. Apparently, per the article, I would have problems buying bog standard DDR5 DIMMS to build my computer.
Well put. Since the 1980's consumer has been driving the segment. Even supercomputers were built out of higher end consumer hardware (or playstations in one example).
The move to cloud computing and now AI mean that we're back in the mainframe days.
>We all had supercomputers in our pockets.
You still do. There is no "AI movement" you need to participate in. You can grab a copy of SICP and a banged up ten year old thinkpad and compute away, your brain will thank you. It's like when people complain that culture is unaffordable because the newest Marvel movie tickets cost 50 bucks, go to the library or standardebooks.org, the entire Western canon is free
[dead]
It's not like you need 64GB to have "democratized computation". We used to have 64MB and that was plenty. Unfortunately, software got slower more quickly than hardware got quicker.
> Unfortunately, software got slower more quickly than hardware got quicker.
Hard disagree. A $600 Mac Mini with 16GB of RAM runs everything insanely faster than even my $5000 company-purchased developer laptops from 10 years ago. And yes, even when I run Slack, Visual Studio Code, Spotify, and a gazillion Chrome tabs.
The HN rhetoric about modern computing being slow is getting strangely disconnected from the real world. Cheap computers are super fast like they've never been before, even with modern software.
It is pretty important if you are doing things like 3d animation, video editing, or advanced CAD software. Plus software in general has ballooned its memory requirements and expectations. Even my 11 year old PC had to have a RAM upgrade a few years ago just because software updates suck up so much extra memory, and there is almost nothing consumers can do about it.
> We used to have 64MB and that was plenty.
Bullshit. It was cramped and I wasn't able to do half of what I was wanting to actually do. Maybe it was plenty for your usecases, but such a small amount of memory was weak for my needs in the late 90s and 2000s. 64MB desktops struggled to handle the photo manipulations I wanted to do with scanned images. Trying to do something like edit video on a home PC was near impossible with that limited amount of memory. I was so happy when we managed to get a 512MB machine a few years later, it made a lot of my home multimedia work a lot better.
"memory manufacturers have been operating on the margins of profitability for quite a while now."
The manufacturers are scumbags is more likely answer.
https://en.wikipedia.org/wiki/DRAM_price_fixing_scandal
I wonder how long it will take for China to flood the market with state-of-the-art modules. It's a pretty decent opportunity for them. They probably can hasten the build of new fabs more than many other nations.
But my guess is that this shortage is short-lived (mostly because of the threat above). There's no OPEC for tech.
I don't disagree per-se, but this is the sort of thing which happens when only a few businesses exist in a commodity market with high entry costs. IOW, it's not great, but it is predictable. See: Oil.
Looking forward to the "Organization of Processor-Etching Corporations".
It's usually not only illegal, but also a crime.
Anyway, that's the kind of market that governments always need to act upon and either supply directly or regulate intensively.
It's not just predictable, it's illegal. Of course, if you have an executive that actually cares about enforcing the law.
You mean “capitalists”.
Maximizing profit is the only sane way to play a rigged game
https://videocardz.com/newz/chinese-cxmt-to-produce-ddr5-800...
Manufacturing is based on anticipating demand.
Unforseen things like the pandemic hurt profits.
Letting things go this unmanaged with a 3 year run way for AI demand seems a little hard to understand. In this case, not anticipating demand seems to creates more profit.
I find it hard to believe the pandemic hurt the profits of computing hardware, demand went up from it not down.
I'm not sure if profits were hurt, but the manufacturing did slow and stop and take some time to get going again.
[dead]
I guess we'll just have to stop making computer memory if it ceases to be profitable. The market is so efficient.