Cerebras already runs large models like Kimi 2.6 or GLM at like 30x speed. 100 times is next year, not six years.
You can actually test it out on their website, just imagine 3 x faster and maybe 15% smarter.
Cerebras already runs large models like Kimi 2.6 or GLM at like 30x speed. 100 times is next year, not six years.
You can actually test it out on their website, just imagine 3 x faster and maybe 15% smarter.
Cerebras is literally the entire wafer, so it can't get bigger. So where is the jump from 30x to 100x coming from? Node improvements only yield like 10-20% gains these days...
They have a next generation, I don't really know if it will be 3 x or what but I heard it was significantly better.
Also there are other people innovating in hardware.
Could we not just make bigger wafers, if the technology called for it?
The investment in bigger machines at the fab might set you back billions. I don't know about the lithography technology either, how easy you can scale it to larger wafers?
There's so much you'd need to scale, it's even more expensive than the humble few billions you spend on a new fab. And, beyond R&D, the scale itself also makes it significantly more expensive. This is an energetic dead end, you would need a country to fully commit to making this one thing.
Lithography is AFAIK the most expensive part. If you wanna scale to larger wafers I'd guess you're talking hundreds of billions and at least a decade. And even then, what will your yield be? Surface area grows to the square of the diameter of a circle, and so will errors. The larger your wafer goes the lower your yield will go, especially if you're going to use the whole wafer in one final product.
You also need to worry about yields, Apple, AMD etc can sell ”bad” chips as lower core versions, if you’re depending on whole wafer you have little room for error.
we seem to be in a phase of spending trillions on the computer, so while it isn’t likely, it isn’t impossible
This.