Or Google TPUs.
TPUs don't have enough memory either, but they have really great interconnects, so they can build a nice high density cluster.
Compare the photos of a Cerebras deployment to a TPU deployment.
https://www.nextplatform.com/wp-content/uploads/2023/07/cere...
https://assets.bwbx.io/images/users/iqjWHBFdfxIU/iOLs2FEQxQv...
The difference is striking.
Oh wow the cabling in the first link is really sloppy!
TPUs don't have enough memory either, but they have really great interconnects, so they can build a nice high density cluster.
Compare the photos of a Cerebras deployment to a TPU deployment.
https://www.nextplatform.com/wp-content/uploads/2023/07/cere...
https://assets.bwbx.io/images/users/iqjWHBFdfxIU/iOLs2FEQxQv...
The difference is striking.
Oh wow the cabling in the first link is really sloppy!