Calculate what performance you get per $ spent, and Apple again looks the most expensive out of probably anything else you could buy.
Calculate what performance you get per $ spent, and Apple again looks the most expensive out of probably anything else you could buy.
Are you sure about that? For the price of the top configuration here you can buy a 5-stack of 256GB Mac Studio M5 Ultras.
This dual NVIDIA RTX PRO 6000 setup has 192GB of VRAM at 1.8 TB/s, versus the 5-stack of Macs that collectively have 1.25 TB at 1.2 TB/second... I'm not convinced that equation comes out in Nvidia's favour.
Right, performance is more than just the aggregated memory speed of the hardware you have. I'm fairly sure, at least last time I looked, maybe Apple launched something new in the last 2-3 months that has completely changed the picture?
> performance is more than just the aggregated memory speed of the hardware you have
In other fields, sure, but for big LLMs it's a very significant part of the performance picture. That stack of Macs also is going to be able to natively run models 4-5x larger than the dual Blackwells can hold in memory - any model over about 128GB of weights isn't going to fit on the GPUs, and is going to be heavily performance constrained by moving data across the PCIE bus.
> maybe Apple launched something new in the last 2-3 months that has completely changed the picture
Indeed. The M5 Ultra (currently up for pre-order) has 50% higher memory bandwidth than its predecessor, and a claimed 4x improvement in prompt prefill.
> any model over about 128GB of weights isn't going to fit on the GPUs
Not sure if you misunderstand what GPUs we're talking about, one RTX Pro 6000 has 96GB of VRAM.
> Indeed. The M5 Ultra (currently up for pre-order) has 50% higher memory bandwidth than its predecessor, and a claimed 4x improvement in prompt prefill.
Exciting! Eagerly awaiting the benchmarks and comparisons then. Lets hope "4x improvement" had a good enough baseline so 4x actually ends up useful in practice compared to the current hardware they offer.
> 50% higher memory bandwidth
Seems this lands on ~1.2 TB/s if what Apple claims is correct. For reference, RTX Pro 6000 does 1.8 TB/s, so seems Apple is indeed getting closer incrementally.