Quite courageous to go with AMD. I am very curious to see Chapter 2 and how he solves the software stability that used to plague AMD AI applications.
Quite courageous to go with AMD. I am very curious to see Chapter 2 and how he solves the software stability that used to plague AMD AI applications.
"used to" in your sentence already answers your curiosity.
ROCm runs most things just fine without any ceremony or difficulty. I have a couple of the same cards as covered in the article, bought before they got expensive (I'd recommend current Radeon AI Pro 9700 cards over the V620 now, though, as they have increased in price since I bought mine), and they're at the long end of the supported chart for ROCm...nearly EOL. But, they currently work great with llama.cpp and current ROCm 7.14. They're pretty fast and stable. I also have a Strix Halo 128GB, and while the Strix Halo can run bigger models, the dedicated GPUs are quite a bit faster. And, since the best models you can run at home are probably Qwen 3.6 27B and Gemma 4 31B, and those both fit comfortably on dual 32GB GPUs, I find I use the desktop more often than the Strix Halo.
Anyway, there's very little reason to spend 3x or more for the Nvidia ecosystem these days. The only exception is the Strix Halo vs the Nvidia GB10 platform. Strix Halo was a no-brainer when it was half the price, but it's risen in price to be almost the same price as an Asus GX10. In that one instance, I think the Nvidia based Asus is a better choice. The Strix Halo is almost too slow to make use of 128GB. Some MoE models are comfortable, like Laguna S2.1, so it's probable that there will someday be an MoE model that is better then Qwen 3.6 27B or Gemma 4 31B that won't run on 64GB but will run on 128GB. I don't know of one, yet, though.
AMD also used to focus almost entirely on their data center line and not pay any attention to their lower end stuff for more advanced AI features, but that's been changing, and ROCm support has broadened to include almost everything AMD ships now, including the smaller embedded stuff. I think they've realized that as long as the only way to develop AI for AMD was to have a quarter million dollars worth of hardware, they would always fall behind a platform that can be developed for and tested on consumer hardware.
>I'd recommend current Radeon AI Pro 9700 cards over the V620 now
That's about 4x the price of v620, which is 450 eurobucks. Can't imagine buying 4 of those at 4x the price.
Uhh.... I have a box with two AMD Radeon AI PRO R9700 32GB.
It was the most trouble-free AI setup that I have done. The driver worked right out of the box with the stock Fedora kernel, ROCM can be installed from the regular repo, and most AI software has ROCM builds by now.