https://huggingface.co/unsloth/Mistral-Large-4-1T-A52B-GGUF
I have no idea how you could shrink this down to fit on any type of prosumer hardware though. I doubt it's possible.
https://huggingface.co/unsloth/Mistral-Large-4-1T-A52B-GGUF
I have no idea how you could shrink this down to fit on any type of prosumer hardware though. I doubt it's possible.
Yes, you are right, and I was too optimistic. I'd have to level up.
This is the stuff I've put together and it is clearly too small for this:
https://jacquesmattheij.com/local-inference-with-large-langu...
Could you mention the exact model of the PCIE switch/host adapter you are using? The Aliexpress links are always a bit ambiguous. I am working with the c-payne switch https://c-payne.com/products/pcie-gen4-switch-5x-x16-microch...
The system is a $25k custom build with 8 video cards (4x RTX PRO 4000, 2x RTX 4090, 2x RTX 3090). That price was only valid for early 2026.