I saw a (highly reputable of course) tweet a few days back suggesting all of OpenRouter's DS4 traffic would fit on a single rack of modern Nvidia kit.
I saw a (highly reputable of course) tweet a few days back suggesting all of OpenRouter's DS4 traffic would fit on a single rack of modern Nvidia kit.
Hm, that’s doubtful. I think even one instance of DS 4.1 Flash alone at ~500B parameters needs a full node (8 GPUs).
4xh200 seem to be enough if you cpu offload the engrams (which is pretty typical and even expected).[1]
And these are quite old cards by now. But yeah "all of OpenRouters traffic" is gonna take more than a rack
https://recipes.vllm.ai/deepseek-ai/DeepSeek-V4.1-Flash?hard...
[dead]