I get about 50 t/s with the same model on a 3090, with about 100K context. I usually set harnesses to 90k.
It would be neat to also include NVFP4 variants for the Blackwell hardware.
I get about 50 t/s with the same model on a 3090, with about 100K context. I usually set harnesses to 90k.
It would be neat to also include NVFP4 variants for the Blackwell hardware.