Depends on quant size etc... Qwen3.6 35B-A3B Q4_K_XL a multiple 5060ti + tensor split mode + MPT will hit ~100/TPS without problem, and I have personally hit ~190/tps on a single 5090 on a friends machine getting them setup up. If you use Q6_K etc... it slows down, what quant were you using?
Quantization + KV cache paging + speculative decoding (MPT or draft) is a fairly good mixture here.
Some examples as I don't have access to run tests on a 5060ti right now:
Depends on quant size etc... Qwen3.6 35B-A3B Q4_K_XL a multiple 5060ti + tensor split mode + MPT will hit ~100/TPS without problem, and I have personally hit ~190/tps on a single 5090 on a friends machine getting them setup up. If you use Q6_K etc... it slows down, what quant were you using?
Quantization + KV cache paging + speculative decoding (MPT or draft) is a fairly good mixture here.
Some examples as I don't have access to run tests on a 5060ti right now:
And here are some logs on unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_XL with the 2x 1080ti + 1x titan from above: Q4_K_XL is a slight, acceptable degradation IMHO for performance like that.