No, this is not a small model. I believe 4x Sparks is possible but 4.1 is pretty big for a flash model.
MiMo 2.6 Flash is the most capable model you can currently run on 2x Sparks without resorting to a quantised version of something.
No, this is not a small model. I believe 4x Sparks is possible but 4.1 is pretty big for a flash model.
MiMo 2.6 Flash is the most capable model you can currently run on 2x Sparks without resorting to a quantised version of something.
I meant quantized. You can get an EXL3 quantized version thats almost the same performance.
My experience running quantised models is that they look the same at first but are all noticeably worse once the context size gets beyond anything trivial.
I've tried both 2bpw and 3bpw quants for QWEN3.8 27b, and while i do notice a difference in terms of higher rates of getting stuck in loops, they are both reasonably usable.
Are they as good as opus 5.5? No.
Are they about on par with opus 4.6? Yes.
The long run tasks i run seem to be about the same level of quality, though there are obvious differences in model styles, token budgets, etc....