My experience running quantised models is that they look the same at first but are all noticeably worse once the context size gets beyond anything trivial.
My experience running quantised models is that they look the same at first but are all noticeably worse once the context size gets beyond anything trivial.
I've tried both 2bpw and 3bpw quants for QWEN3.8 27b, and while i do notice a difference in terms of higher rates of getting stuck in loops, they are both reasonably usable.
Are they as good as opus 5.5? No.
Are they about on par with opus 4.6? Yes.
The long run tasks i run seem to be about the same level of quality, though there are obvious differences in model styles, token budgets, etc....