yeah opencode just runs with it with lmstudio as llm engine.
It won't win benchmarks of course, but 20 tps generation is okay and letting it chug away on my local kubernetes cluster to troubleshoot is pretty nice. First start is pretty slow though, parsing is around 100-200 tps. Which is crap for long questions or harnesses like opencode which start around with thousands of tokens. Once cache is warm, pretty enjoyable.
I think it could have way more use cases where it's just 'good enough'. Email categorization, voice commands for the home. 1 year ago this was still a very miserable experience, but qwen 9B + better cache quantization and more tricks in the latest llm engines makes it viable.
Will it do difficult architectural refactors by just giving a one-shot prompt like opus allows, nope, not gonna happen.
On 8GB VRAM (RTX 2060 Super) + 64GB DRAM I can run Qwen 3.8 Flash Next in usable speeds. The last 2 weeks have resulted in quite some speed ups in Strata land.
Yeah simple, but menial refactors where IDEs don't have clear cut refactors yet. Or languages like CUE which has awesome tooling, but not so great refactors in its lsp yet.
Tell us more about what your RX580 can do with the small Qwen models. What workflows are you doing, and what kind of performance are you seeing?
yeah opencode just runs with it with lmstudio as llm engine.
It won't win benchmarks of course, but 20 tps generation is okay and letting it chug away on my local kubernetes cluster to troubleshoot is pretty nice. First start is pretty slow though, parsing is around 100-200 tps. Which is crap for long questions or harnesses like opencode which start around with thousands of tokens. Once cache is warm, pretty enjoyable.
I think it could have way more use cases where it's just 'good enough'. Email categorization, voice commands for the home. 1 year ago this was still a very miserable experience, but qwen 9B + better cache quantization and more tricks in the latest llm engines makes it viable.
Will it do difficult architectural refactors by just giving a one-shot prompt like opus allows, nope, not gonna happen.
Thank you
that's 8G vram? you're using it for coding?
On 8GB VRAM (RTX 2060 Super) + 64GB DRAM I can run Qwen 3.8 Flash Next in usable speeds. The last 2 weeks have resulted in quite some speed ups in Strata land.
Yep, it's 8GB vram from 2017 :D
Yeah simple, but menial refactors where IDEs don't have clear cut refactors yet. Or languages like CUE which has awesome tooling, but not so great refactors in its lsp yet.
[flagged]