Fable would be unusable for you in a more literal way, since it just directly refuses to answer any query even remotely related to biology
Fable would be unusable for you in a more literal way, since it just directly refuses to answer any query even remotely related to biology
I’m very aware of this as well.
How do local models work? I’m specifically interested in things that run in the 32-128GiB space. (I don’t care about bio specifically; just trying to track when local models start surpassing cloud ones in some practical dimensions).
At different size ranges:
- Qwen3.6 27B runs quite nicely on a 32GB GPU, and it's a mostly usable coding agent. The biggest difference with a frontier model is that a 27B forces you work in chunks between 100-200k tokens, and to maintain a clear understanding of how your code works. If you try to vibecode without understanding, yeah, it's going to get ugly. Also, it's better at coding than many other tasks.
- DeepSeek V4 Flash is apparently quite nice if happen to have 256GB of RAM lying around, lol. Again, not a frontier model, but antirez really likes it.