Curious how people feel about this compared to DS4 Flash, given they are pretty close in size. Also curious how well it holds up to heavy quantization.
DS4 Flash can currently run reasonably well on systems with ~96gb+ RAM, I wonder if Hy3 can compete there.
> given they are pretty close in size
One thing that might not be obvious about about DSV4 is how much innovation the Deepseek team implemented in its architecture. When llama.cpp fully supports its lightning indexer, the full 1M context will only require about 6G of RAM. So even though they are similar in size, I believe Deepseek will be much more efficient in that regard.
> I wonder if Hy3 can compete there
Highly depends on how well Hy3 is resilient to quantization. DSV4 is useful even at 2-bit quants.
I have been telling people exactly this the last few months.
We have not seen the full power of deepseek v4 yet.
That's a 2-bit quant of DS4 flash. You're probably better off running Qwen3.6-27B at Q8.
Having heavily evaluated both antirez’s ds4 flash and Qwen 3.6 27B at FP8 and Q8: it depends. The quantised Flash is better in a number of tasks despite running much slower on my DGX Spark-alike.
27B is amazing for its size but has some surprising limits when used for longer agentic coding sessions, especially if you’re using tools that are outside the stock standard web tech stuff: it really isn’t good at Relay, for example.
I think its good advice to test both on your own evals for sure, but the MoE parameters are already natively FP4 in ds4. Dropping to 2bpw isn't as big of a loss as it seems (and as corroborated by antirez's work).
Its also only 13B active, so your decode speed would be nearly 2x that of Qwen3.6-27B. So there are other latent benefits as well.
z-lab has been dropping dflash addons for a lot of models
https://huggingface.co/collections/z-lab/dflash
I'm running the qwen3.6-27B + dflash on a spark and tgen is way up, but keep the draft count low, acceptance rate is terrible beyond half a dozen and it requires more memory
Wow thanks for pointing this out! This is actually what I was hoping would happen when the deepspec stuff dropped! And having zlab create these confirms my bias that I think open models are the way.
For most coding or agentic tasks, Qwen 3.6 27B likely outperforms, yes.
For 'general intelligence', DS4 Flash seems to be a noticeable step up still.
I suspect it would depend on the task. DS4-flash does, as previously mentioned, handle quantization very well. Even at 2-bit it's still very coherent.
qwen 27b at q8 is slower and worse than ds4 at q2 in my experience.
Isn't Q8 way overkill these days? I see many graphs showing Q4 or Q5 having less than %1 deviation. Nvidia's NVFP4 Qwen quantization should be even better due to its better training methods.
Q8 isn't overkill if you have sufficient RAM to fit the whole model, and you care about quality. There's a number of people who have enough hardware to fit exactly one 27B to 35B size Q8 model and not more than that, so if you can fit the whole thing in Q8, no reason to use Q4 or Q6.
When orgs/bencmarks claim 1% deviation, in most cases that means measuring perplexity loss on datasets like wikitext or c4. Even if the loss is calculated via KLD or similar, its not a good proxy for whats actually degradaing at the task level across an entire rollout.
And for MoEs, very small amounts of loss can mean you're flipped to entirely different experts (this is also a problem more broadly with numerical stability issues too).
Qwen3.6 below Q8 often can't exit a reasoning loop (until it hits max output token count), forgets to insert a tool call, often mistakenly inserts them inside the thinking block... It's still usable though.
1.01 over 30k tokens is over a googol (a large number with 100 zeroes)
It depends on model size I think, but yeah, from my understanding at ~30B and below Q6 or even Q4 will get you 95%+ of the way there
Careful with those graphs, they're usually evaluating the model on KLD on relatively short transcripts. When you're running with 100k token contexts and the model running close loop a difference that looks small in terms of KLD may be quite substantial.
I'm not aware of any great benchmarks that work by giving it a live agentic harness and a number of realistic tasks that take most of the context window to accomplish and evaluate success rate and tokens to completion... but that's what you'd really want to use to judge different quantization levels.
[dead]
I was playing with Hy3 via openrouter yesterday (and I've also been using DS4 Flash/Pro as a daily driver since I cancelled my Anthropic sub a week ago).
I've found DS4 Flash to be very temperental (via Claude Code). The speed is great, but it often builds a completely wrong mental model and charges off down the wrong path. I find myself needing to rein it in regularly (and also compact the history, which undercuts the whole cache price advantage).
Hy3 isn't as fast, but so far it seems to stay on track much more reliably than DS4 Flash. It also doesn't seem to degrade as much with longer context. I'm not sure what the real pricing is, but I feel like it's a very competitive model.
As an aside, I also nabbed a 50m token pack for LongCat 2.0 to give it a whirl. Not free, but it's so cheap they're basically giving it away. Very impressed too - seems roughly on par with Hy3. Not frontier-level intelligence, but a dependable workhorse that can navigate a codebase well and can reliably execute what you tell it to do.
Hy3 lacks the DSv4 architecture's KV Cache efficiency.
Whereas I can run DSv4 Flash on a pair of DGX Sparks and have enough memory left over for 3M tokens of KV cache, with Hy3 (quantized to FP4), there is only room for ~130K tokens of KV cache.
Lower context window notwithstanding, Hy3's coding benchmarks hold their own against DeepSeek v4 Pro & MiMo v2.5 Pro. That's quite something for a model priced like DeepSeek v4 Flash & MiMo v2.5 (for non-cached tokens), which are 3x cheaper than their respective Pro variants.
It's impressive indeed. I would also expect the next checkpoint of DSv4 Flash to come in somewhere at this level (DeepSeek has had over 2 months to continue training since it released).
It's exciting that the open models continue to get better and more efficient across the board!
DS4-Flash is not only "significantly" smaller, it will also benefit from a lot more speed thanks to DSpark
299B for Hy3 vs 284B* for Flash
Edit: fixed, got bad info
flash is 284b isnt it? https://artificialanalysis.ai/models/deepseek-v4-flash
Oh, it is. I was looking at the Huggingface repo which listed the lower number at the top of the page, looks like that's wrong.
Same, I was relying on that. Weird!
I don’t like DS4 in my experiences with it I still prefer qwen locally and glm on api