The N-gram embedding model thing is absolutely crazy. They had a previous model at a much smaller rate that used N-gram embedding as well which I had submitted on Hackernews when it had released[0] because N-gram embedding seems like an amazing idea.
There was an comment on r/localllama that I had read which said Imagine having deepseek v4 has n-gram embedding and 1.3 (ternary) or 1 bit model combined, it was when deepseek v4 hadn't released.
I think that there is a lot of research and proof's being released. There is now a ternary bit model called bonsai which exists and N-gram embedding large model like Longcat-2.0 existing as well. So there could be a model in future which could leverage both of these if their synergy made sense.