I also find that the 35B A3B in Q4 outdoes Ternary Bonsai 27B.
Mind you, because I am not that interested in long-context, fire-and-forget tasks, I think the faster MoE model has more value generally.
I also find that the 35B A3B in Q4 outdoes Ternary Bonsai 27B.
Mind you, because I am not that interested in long-context, fire-and-forget tasks, I think the faster MoE model has more value generally.