Their last 2 models were a disaster and didn't live up to their self reported evals. I hope this time around is much better. I'm happy tho that they are still figuring out how to get better even if only measured against themselves. This is great, the more options we have the better for everyone.
I’m a casual user out of principle and it’s fine. Nothing extraordinary but capable of assisting coding sessions. It won’t one shot a cancer medicine but it handles my Rust projects well. Mind you I don’t like the style of LLM use where people just brute force their way to a working state from a very high level prompt. I mostly ask it to implement tickets/issues I write beforehand.
I've been using it since February this year almost exclusively. Occasionally ran the same prompt through openrouter to check if it's "my context" or "the model" when I get a poor result.
Mistrals models aren't the best. They certainly are measurably worse when I run them "blank slate". That is: without skills, AGENTS.md, MCPs and so on. But who does that anyway?
Aside from these harnesses, I run many one-off agent threads. They run at most half an hour and fill the context window not even 20% before I stop and "handover". And I tend to be annoyingly demanding and nitpicky in my skills, AGENT.md, prompts and issues (that an agent uses as prompt).
In my workflow, the context is what makes or breaks how well the task is performed almost all of the times. Mistrals models aren't "the best" - some Chinese models are cheaper (with similar outcome), and anthropics' models outperform the output consistently but for 20x or more of the price of Mistral (that is: when using the API/token pricing). But Mistrals models are certainly "good enough". Good enough is good, too.
[dead]