First impressions:
- Very fast, easily beats GPT 5.5/Opus 4.8/GLM 5.2 because of higher t/s (around 90?) and very high token efficiency
- Very good price, no contest vs GPT and Opus which are very overpriced if you pay API costs, and probably cheaper than GLM 5.2 when you take into account the token efficiency.
- Will take quite a while to get a feel for how smart it is, but it's definitely good, I'd say in the same tier as opus, occupying the lower end of that tier together with GLM 5.2.
Concur.
Tried on a "this test suite is weaker than I'd like, too often depending on internal state rather than outcomes" problem via Cursor, asking it to "review and suggest solutions." It gave me a quality overview of the test approaches, strengths, weaknesses, and gaps then recommended a disciplined multi-prong approach based on a common, trusted testing library (https://hypothesis.readthedocs.io/en/latest/). It broke down the things we could do this improvement pass or leave to later (staged scoping), identified some very hard/possibly-out-of-scope cases and gave me the option of focusing on them or not, and organized new tests in a logical way. After one round of feedback and plan tuning, I put it in agent mode and let it work. A few minutes later I had a much better test suite.
Have not tried Grok before and didn't have much confidence, but it did great. Exactly the sort of complex, detailed, nuanced analysis and multi-step task I would previously only trusted to GPT or Opus.
_Update_: It's now also found a substantive long-standing bug. After testing improved asked it to do overall code and packaging review. It caught a few glitches and oversights, mostly cosmetic IMO, but certainly worth cleaning up. But also some error-handling weaknesses, and one embarrassing functional bug. Which it has now also fixed and added to the tests. Color me impressed.
My benchmark is ripping tailwind out of a few year old elixir Phoenix liveview app, and replacing it with component level scoped styles
It's a good and complex task, that requires touching the build system, most components, the stylesheets, and more. Opus 4.6 could barely do it. Sonnet 4 cannot (haven't tried 5 yet). MiniMax actually did fairly well
Grok aced it, rather quickly and cheaply, surprisingly
I run each through Oh my pi, with dexter providing the LSP for elixir
This is what I don't understand. Why would I use this "cheaper" model when it's still going to be more expensive than Codex on the $200 plan? Are they only targeting business users who pay per token?
You can buy the Grok plan, Cursor also has a plan which includes grok 4.5, but I don't know how subsidized they are compared to codex or claude code plans.
Got it, I got confused then. Would be good to see how it compares, Codex is quite generous for the $200 with GPT 5.5. They give a lot of resets, I just use mine in Fast mode (1.5x speed, 2.5x usage) almost all the time. But it's not exactly fast.
+1. Also the TUI feels well made, in striking contrast to recent Claude Code.
The input price is quite high. That's what gets you.
hmm interesting. maybe im doing something wrong. this model feels borderline unusable to me. it fumbles the most basic asks that require very little to no context consistently (inline these helper functions - re-rewrote half of the modules involved instead of making a 10 line change)
Sounds like an issue with your harness or something.
There is one thing to note here.
The fact that it is more token efficient will itself lead it to be smarter since the context will be smaller for the same task. However, in opus models, you'll have built internal correlations like "if it did X it will usually do Y" which may not be true here, since grok 4.5 may have done X purely due to the smaller context size, but can't do Y cos it wasn't RLd on that pattern enough. So it will be a unique experience as far as opus tier models go.