I love testing the new models by asking them to code a toy RTS game. Here's what Terra did: https://senko.net/vibecode-bench/2026/rts-gpt-5.6-terra.html (one try, in codex app, xhigh effort)
Comparing this to other models, I find it similar to GPT-5.5 and a bit behind Sonnet 5. You can see how other models fared here: https://senko.net/vibecode-bench/ (you can also fetch the prompt and the the 5.6 Terra resulting code on from that page).
I don't have access to Sol yet (on a Plus sub, which should get it according to what I've read), so can't do the more interesting test. I'll update the above page as soon as I get access - hopefully soon.
this is so cool: it's playable (even though super boring since there are no enemies) and you can feel that a few iterations would make it very usable.
Which model is the best at the moment, for this kind of stuff, in your experience?
I'd say Fable 5: https://senko.net/vibecode-bench/2026/rts-fable-5.html
It even has enemies! (I'm not too mad about it not following my instructions because it can be fun to play :) And I generated that from Claude Code on my phone.
Sonnet 5 also produced a pretty nice version. You can see all of them here: https://senko.net/vibecode-bench/
Could we see the prompts, though?
I did a quick comparison of models a couple of months ago by giving it a MYTXTADV.BAS file and give them all the same prompt to create a sprite-based version of a text adventure game I wrote in Basic over 30 years ago.
It was interesting to see where the approaches were similar and where they diverged.
Very neat! Can't wait for the Sol version!
So the measure of a model is how well they can recreate something they easily have thousands of examples of in their training data. There's probably a better base RTS on github somewhere for free.
Well, it is a silly test, not a scientific benchmark.
However, I would say it is a measure (not the measure). If you look at the entries, there's a lot of variation - definitely not something they memorized outright.
And the test itself is deceptively simple. You need to do canvas rendering, there's pathfinding, command queueing, terrain generation, etc. There are some subtle click handler bugs (various LLMs often stumble on those). And I ask the model to do it all in one file, further increasing the complexity of the task.
And the result is something that you can instantly evaluate. And if the result is any good, even play! So yeah, I think it's a fair test.
I'm sure it'll get saturated at some point. Actually I started with Minesweeper and switched to RTS last December, because Minesweeper was being saturated. I'm expecting (hoping?) the RTS test will last until the end of this year...
Ask it to generate a completely bananas game idea and it will easily do it. Prototyping any type of game with simple graphics has been solved since many generations of models back. Claiming it's because it has that game in its training data is nonsense.
The goalpost velocity is approaching light speed…
>There's probably a better base RTS on github somewhere for free.
I... I think you are missing the point.