> The benchmark is saturated. Frontier models are tested with an armadillo in fishnet tights jaywalking on Mars.
OK well I couldn't resist this one:
llm -m claude-opus-5.5 'Generate an SVG of an armadillo in fishnet tights jaywalking on Mars'
llm -m gpt-6.1-sol 'Generate an SVG of an armadillo in fishnet tights jaywalking on Mars'
llm -m gemini-3.8-flash 'Generate an SVG of an armadillo in fishnet tights jaywalking on Mars'
llm -m mistral/mistral-large-4 'Generate an SVG of an armadillo in fishnet tights jaywalking on Mars'
> The benchmark is saturated. Frontier models are tested with an armadillo in fishnet tights jaywalking on Mars.
OK well I couldn't resist this one:
Default reasoning levels for each: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...The rover honking is pretty silly, opus has a good sense of humor
I'm getting 403 inside the tool for this one. (The pelican bike on top works) This has been happening a lot recently.
That's a GitHub rate limit. Try again now, I just pushed a hopeful fix: https://github.com/simonw/tools/commit/7793fb74c2d37bd613cdc...
It does seem to, thanks!
Big L for mistral in this benchmark. Sorry Europe.
Not so sure, apparently it is the only one that considered that there are no paved roads on Mars.
To be fair, they don't have Armadillos in Europe. of course, you could say the same for Mars...
Gemini wins this one clearly. Honey please!
https://chatgpt.com/s/m_6ac53d4e5b0c8191949050dbf1f402d7
Not sure I'd call it jaywalking exactly but pretty good