In SWE I've found gpt-6-astra (high) inconsistent and oddly focused on overtly taking responsibility for mistakes it made rather than prioritizing concrete steps to rectify problems. Such steps once elicited are often either incomplete or beyond the scope.
Opus also does this and then writes comments in code or PR descriptions describing how it went wrong earlier in the session.
Just imagine how confused a human would have to be to do that. And we want to trust these clankers to build software.
The biggest issue with LLMs is that they still suck at general contextual awareness and ability to judge what is appropriate.
Yes I agree. I got it to vibe up a simple react router app. When it crashed it was obvious that it had totally swallowed all errors in the name of a tidy error page. Getting it to re-add logs and debuggable errors was an exercise in patience as astra just got more and more tweaked while trying to solve the problem.
From an alignment perspective I’ve got no idea who it’s aligned to but it isn’t me, the meat proxy, who just wants to know why it crashed.