What is more frightening about this than safe C assembly is that this level of implementation is achievable not with a SOTA model, but with a cost effective model like KIMI. There was human judgment involved in the middle, but reading the article, My reading of the process is as follows:
1.A developer identified the necessity of inline assembly.
2.Defined the safety boundaries for 'memory-safe' inline assembly.
3.Established strict policies for memory access.
4.Curated an allowlist of permissible instructions.
5.Set rigorous test criteria and 'done' conditions.
In short, with the overall guardrails in place, a sub agent loop was run, and this level of code was produced. This raises a number of interesting points about how we should use AI. I haven't looked at all the code, but the idea of passing assembly through safe zones without memory access, and using that as a foundation to achieve this level of implementation through AI, is quite impressive
The utility margin of SOTA models is greatly overstated. You have to pick seriously niche puzzles to make the money worth spending.
Anyway, this is also very useful for humans to use, so it's mostly a lovely coincidence this level of safety arrived with useful chatbots.
I honestly would be surprised if someone used AI in any other way to achieve this
Same here.