> but I'm surprised to hear someone think that it's not even "barely usable".
write performance oriented and memory safe C++ code. Current coding assistants are glorified autocomplete for unit tests or short api endpoints or what have you but if you have to write any safety oriented code or you have to think about what the hardware does it's unusable.
I tried using several of the assistants and they write broken or non-performant code so regularly it's irresponsible to use them.
I've also had trouble having assistants help with CSS, which is ostensibly easier than performance oriented and memory safe C++
Isn't this a good reward function for RL? Take a codebase's test suite. Rip out a function, let the LLM rewrite the function, benchmark it and then RL it using the benchmark results.