Human beings make relatively predictable mistakes, to the extent that I can skim read large PRs with mental heuristics and work out whether the dev thought carefully about the problem when designing a solution, and whether they hit common pitfalls etc.
AI code generation tends to pass a bunch of those heuristics while generating code that you can only identify as nonsense by completely understanding it, which takes a lot more time and effort. It can generate sensible variable and function names, concise functions, relatively decent documentation etc while misunderstanding the problem space in subtle ways that human beings very rarely do.
No. They’re not hard to detect because they’re good, they’re “hard” to detect because understanding code takes time, and you’re putting that work on the maintainer.
I find it hard to believe that people who don’t intuit this have ever been on the receiving end. If I fill up your email inbox with LLM slop, would you consider that I’ve done you a favor because some of it’s helpful? Or would you care more about the time you’re wasting on the rest, and that it’d take longer to find the good bits than to make them yourself, and just block me?
If I want to ban variables that end in an ‘S’ I can, and I can write a script to detect them.
I want to ban AI PRs because they cause technical, social and potentially legal problems, but I can’t. If I put a policy in place, people will ignore it, and I can’t write a script to detect it.
This will cause problems for many open source products because ignorant accidental and/or wilful ‘contributors’ will choose to do so and now the responsibility on me is to detect their contributions at scale.
Human beings make relatively predictable mistakes, to the extent that I can skim read large PRs with mental heuristics and work out whether the dev thought carefully about the problem when designing a solution, and whether they hit common pitfalls etc.
AI code generation tends to pass a bunch of those heuristics while generating code that you can only identify as nonsense by completely understanding it, which takes a lot more time and effort. It can generate sensible variable and function names, concise functions, relatively decent documentation etc while misunderstanding the problem space in subtle ways that human beings very rarely do.
Sounds like it raises the bar on verification requirements.
... In a world where someone almost compromised SSL via a detail-missed trust attack... Maybe that's okay?
It’s easy to point out case studies of humans when AI is relatively new and case studies of its commits are limited.
No. They’re not hard to detect because they’re good, they’re “hard” to detect because understanding code takes time, and you’re putting that work on the maintainer.
I find it hard to believe that people who don’t intuit this have ever been on the receiving end. If I fill up your email inbox with LLM slop, would you consider that I’ve done you a favor because some of it’s helpful? Or would you care more about the time you’re wasting on the rest, and that it’d take longer to find the good bits than to make them yourself, and just block me?
If I want to ban variables that end in an ‘S’ I can, and I can write a script to detect them.
I want to ban AI PRs because they cause technical, social and potentially legal problems, but I can’t. If I put a policy in place, people will ignore it, and I can’t write a script to detect it.
This will cause problems for many open source products because ignorant accidental and/or wilful ‘contributors’ will choose to do so and now the responsibility on me is to detect their contributions at scale.
That's like saying if you could not tell the person calling you was a scammer and lost money, then the call passes muster.
As long as the person submitting PR has put in the effort to ensure it is of high quality, it should not matter what tool they used, right?
Well, overwhelming majority vibies seem not to. Welcome to "block all chinese and russian IPs" era, open source AI edition.
[dead]