There's a mass psychosis going on in companies right now. Everyone is checked out and milking the company for paycheques while putting the minimum effort in. Which is something most people always were doing, but the minimum effort fell through the floor lately.

User forgot to tell LLM not to duplicate. LLM produced a lovely, working, big line count PR: manager of LLM was happy, keeps paying for it, so all is good right?

Exactly. Precisely. Definitely.

I see what you describe all the time, because I do review the code the models do produce.

It's not just incredibly verbose: it's constantly missing that there's an obvious, elegant, small, way to solve what was asked and instead it goes ballistic and creates nonsense.

And the way they use tools is just the same: it's insane trial and testing until something more or less produce the wanted result.

I've explained it here already but the craziest I had was, like you, a one line test that was basically the following:

    if ( a >= 0xab000000 && a <= 0xabffffff)

But the model decide to go nuts: it noticed a pattern (just like it notices a pattern in your example) and decided to convert the native integers to strings to then do substring matching on the hexadecimal representation of the number.

I.

Shit.

You.

Not.

And all the people here who are saying that "it works" have no idea as to the amount of technical debt they're creating.

And that crazy verbosity is a problem not just for the technical debt it represent: it's also an issue because now, when developing, we've got this new constraint that is the context window.

It's a nice tool but it should be used with caution.

Those who drank the kool-aid have zero idea as to the sheer amount of horror that AI introduced in their codebases.

Does submitting this kind of PR count against their promotion or productivity?

Is there any way it could?

Love to hear from companies making progress on this front.

[flagged]

>but a human's still the one submitting the PR for review

Where I work, the LLM writes the ticket and does all the coding. As soon as the LLM feels like it's done, it automatically submits and reviews the PR itself. The humans blindly click "approve" without reading the PR. And when the required number of humans have blindly clicked approve, a human blindly presses another button that merges the code. All the text in the ticket, the code, the PR and review is far too voluminous and verbose to easily read, so nobody does. These humans didn't start out as vibe coders, they used to be engineers.

How do you think this will work out for us?

A PR with a minimal title and empty description should be refused at submission. If the human is so disinterested that they're using LLM generated code and then can't explain the purpose, that human should be prevent from making the PR. Working as a solo dev, it is very easy to be lazy like that, and I'm as guilty as anyone. Working in teams with actual reviews should absolutely have much more strict policies of what is considered a valid PR

It was already a well-known review point way before LLM's. Every book about code cleanliness has some point about "write WHY not WHAT when commenting code". It's a point everyone makes, because it's such an ubiquitous thing.

Of course the standard bad example is

    // add 1 to a
    a++;

    // the flipDinkleWooptie method doesn't add one in this case
    // because there is no wooptie register, so we manually
    // add one here.
    a++;

He's been complaining about this pretty much since we started working, more than 10 years ago.

The challenge of hard rules like this is that they're always overly restrictive. I've made multi-thousand file PRs that needed two lines of description including the title, and 5 line PRs that needed a 1hr presentation to fully explain them.

My personal guideline is that writing for humans should be done by humans.

Your team can't spend effort to write 3-5 sentences by yourself ?

Work doesn't start with a PR description though. I'm assuming most people that are using LLMs start with some sort of document (plan, spec, intent, etc) which captures intent.

I guess you could also use all the session rollouts saved to disk that were related to that task, and distill them somehow.

The difference is an LLM can convert a stream of consciousness into well-formed prose for approximately free; I assume ‘provide some context’ means ‘brain dump’ in the OP

> you need to actually put some thought and effort into your prompts and provide appropriate context and examples a lot of the times if you have a very specific result that you're envisioning.

Ain't gonna happen. By that point in time, I might as well do it myself. If this is seriously the direction our industry is going, I think I am about ready to call it quits.

There's centralized tooling for the PR descriptions, but I have some local flows where I try to provide more careful prompting and examples to get it to write better. It definitely helps but it's still not great and I'm often unsure if all the extra prompting is worth the effort.

It seems to me that you might as well put that thought and effort into describing the changes yourself. Most of my changes warrant 0-2 sentences of description below the header. A few particularly subtle bug fixes may need more to make sense of, but detailed descriptions of why certain mitigations are necessary are in my case more likely to end up as code comments.

That describing a change should frequently be such a difficult problem that instead of just doing it you prefer to put thought and effort into telling an LLM to do it smells bad to me. For me, the thought and effort spent writing a description is mostly already amortized through thinking clearly about the problem and performing the work. I have a much easier time describing what I just did and why than a machine that has no access to that information unless I tell it.

It’s definitely possible to get much better output with prompting. I know, because when I’m faced with a “standard” PR description full of technical clutter, I can paste the link to Claude and ask “ELI5 what the problem actually is, any important context, what changed, and why that solves the problem.” And most of the time it converts it into something pretty good and readable.