For this "LLM style were already the most popular, that's how LLM works, then how come LLM style is so weird and annoying" I have 2 theories.

First, LLM style did not even exist, it's a match of several different styles, choice of words and phrases.

Second, LLM has turned a slight plurality into a 100% exclusivity.

Say, there are 20 different choices to say the same thing. They are more or less evenly distributed, one of them is a slightly more common. LLM chooses the most common one. This means that

   situation before : 20 options,  5% frequency each
   situation now    :  1 option, 100% frequency

I think these 2 theories explain how can LLM both sound bad, and "be the most common stye, how humans have always talked" (it isn't).

Also, if the second theory is true, that is, LLM style is not very frequent among humans, that means that if you see someone on the internet that talks like an LLM, he probably is one.

This is total speculation, but my guess is that human reviewers of AI-written text (whether code or natural language) are more likely to think that the text with emoji check marks, or dart-targets, or whatever, are correct. (My understanding is that many of these models are fine-tuned using humans who manually review their outputs.) In other words, LLMs were inadvertently trained to seem correct, and a little message that says "Boom! Task complete! How else may I help?" subconsciously leads you to think it's correct.

You may thank millenial hipsters who used think emojis are cute and proliferation of little javascript libraries authored by them on your friendly neighborhood githubs.

Later the cutest of the emojis paved their way into templates used by bots and tools, and it exploded like colorful vomit confetti all over the internets.

When I see this emojiful text, my first association is not with an LLM, but with a lumberjack-bearded hipster wearing thick-framed fake glasses and tight garish clothes, rolling on a segway or an equivalent machine while sipping a soy latte.

LLMs write things in a certain style because that's how the base models are fine tuned before being given to the public.

It's not because they can't write PRs indistinguishable from humans, or can't write code without Emojis. It's because they don't want to freak out the general public so they have essentially poisoned the models to stave off regulation a little bit longer.

My impression is that this style started with apple products. I remember distinctly opening a terminal and many command lines (mostly Javascript frameworks) applications were showing emoji in the terminal way before LLMs.

But maybe it originated somewhere else.. In Javascript libraries..?

I do remember 1 example of an emoji in tech docs before all of this -- learning github actions (which based on my blog happened in 2021 for me, before ChatGPT release), at one point they had an apple emoji at the final stage saying "done". (I am sure there are others, I just do not remember them.)

But agree excessive emoji's, tables of things, and just being overly verbose are tells for me anymore.

I wonder if there's an analogy to the style of Nigerian e-mail scams, that always contain spelling errors, and conclude with "God Bless." If the writing looks too literate, people might actually read and critique it.

God Bless.

RLHF and system prompt, I assume. But isn't being able to identify LLM output a good thing?

I wonder if it's due to emojis being able to express a large amount of infomation per token. For instance, the bulls-eye emoji is 16 bits. Also, Emoji's don't have the language barrier.

There's some research that shows that LLMs finetuned to write malicious code (with security vulnerabilities) also becomes more malicious (including claiming that Hitler is a role model).

So it's entirely possible that training in one area (eg: Reddit discourse) might influence other areas (such as PRs)

https://arxiv.org/html/2502.17424v1

It reminds me of this, but without the logic and structure: https://gitmoji.dev/

Don't Github have emoji reactions? I would assume that those tie "PR" and "needs emojis" closely together.

I'm glad that AI slop is detectable. So, for now the repulsive emoji crap is a useful heuristic to me that someone is wasting my time. In a few years once it is harder to detect I expect I'm going to have a harder and more frustrating time. For this reason I hope people don't start altering their prompts to make them harder to detect as LLM generated to people with a modicum of intelligence left.

> Why have the LLMs „learned“ to write PRs (and other stuff) this way?

They didn't learn how to write PRs. They "learned" how to write text.

Just like generic images coming out of OpenAI have the same style and yellow tint, so does text. It averages down to a basic tiktok/threads/whatever comment.

Plus whatever bias training sets and methodology introduced

I tell my team that the diff already perfectly describes what changed. The commits and PR are to convey WHY and in what context and what we learned (or should look out for). Putting the "what" in the thing meant for the "why" is using the tools incorrectly.

you mean we will get even more of these sort of useless comments?

  // loop over list and act on items
  for each _, item := range items {
    item.act()
  }

It destroys the value of code review and wastes the reviewers time.

Code review is one of the places where experience is transferred. It is disheartening to leave thoughtful comments and have them met with "I duno. I just had [AI] do it."

If all you do is 'review' the output of your prompting before cutting a CR, I'd prefer you just send the prompt.

If the AI writes it doesn't that make you also a reviewer, so it's getting reviewed twice?

I don't think this is what they were saying.

  > But once that’s done, why not?

Be honest here. I don't think you do. Just like none of us have the same understanding of the code somebody else wrote. It's just a fact that you understand the code you wrote better than code you didn't.

I'm not saying you don't understand the code, that's different. But there's a deeper understanding to code you wrote, right? You might write something one way because you had an idea to try something in the future based on an idea to had while finding some bug. Or you might write it some way because some obscure part of the codebase. Or maybe because you have intuition about the customer.

But when AI writes the code, who has responsibility over it? Where can I go to ask why some choice was made? That's important context I need to write code with you as a team. That's important context a (good) engineering manager needs to ensure you're on the right direction. If you respond "well that's what the AI did" then how that any different from the intern saying "that's how I did it at the last place." It's a non-answer, and infuriating. You could also try to bullshit an answer, guessing why the AI did that (helpful since you promoted it), but you're still guessing and now being disingenuous. It's a bit more helpful, but still not very helpful. It's incredibly rude to your coworkers to just bullshit. Personally I'd rather someone say "I don't know" and truthfully I respect them more for that. (I actually really do respect people that can admit they don't know something. Especially in our field where egos are quite high. It's can be a mark of trust that's *very* valuable)

Sure, the AI can read the whole codebase, but you have hundreds or thousands of hours in that codebase. Don't sell yourself short.

Honestly I don't mind the AI acting as a reviewer to be a check before you submit a PR, but it just doesn't have the context to write good code. AI tries to write code like a junior, fixing the obvious problem that's right in front of you. But it doesn't fix the subtle problems that come with foresight. No, I want you to stumble through that code because while you write code you're also debugging and designing. Your brain works in parallel, right? I bet it does even if you don't know it. I want you stumbling through because that struggling is helping you learn more about the code and the context that isn't explicitly written. I want you to develop ideas and gain insights.

But AI writing code? That's like measuring how good a developer is by the number of lines of code they write. I'll take quality over quantity any day of the week. Quality makes the business run better and waste fewer dollars debugging the spaghetti and duct tape called "tech debt".

You're not "reviewing" ai's slop code. If you're using it for generation, use it as a starting point and fix it up to the proper code quality

I would love to know how much time and computing power is spent by people who write bullet points and have ChatGPT expand them out to full paragraphs only for every recipient to use ChatGPT to summarize them back down to bullet points.

> AI tools just made it worse.

That's why it isn't necessary to add the "to be fair" comment i see crop up every time someone complains about the low quality of AI.

Dealing with low effort people is bad enough without encouraging more people to be the same. We don't need tools to make life worse.

There is one way in which AI has made it easier: instead of maintainers trying to figure out how to talk someone into being a productive contributor, now "just reach for the banhammer" is a reasonable response.

This comment seems to not appreciate how changing the scope of impact is itself a gigantic problem (and the one that needs to be immediately solved for).

It's as if someone created a device that made cancer airborne and contagious and you come in to say "to be fair, cancer existed before this device, the device just made it way worse". Yes? And? Do you have a solution to solving the cancer? Then pointing it out really isn't doing anything. Focus on getting people to stop using the contagious aerosol first.

If my neighbors let their dog poop in my yard and leave it I have a problem.

If a company builds an industrial poop delivery system that lets anyone with dog poop deliver it directly into my yard with the push of a button I have a much different and much bigger problem

This makes no sense, and it’s absurd anyone thinks it does. If the AI PR were any good, it wouldn’t need review. And if it does need review, why would the AI be trustworthy if it did a poor job the first time?

This is like reviewing your own PRs, it completely defeats the purpose.

And no, using different models doesn’t fix the issue. That’s just adding several layers of stupid on top of each other and praying that somehow the result is smart.

> You know you can AI review the PR too, don't be such a curmudgeon. I have PR's at work I and coworkers fully AI generated and fully AI review. And

Waiting for the rest of the comment to load in order to figure out if it's sincere or parody.

One Furby codes and a second one reviews...

Please be doing a bit

If An AI can do a review then why would you put it up for others to review? Just use the AI to do the review yourself before creating a PR.

When I picture a team using their AI to both write and review PRs, I think of the "obama medal award" meme

If your team is stuck at this stage, you need to wake up and re-evaluate.

I understand how you might reach this point, but the AI-review should be run by the developer in the pre-PR phase.

did AI write this comment?

> fully AI generated and fully AI review

This reminds me of an awesome bit by Žižek where he describes an ultra-modern approach to dating. She brings the vibrator, he brings the synthetic sleeve, and after all the buzzing begins and the simulacra are getting on well, the humans sigh in relief. Now that this is out of the way they can just have a tea and a chat.

It's clearly ridiculous, yet at the point where papers or PRs are written by robots, reviewed by robots, for eventual usage/consumption/summary by yet more robots, it becomes very relevant. At some point one must ask, what is it all for, and should we maybe just skip some of these steps or revisit some assumptions about what we're trying to accomplish

> And

Do you review your comments too with AI?

> I have PR's at work I and coworkers fully AI generated and fully AI review.

I first read that as "coworkers (who are) fully AI generated" and I didn't bat an eye.

All the AI hype has made me immune to AI related surprises. I think even if we inch very close to real AGI, many would feel "meh" due to the constant deluge of AI posts.

So how do you catch the errors that AI made in the pull request? Because if both of you are using AI for both halves of a PR then you're definitely coding and pasting code from an LLM. Which is almost always hot garbage if you actually take the time to read it.

Hahahahah well done :dart-emoji:

AIs generating code which will then be reviewed by AIs. Résumés generated by AIs being evaluated by AI recruiters. This timeline is turning into such a hilarious clown world. The future is bleak.

"Let the AI check its own homework, what could go wrong?"

Satire? Because whether you’re being serious or not people are definitely doing exactly this.

I don't need an AI generated PR summary because the AI is unlikely to understand why the changes are being made, and specifically why you took the approach(es) that you did.

I can see the code, I know what changed. Give me the logic behind this change. Tell me what issues you ran into during the implementation and how you solved them. Tell me what other approaches you considered and ruled out.

Just saying "This change un-links frobulation from reticulating splines by doing the following" isn't useful. It's like adding code comments that tell you what the next line does; if I want to know that I'll just read the next line.

But that's not what a PR summary is best used for. I don't need links to exact files, the Diff/Files tab is a click away and it usually has a nice search feature. The Commits tab is a little bit less helpful, but also already exists. I don't need an AI telling me stuff already at my fingertips.

A good PR summary should be the why of the PR. Not redundantly repeat what changed, give me description of why it changed, what alternatives were tested, what you think the struggles were, what you think the consequences may be, what you expect the next steps to be, etc.

I've never seen an AI generated summary that comes close to answering any of those questions. An AI generated summary is a bit like that junior developer that adds plenty of comments but all the comments are:

    // add x and y
    var result = x + y;

I'm going to read the code anyway to review a PR, a summary of what the code already says it does is redundant information to me.

Probably to understand what is going on there in the context of the full system instead of just reading letters and making sure there are no grammar mistakes.