> Of course both are useful, but I’d rather have familiarity with the codebase than a deep general understanding of software systems.
In my experience, getting that familiarity with a particular codebase in a way that isn't surface-level has always been a hands-on process. E.g. just because I know many general things about software, I need to know the particulars of the current codebase I'm in to know what is reasonable to actually apply to it.
This is a chicken and egg problem I find hard to resolve with LLMs. If we're pushed to delegate most work to them, how do you build that expertise? Sure you can ask questions about the codebase, but IMHO that falls under surface-level information, and the devil is often in the deeper details. Hmm.
We are having trouble onboarding engineers with AI now. Some still struggle after their first year with very basic concepts/patterns we use and make the same mistakes again and again in their PRs because they just ask Claude to fix it and never internalize it. I think using LLMs feels good at first because you can get tickets out the door faster but you never develop enough knowledge to make a big impact or become an expert in the code or business.
These would have been good hires before AI but now they struggle?
I imagine they are still good hires - however there is a real cognitive debt that happens when we code with LLMs in my experience.
It’s harder to internalise concepts because you don’t go through the struggle of understanding them and finding the mental links you need to remember later.
I notice this with people around me - all of them are doing more things, but I am also catching more issues when reviewing docs and code.
Obviously YMMV.
My guess is they would take longer to ramp up but then have a better understanding of the codebase and domain.
But there’s a lot of pressure to get up to speed as a new hire and it’s easy to move fast with AI
Honestly you should probably invest in baking the basic patterns and concepts of your codebase into shared skills that are applied to everyone's Claude sessions.
Or documentation that everyone could read and refer to
It's not for everyone, but I write all the code and then I ask the LLM for a review. It catches something minor thing every single time (out-of-date comments on a function or method, typo, etc.), something that's clear a mistake maybe a third of the time and, occasionally, points out something that looked fine to me but actually won't work because of some other code that I forgot about. I'm a person, I can't remember everything I guess!
I have to say on those occasional times where it finds something that I totally missed or misunderstood, those are for sure the most productive sessions. I find I'm actually working with the model, while I read the code it's pointing me at, and getting a good solution together. Often the model suggest something that's maybe too simple or, weirdly way way too complicated but it's definitely helps me zero in on a decent solution.
I think this is only a partial answer but I've been surprised by how familiar dev leads become with the app even if they are not in the code.
They tend to stick around and they engage in the problem solving on a higher level and develop a detailed picture of how the app does and should behave. So at least that part of the expertise may come from working with an LLM to solve problems.
They also review a huge amount of the code changes and rubber duck/plan/troubleshoot a lot
In the Linux kernel I've had a lot of luck with:
1. "find the code that does X"
2. go read that code
3. When you hit a bit you don't care about, go back to the model and ask it for the pertinent details
4. When you hit a really confusing bit, ask the model for hypotheses about what's going on. (I always phrase it as "give me some hypotheses" not "what is going on here". I dunno if this changes the output but I think it helps me stay in a mindset of uncertainty, it's important to avoid locking in any misunderstandings. Anyway I find the models do well at this task, and when they bullshit here it has a strong smell).
Before AI, parts 1 and 3 could be insanely time consuming, sometimes it felt like a infinite breadth-first-search. And part 4 was basically: either you find a human who knows the code, or you just make a mental note and hope that later on you find something that makes you go "oh, THAT'S why they <do weird thing that should 100% have a comment>!".
So yeah even though you're still reading code with your wetware the AI makes you dramatically more powerful.
This is also extremely helpful for unpicking undocumented API contracts. E.g. you can say "the x86 implementation of this API is safe to call under a spinlock, go read the other arch versions and tell me if that's true there too".
I’ve found that planning tickets with granular details (like semi specific code changes needed) is one of the best ways to get that deep knowledge of the system. Even if ultimately I delegate most of the implementation to an LLM. I also heavily verify the changes, but I find that that’s less impactful than planning a feature / tickets.
> If we're pushed to delegate most work to them, how do you build that expertise?
Have you ever pair programmed with someone? It's the same idea. You can be an active enough participant in the process if you wish to be and can be just as knowledgeable even if some of that knowledge lies in transactive memory. https://en.wikipedia.org/wiki/Transactive_memory As long as you have the map, and the map to the map, you don't need to retain every fact about the landscape.
the devil is always in the details. those details are on every level you look at: human minds, nature around us, space. so if your inputs are vague, you should only expect outputs that are vague and generalized
Read the code.
What if the code sucks, because it was vibe coded by an LLM over a dozen disparate sessions?
If it performs the overall function but the code is a mess, then you can ask the model to describe parts of it, the structures and what the structures are for. When it shows you something that is messed up, suggest a better framing for the same thing and ask the model to implement the better framing without regressing on already passing tests.
I'm going through this right now on a very difficult to implement task, the original was python and very verbose. But had facilitated a rust implementation that produced byte identical outputs. Then I asked it for what data was being passed around, placing restrictions on what passed between interfaces I could tell it what parts should be immutable and what parts should have no presence outside it's local context. Placing those limitations while having a exemplar of what it should be doing gives it little choice but to make better code if it meets the conditions set while at the same time not regressing.
I say things like 'this field is a implementation detail in a declarative data structure, it should not exist here.
This can mean hours of work with no observable change in program function, yet it is directly addressing the limitations that prevents it from being used in larger tasks.
This sounds like coding manually would take less time.
I think you would be incorrect. While AI could not write what what I have so far on its own. Neither could I.
But coding manually would use fewer tokens.
Then that's the point. You know the code sucks, the guy who vibecoded it either didn't know or didn't care. That's the added value.
But how would you know? If you're an inexperienced developer you may well end up ingraining bad patterns without even realizing it.
Yes, how does the LLM make a difference there? Inexperienced developers write bad code. Hell, experienced developers who are outside their domain write bad code too, then you later realize that it's hard to read, or that it leads to architectural problems or such, and hopefully learn from the experience.
In the context of what I was replying to:
> This is a chicken and egg problem I find hard to resolve with LLMs. If we're pushed to delegate most work to them, how do you build that expertise?
Learning from LLM written code is significantly and meaningfully different from struggling on your own or learning from more experienced human co-contributors. Especially in a large, complex, iteratively developed codebase.
Worse, once you're in that situation you are now at the juxtaposition of: "I did this. I understand what my reasoning was, and now I know why it is wrong and how to fix it" vs. "An LLM did this, I don't know why it did this, I'm not sure what it was trying to do or what pattern it followed and I'm not sure how to make it better because I didn't write or understand the original implementation either"
Is it impossible to learn and gain experience this way? Not at all. But it's definitely not equivalent.
Claude, make this codebase less ass
That will actually work pretty well.
I legitimately caught Claude calling things "ass" while a friend was using it earlier, which I think is pretty funny.
Then fix it using the llm
Reading it is good, but working with it more directly seems to help retention.