>- systemic tech debt is now addressable at scale with LLMs. Future models will be good enough to sustain this, if people don’t believe this I would challenge them to explain why.
Is this some sort of troll attempt? Like, are you fundamentally misunderstanding the problem with tech debt? This is the equivalent of throwing garbage on the floor and expecting professional cleaners to keep your house clean.
You can produce tech debt faster than you can pay it back, that's the core aspect of tech debt. If tech debt was more expensive in the short term than not doing it, nobody would be doing it.
A labor saving device doesn't reduce or deal with tech debt since tech debt is a decision made independently of the competence of the developers. If you have a company with a tech debt culture, the labor saving device will just let you accumulate more tech debt until you reach the same level of burden per person.
>First consider if you understand what scaling laws are like chinchilla and how RL with verification works fundamentally
Honestly, this tells me that you basically understand nothing, not even chinchilla scaling laws and how RL works. Not only are you trying to brute force the problem, you're listing completely irrelevant factors to the problem at hand.
Chinchilla scaling laws are "ancient" by LLM standards. Everyone who designs a model architecture that is supposed to beat their competitors is pulling out every trick in the books and then come up with their own on top of that and chinchilla scaling laws have been done to death in that regard.
Reinforcement Learning is also a pretty bad example here, because there is no obvious way to encode a reward function to deal with something as ill defined as tech debt. You didn't even say avoid tech debt which would be actionable to some extent, just "systemic tech debt is now addressable at scale with LLMs". I.e. you're implying that if LLMs were to generate tech debt, you can just keep scaling and produce more of it, solving the problem once and for all Futurama style with ever bigger ice cubes.
- Not a troll.
Both of these lectures misunderstand my point and how things work.
- “tech debt” is not some special problem…? You accumulate cruft and bad design decisions…you spend tokens to fix this. Is your point there is always a fundamental tension between spending tokens on new stuff and spending tokens on cleaning stuff?
> Honestly, this tells me that you basically understand nothing, not even chinchilla scaling laws and how RL works. Not only are you trying to brute force the problem, you're listing completely irrelevant factors to the problem at hand.
That’s a very interesting take because I would say the same thing! RL and scaling laws are not relevant to the performance and capabilities of coding agents? Thats something you don’t hear everyday
- chinchilla-like scaling laws are not ancient…people try to derive scaling laws for new paradigms all the time it is how researchers get their company/lab to invest in scaling up a new idea. No idea what you mean here. Maybe you think I meant “the literal constants from the chinchilla paper”? No I mean: scaling laws generally, and Chinchilla, due to the impact of that work, is used more generally. Regardless, scaling laws generally continue to hold, and in fact improve with architectural/data mix/training recipes.
> Reinforcement Learning is also a pretty bad example here, because there is no obvious way to encode a reward function to deal with something as ill defined as tech debt.
Well that’s a bit of a strong claim to make… I don’t agree with this at face value but even if I did, you don’t need to explicitly do RL on tech debt as a specific task.. you do RL to build better programming skills generally which then generalize to many coding tasks.
> You didn't even say avoid tech debt which would be actionable to some extent, just "systemic tech debt is now addressable at scale with LLMs".
Tech debt is strategic, why avoid it?
> you're implying that if LLMs were to generate tech debt, you can just keep scaling and produce more of it, solving the problem once and for all Futurama style with ever bigger ice cubes.
I’m saying you can take, successively over time larger and larger, and more complex codebases with thorny debt problems and resolve them by spending money on tokens.
You keep scaling and, just like we do today, decide when some tech debt austerity needs to take place. I’m saying “the guy that built our house of cards over 10 years and left” is no longer so devastating and expensive a problem as it was before