In our team's experience, the product of agents is generally The Homer (1). It does work, but it's vastly overengineered.
When I personally want tight code, I have to spend a considerable amount of time adjusting it manually:
- It needs to be trimmed down. In my experience, at least one agent I use struggles to produce minimalist designs, and it's very frustrating
- I need to consider whether there are solutions based on higher-level assumptions, that AIs typically miss
- I need to check whether there are off-the-shelf solutions - AIs like to reinvent the wheel
IMO, software production has become a mass-produced commodity in every sense - it's much more expensive to produce software manually, but the quality is not the same.
It took me a few seconds of deliberating if The Homer was a reference to baseball or "The Odyssey" and then realized there was a footnote
> software production has become a mass-produced commodity in every sense - it's much more expensive to produce software manually, but the quality is not the same.
Agreed.
I believe “The Homer” is a reference to Season 2, Episode 15 of The Simpsons, “Oh Brother, Where Art Thou?”
Homer reunites with his long lost brother, who runs car company Powell Motors. Homer is ultimately tasked by his brother with helping to design a car for the “average man” that ultimately bankrupts the company for being wildly overengineered and costing too much ($82,000 in 1991-money).
Here’s the car: https://simpsons.fandom.com/wiki/The_Homer
If software has become a mass-produced commodity then seems to me the software business will become a much more finance focused business
You will really have to weigh the cost of making the software against the expected revenue.
Since software is still a winner-takes-all market, the mass-production property of software doesn't really matter.
In such markets, what you produce is either worth nothing or worth millions of dollars. For as long as it's the case that well-constructed code (with or without LLM help) is more likely to be in the latter category, the economics of software don't really change.
Even before LLMs, you could've commissioned a half-assed clone of any app you wanted from a 3rd world consultancy for a few thousand dollars. LLMs are basically Bangalore-as-API.
I think part of the reason software was winner take most was the difficulty of making software.
I remember hearing a story that in the past movies were so technically difficult to make that any movie that got made had a good chance to be a profitable hit. But as movies got cheaper to make, more movies got made. Nowadays movie studio execs have to really calculate out the audience and expected revenue for any new movie and balance that against the budget and the cost of the studio's failed movies.
I think a similar dynamic may happen in software
> I think part of the reason software was winner take most was the difficulty of making software.
That might be part of it, but I think it also has to do with the reality of replicating and scaling. Hardware or physical goods simply don’t scale like digital goods. There can be hundreds of knock-off physical products that have lower quality and lower cost but serve 90% of the same purpose, because physical capacity for raw materials, construction, labor, shipping, etc. have scaling limits in each market and economy. Digital goods are just so much easier to replicate and scale, so it often doesn’t make sense to buy software at lower quality and lower price if it doesn’t do most of the job. There are still limits of course, and different from physical goods, but I think this is a key reason why software is seen as winner-take-all.
Yes, but there are other digital goods, like music, movies and books which are not quite as hard to make as software. In those you have a hit driven market dynamic with lots of niches instead of winner take all.
I.e. because software was hard to make and complex to copy you would tend to have "natural monopolies" that were hard to compete with. Who wants to try to build a new desktop OS to compete with Windows? Or a web browser from scratch? Or a new search engine? Etc.
Those and other pieces of software were complex and hard to make. The cost to copy and compete was very high. So one winner took most because that winner was the company who could figure that software out.
But as we can see with Kimi Work and other such things, software is now much easier to copy. Let's say it took $1 billion to make a copycat piece of software with people but now takes $100 million or $10 million with AI. Suddenly a copy and compete tactic makes much more sense than before
For example, with AI it might make financial sense to build a Chinese Native OS instead of Windows. Similarly for Russia, Iran, the EU, and a whole bunch of other places. All of a sudden, Windows might not be the winner take most OS, we might have lots of Operating Systems, with smaller markets and lower profits, which require much more careful financial analysis to stay profitable.
This would be just like Movies, TV Shows, Books or Music. When something works, people relentlessly copy it and different regions put their own spin on the idea. After Iron Man succeeded we had so many super hero movies. Etc. So there is not really a winner take most dynamic in these other digital products. Software may be moving that way
That honestly sounds a lot healthier than “just ship what the CTO/Product team wants” with as much hand waving as is necessary to very roughly estimate ROI and then pray it hits with the market. In anything that’s not a startup operating in a new industry, the “old way” is a hard way to run a business
> Since software is still a winner-takes-all market, the mass-production property of software doesn't really matter.
Fairly sure software dev was always an iceberg. Most software and most software devs aren't working on horizontal software, but on vertical software, in cost centers. Sales for that kind of software don't scale as much.
Most vertical software is just horizontal software glued together
No surprise, LLM companies optimize for waste. More tokens, and more prompts means more revenue. Reminds of Google’s Prabhakar Raghavan story: deliberately making search worse [1]
[1]: https://pluralistic.net/2024/04/24/naming-names/#prabhakar-r...
Or, more likely, it's that concise code requires a much deeper, wholistic, understanding that these models just are capable of yet.
Same with a junior dev. They don't write long form spaghetti because they're trying to write more LOC. They do it because not doing it is hard, literally above their pay grade.
I use LLM every day, but they're still completely awful at architecture. I don't think this clear lack of ability is some conspiracy.
Personal anecdote: I spent a few days hacking on my compiler to remove 1k lines of code (about 15% of total code) while preserving behavior
I was only able to do that after I had solved multiple related problems in different places and started introducing subtle bugs by accident / had difficulty detecting all edge cases
I've noticed whenever I use LLMs they introduce the same kind of thing but at much smaller scales than I would. They often suggest solving the wrong problem when I prompt them to diagnose specific bugs too. Usually opting for a shortcut that introduces its own issues and ironically calling the proper direction "too complex" when it's really not.
As a business user, the same thing is true for non-code documents. The biggest exertion is reducing the excessive slop down to concise, clear points.
> software production has become a mass-produced commodity
For who?
The public? The public has never liked buying software at any price.
Businesses? Businesses need higher quality software when it's relevant to their core competencies, so they hire people instead. Buying competing SaaS or depending too much on AI is throwing the baby out with the bathwater.
This was more true a few months ago but Fable has improved the situation considerably.
Also just remember - minimalist code looks and feels great but customers do not read your code. I have caught myself many times providing "corrections" to abstractions that were already ~fine, just not perfect. The average SWE costs $200/hr. Careful you don't burn $50 worrying about code that will likely be rewritten or can be better abstracted when that's actually needed.
> The average SWE costs $200/hr
This is a pointless quibble but the hourly rate claim is not true--it's like ~$60 in the USA [0]. Maybe you meant at a specific Org but this is important context when comparing "pricing" between human and AI.
[0] https://www.salaryexpert.com/salary/job/software-developer/u...
How is this not true? Taking a Senior SWE @ ~$200K, even just the base salary cost / 2080 working hours is $100/hr. Fully loaded employer cost + accounting for non-coding time gets you to upper 100s easily.
Even for a junior making $100K, I have a hard time believe their time is worth less than $75/hr or so.
Edit: Fine, "Senior" is not "Average". But naive salary is not the true numerator.
It is. And the quality is on par with any us eng. People here forget that the big comp packages are a minority even in the US. The cost tho is much higher than just salary.
Western Europe is mostly consultancy, and the rate paid by client is usually higher, and doesn't matter if it's eastern Europe, Portugal or even India.
Company time != Pay rate, if you're working somewhere that's publicly traded check out "revenue per employee" metrics sometime.
I hire contractors for a large enterprise in the US. The going rate is typically $85-$100/hr for a senior dev, depending on specialization. Lead-level maybe $120 for the right skill set.
Of course, the SWEs making that much (over 200k) are not representative of the broader field. That's the point.
Pay hits a ceiling, and that ceiling is moving lower regardless of experience. That has nothing to do with AI, but what the market will bear. Hiring counts of humans must increase no matter what. Moving some of the spend to AI reduces the risk of hiring less qualified employees they might have rejected a decade ago.
Wages at the top end are stagnating to subsidize this. That's undeniable.
$15, where we're going.
Federal minimum wage is $7.25 per hour
An MBA's rule of thumb is that a full time employee's hourly cost to a business is at least 1.5x to 2x times their salary depending on employer taxes, benefits, offices, travel, training, hardware, perks, etc.
Minimalist code is necessary to keep AI agents working well for longer than a month on a system IME. At a certain point, their own machinations overwhelm them and they both slow down, and make worse and worse decisions.
[dead]
Also, you can probably think about it like this:
"Will I benefit from this code being minimalist before [date]", where [date] is whenever you think the agent will be good enough to come back and make the corrections you would make today.
I'd caution that some corrections become harder to make over time, rather than easier. A bad architecture now can become much harder to fix once other things have grown up around it.
Even as a self-contained unit, you can't step in the same river twice, and on [date] some important details may have seriously faded, both in terms of text that can be mined and also in terms of human "why did we do that" and "what was the reason we did it this way and not that way" etc.
> The average SWE costs $200/hr.
And this is how I find out I'm woefully underpaid.
That "cost" includes all the overhead provided by the company: benefits, rent for offices, utilities, equipment, etc. The average SWE is not taking home anything close to that, outside of Silicon Valley and a few other limited areas.
Whatever you are making this year as SWE you'll be making less next year if the current trend in improvement of AI coding aids is going to be sustained. Think about it : programmers used to derive a lot of their value from the fact that it was a hard skill to acquire. My kids can now 'vibe code' stuff faster (and better looking) than what I could come up with as the beginnings of a design plan. And then I still need to implement it.
There's a massive difference between your kids vibe coding something and an engineer using AI to implement something. If you're unable to discern the difference, that's something to reflect on :)
It doesn't matter if GP is able to discern the difference, it matters if your CEO is forced to care about the difference.
CEOs will always want someone who knows to implement so we're safe from kids vibe coding their way in but in a short while it becomes AI who knows who is managing less expensive AI.
The average SWE makes $400k a year? Are you being serious?
Costs, not makes. That includes employer taxes, benefits, offices, travel, training, hardware, any perks.