> so without LLMs you’re now doing only beginner stuff. You won’t seem like a real dev anymore.
I think it's the opposite. When I'm on the edges of Real Difficult Stuff (TM), then LLMs are less of help compared to when I just need basic web applications or similar "beginner stuff". If what you're working on doesn't exists in the training data at all, the LLMs can sometimes be more in the way than helpful, as they don't seem to even "grasp" the problems correctly.
I still get tons of productivity boosts from LLMs, don't get me wrong, and I wouldn't stop working with them even though they aren't perfect for everything, they do with a lot of monotonous tasks, like all the stuff you outlined. But the moments I'm working without LLMs, that's the time I'm working very much on "not beginner stuff".
This is my experience as well. It gets worse when the LLMs have strong priors in subject matter adjacent to your novel work. The generated output will incorrectly pull towards things the LLM has seen in it’s training data.
This is absolutely an issue, and I've run into it several times myself, because I am doing Real Difficult Stuff that I don't think anyone has ever done quite like this before. But I'm leaning heavily on Claude for it. Mostly for the stuff that has already been done before but I need to redo in order to be able to improve on it (mostly various aspects of graph layout algorithms of for the Sugiyama framework), but it's also been surprisingly helpful at helping me write some of the complex new stuff. Those parts require a lot more direct guidance and careful review, but it's still helping me keep a firm grasp on the whole thing. Which I also find surprising, because in the past I've experienced many times that I was losing grasp on a project because I let LLMs do too much. But this time it's the opposite.
The critical IP that isn’t in the weights yet, is often a few hundred to a few thousand lines of code. All the rest, surrounding code, is infrastructure that is indeed in the weights.
yea so many times i have asked my self if i am actually saving time on the actual hard stuff.
What is this real difficult stuff you speak of that only you can do and LLMs can not?
I don't have a concrete example, but it's probably the stuff that starts taking longer to describe in English than to code.
"Make me this thing there are a million examples of" vs. "make me this unique thing in this exact, particular way".
I don't think there's anything an LLM can't write given enough direction. (It's a one-sentence fuck-it one-off on one end of the spectrum, and it degenerately ends with you giving it exact code and prompting it to echo it back to you on the other.)
But the jobs of making basic apps and web pages are the new WordPress jobs in terms of pay, I'd agree.
[dead]
Write a decent comment, for one. It struggles with some of the firmware stuff I do, though that's more due to a lack of a proper automated feedback loop.
Yeah, it's generally the harness around the model that makes it useful.
Two recent examples I tried to get Sol/Opus to get help with, were they gave more misdirections than beinghelpful: Tracking down the root cause of a synchronization bug in a distributed platform using CRDTs, especially dumb around sequence CRDTs. Also had less fun using Sol/Opus for building and troubleshooting fractures and constraint networks in Houdini.
I'm sure these will be much better as soon as enough data is in the training set, as always. GPT models used to be horrible at electrical stuff as well, today Astra seems pretty OK at it. It changes as they keep extending what use cases are covered by training data and what they test for and so on.
Think we’re saying the same thing.
I very actively work using LLMs (no “agentic” stuff) and agree with you - they are only really worth tasking out tedium to.
But a lot of - say a DM feature - is tedium. Something simple like adding reactions, adding read receipts. It’s not hard at all. The LLM easily excels at the task and yet I didn’t have to do it. There are 1000s of those tasks. Handling file uploads, filling out a W9 and emailing it to the user, cron jobs for digest emails. All very easy problems most LLMs can do. There are just countless tasks if you’re building a real SaaS and I don’t want to do them all.
The big problems LLMs can’t do are only as such because there’s no way to put it into a simple, linear instruction. Once you can though, it’s now an easy task (whether or not you use an LLM).
But yeah we’re saying the same thing - let the LLM do the 1000s of tedious tasks you normally wouldn’t be bothered to spend time on.
Build the entire SaaS (which is mostly that tedium) instead of just the MVP thing you need without any niceties. I find it opens up more opportunity, like for example, we are about to hire our first full-time employee because we’re taking on so much more business.
> There are 1000s of those tasks.
Yep.
Plenty of products attacked this tedium from different angles, from Ruby and Rails to Supabase. And half of NPM. Heck, more than half.
LLMs are doing it a bit better, IMO. I don't have to rewrite the app in <whatever> or install <horrible dependency> to take advantage of it.
> The LLM easily excels
The agents are building all these small features, up to a full SaaS; and all the code is excellent?
How much time do you spend in code review?
[dead]
That's fine, but then the value of the tool evaporates. I agree you have to hold the LLM's hand so that it doesn't do crazy things, but that takes so much time that you don't actually save time over just writing it yourself. That is why I continue to maintain that LLMs are not a useful tool: either you completely give up on quality to get speed gains, or you maintain quality but get no speed gains.
You definitely save a ton of time over writing it yourself, just not as much time saved as “agentic” coding. I don’t do agentic coding either, I think it’s basically a scam. But using an LLM does mean creating stuff faster, much faster. The LLM can write 500 lines of code in seconds. That takes me hours.
I still design the APIs, the system, the tests. But the specific implementation of functions, I am freed from. There is a large speed gain there.
A calculator does no design or review, so what exactly is the llm being used for, no metaphors or riddles please.
because to me this:
"Something simple like adding reactions, adding read receipts ... The LLM easily excels at the task and yet I didn’t have to do it."
Implies to me code gen.
That aside, are you a 'DidntUseIt' sockpuppet, given there are lots of green accounts responding on their behalf..
The "agentic" stuff has come a long way recently, it's worth revisiting it every few months, it is better and better at taking on the tedium especially.
what business are you in?