But there's a new skill blossoming there, right? That's what I'm noticing. The atrophying skill is one you don't need anymore. The new one - prompting, routing the output, spotting the hallucinations, managing singal/noise etc. - is now where you can add tremendous value.
To say nothing of being bold enough to run systems like that in the first place. I know some of engineers that are still folding their arms and refusing to engage with AI!
>spotting the hallucinations
SRE here, spotting the hallucinations requires you to have the knowledge to know when it's wrong. Best way to gain knowledge on systems is actually working on them. If AI do all the work, then you don't get the knowledge.
Exactly. How hard is it to understand for people?
If you don't ride your bicycle for years and years because you can just write "drive yourself"
Then your skills are atrophying. Typing isn't a skill
I don't know that that analogy fits for this comment. The argument is that the old skill is genuinely no longer needed in favour of the new skill, but the new skill requires experience from the old skill. I don't think being a good driver needs you to have already been a good cyclist.
> I don't think being a good driver needs you to have already been a good cyclist.
It's an established fact by now that if you know coding then you can get better output from Sonnet than vibe coders can from Opus
I think it basically stems from the idea of “conceptual learning”. Which has been a pedagogical ideology for quite some time now. And is also convenient if you’re a bit addicted to your phone, a bit lazy due to AI, and want to still feel good about “learning” things.
We’ve known that one important component for learning (not the only thing that matters, but an important one) is the memorization of facts.
There seems to be a big urge to pretend that’s not true. E.g. you can learn arithmetic without memorizing your multiplication tables, literature without engaging with the canon, and now apparently coding without even having coded.
The idea is if we start with a top-down conceptual approach we will eventually vibe our way into knowledge acquisition. Which does sort of work.
However, sorry to be a curmudgeon, but the people who just “do the work” and actually spend time on task to learn a bunch of stuff … are actually more knowledgeable on average.
> you can learn arithmetic without memorizing your multiplication tables
How? If you don't learn multiplication tables you're multiplying on the go. And that's inefficient
Also, this isn't the right analogy. I get where you're coming from.
We move on to the higher form. Like we don't need to code in binary, or assembly language We got C, Python. We then got compilers.
But LLMs are just english. There is no skill here in writing the prompt. "Give me a crud app that does xyz" is hardly a skill
Tomorrow if AI data centers fail then what do we do of the typing skills?
When we shifted from Assembly to Compiled languages or to Python. We still had to learn something else
Here, using AI, its just English.
How do technical managers solve this problem? I've never been a real manager.
Good ones force themselves to do technical work hands on regularly, but will generally reach a level where they can't actually do that anymore unless they are really good at building management systems and delegating. For me it has happened in cycles over the last 25 years; after 4-6 years in management I refresh skills and go back to an IC role because I can't stand it anymore. I've done that cycle three times but I won't do it quite the same way again because its probably impossible to do without a pay cut now, and because IC don't write code anymore anyway.
This is both like forcing all IC to develop management skills but its also like the shift to cloud infrastructure or 4GL languages - we are never going back to the old ways and will be building everything at a different level of abstraction with a different span of control.
The good ones still make time to knock out some lifelong backlog resident tickets to sharpen their axe on.
The bad ones get promoted for spending the time instead playing politics.
I guess humans don't hallucinate that much
Humans hallucinate much more, but they're not that good at making the hallucinations convincing.
No they don't. Any human who hallucinated as much as an LLM does would be committed.
It's interesting how machines now mirror our relationship to sanity.
Just under the surface of human consciousness, we are constantly hallucinating and predicting what we may occur next. We are good at discarding what doesn't align with our working model of the world before reaching awareness and taking action. It's our attempt to conceptualize reality. Considering how un-grounded from reality humans are becoming, the fact we have to balance our nervous systems, the mistakes we do make day-to-day, and how minor physiological changes (from poison to injury to drugs to random chance) can disrupt the process, it doesn't take much for humans to make a major mistake or enter a psychotic state.
It wouldn't surprise me if a systematic approach to solve these problems with LLMs could be applied to human/hybrid teams.
I don't think that LLMs mirror human cognition or "sanity" as much as people want to believe. We are not constantly hallucinating in any way that applies to the term as commonly understood relative to LLMs or to humans. Mistakes, minor psychological changes and emotions aren't equivalent to LLM hallucinations or human psychosis. The relationship is superficial due to both humans and LLMs using natural language. But humans understand that language and its context, whereas LLMs do not.
The narrative exists to justify deskilling and replacing human expertise with LLMs on the premise that LLMs are at worse human equivalent and at best human superior but the hyperbole employed to make the false equivalence sometimes borders on the absurd and occult.
>It wouldn't surprise me if a systematic approach to solve these problems with LLMs could be applied to human/hybrid teams.
What you're describing aren't "problems" that need to be "solved" in human beings. They are problems when they persist in software, but we already had software that didn't have these problems.
This is a state of being that we chose because we decided profit superseded all other concerns.
> I don't think that LLMs mirror human cognition or "sanity" as much as people want to believe.
I don't "want" to believe it given the implications, nor support a profit narrative. That incorrectly assigns a frame of thought that has nothing to do with me and actually opposite to my beliefs.
We do know that a single chemical compound can fundamentally alter our perception of reality. Drugs don't contain hallucinations, they reveal our subconscious brain processes by affecting our awareness and disrupting whatever ends up in our cognition. We are constantly trying to predict what will happen next and compare outcomes to expectations, where errors and confabulations occur constantly. False memories and unreliable eyewitness accounts are a matter of fact. Optical illusions are persistent and repeatable. We're often confidently wrong when recalling memory, deducing from logic, transcribing etc., whether it's for milliseconds or decades. We also dream each night which is technically not "hallucination", but clearly a necessary reality-adjacent process.
If you break the problem down and question our ingrained societal human exceptionalism, it's clear humans actually have no real unshakable "understanding" of anything, we simply have more sensory information and reality testing in real-time to maintain a state of reality alignment. This can be affected at any time by physiological changes from illness to brain injury. Perhaps we currently maintain better context overall (reducing the volatility and frequency of errors), but no animal would exist for long and succeed if it didn't naturally do this one task exceptionally well.
Given the pace of technology and lack of evidence to prove digital machines can't outperform us at this task too, long-term it seems like a matter of time until LLM + ? = human ability superseded. At which point we may have an entirely new tool for neuroscience research, before we even have an agreed definition for consciousness.
This was all evident before LLMs reached the stage they're at now, and progress will continue whether or not companies can capture profit in a hype-bubble. The argument otherwise would be to prove that humans have a deterministic-probabilistic relationship that differs from the technology. I don't have answers but we certainly can't close these open questions when the literature and evidence points more towards similarities.
Not even that Proton CEO claimed that. This is another one of those "no u" knee-jerk reactions littered all over this site, and I wonder when we'll first see stuff like "but humans also sometimes fail to load because of network issues", because there is no real distance between "humans hallucinate even more" and something like that.
> "but humans also sometimes fail to load because of network issues"
They do. It's called "stuck in traffic."
Humans don't literally hallucinate much, but we do lie, or are wrong.
Humans are deterministic and have personality traits. The first couple of times getting bs’d and that person is now a known quality.
Agents fail randomly. One day a stellar dev and next a bs artist.
So this idea that LLM hallucination is not unlike human failings is without merit.
> Humans are deterministic
what
A person has what is known as 'character' and 'personality'. These words indicate discenable patterns of behavior. Determinism in this discussion means that a given person will 'behave in an expected manner'. Further feature of humans in that you can coerce them to behave in highly constrained manners.
Entire institutions and organizations in humanity are based on this feature of human beings.
A heuristic software agent that will respond in a spectrum of possible ways and past performance is no guarantee of future performance. It is no way reassuring that your agent has 'never done x' since if 'x is in the space of possible responses' it can certainly do 'x'.
> Determinism in this discussion means that a given person will 'behave in an expected manner'.
Oh I see. That is not what determinism means. Determinism means their actions are predestined, "determined" by the initial conditions. Computer programs - including AI - are deterministic, barring cosmic ray flips. The same inputs running in the same environment produces the same output.
Humans are or are not, depending on whether free will exists.
Either way, it's hard to draw the boundary around determinism such that it includes humans but excludes AI.
No, determinism in this context clearly means precisely being 'predictable' as claimed that individual humans are predictable given sufficient observation.
It's 1930 again
Every new model and you might as well chuck 90% of that knowledge in the bin.
Last Claude major version they released a blog post basically telling you to chuck away your Claude.md file and start again.
It was only 18 months ago when people were telling you to use ultrathink:
https://news.ycombinator.com/item?id=43739997
And then that went away about a year ago, and now it’s sort of back again but not quite as it was.
I really don’t want to see code generated by Claude without some constraint like rules in a Claude.md file. Even Fable on high is lazy enough to forget a ton of things that a senior engineer would not. For example tracing and handling all unhappy paths, not repeating code, not writing terrible doc strings, etc.
We are using quite a big list of rules and the output is significantly more aligned with what we interpret as “good” code.
[flagged]
I don't know how to spot hallucinations, manage signal/noise, without having a pretty good understanding of the problem space. It's a tough balancing act.
No, everything you listed is low effort and minimal friction. It's exactly the luddite's greatest fear: to be delegated to a button pusher, to ensure that blinking green light continue to blink green. Anyone that can read and write can now add "tremendous value," so what do you think makes the value "tremendous"?
A company comprised entirely of management. I'm sure only great things will come of this.
> The atrophying skill is one you don't need anymore.
of course. who needs to walk when we have cars
This is always a good example
People cannot deny that cars have made our physical bodies atrophy. Not all of us, maybe, but many people are way worse off physically than they would have been without cars
There's an analogy in there but I can't think of it. Maybe I'll ask Claude if it can explain it