> Other forms of engineering have to take into account the variability of the world.
> Maybe LLMs mark the point where we join our engineering peers in a world on non-determinism.
Those other forms of engineering have no choice due to the nature of what they are engineering.
Software engineers already have a way to introduce determinism into the systems they build! We’re going backwards!
Part of what got me into software was this: no matter how complex or impressive the operation, with enough time and determination, you could trace each step and learn how a tap on a joystick lead to the specific pixels on a screen changing.
There’s a beautiful invitation to learn and contribute baked into a world where each command is fully deterministic and spec-ed out.
Yes, there have always been poorly documented black boxes, but I thought the goal was to minimize those.
People don’t understand how much is going to be lost if that goal is abandoned.
Agreed. The beauty of programming is that you're creating a "mathematical artifact." You can always drill down and figure out exactly what is going on and what is going to happen with a given set of inputs. Now with things like concurrency that's not exactly true, but, I think the sentiment still holds.
The more practical question is though, does that matter? Maybe not.
> The more practical question is though, does that matter?
I think it matters quite a lot.
Specifically for knowledge preservation and education.
Yes, but : not in all cases really. There are plenty of throw away, one off, experimental, trial and error, low stakes, spammy, interactions with dumb-assery, manager replacing instances that are acceptable as being quickly created and forgotten black boxes of temporary barely working trash. (not that people will limit to the proper uses)
Yep, LLM’s still run on hardware with the same fundamental architecture we’ve had for years, sitting under a standard operating system. The software is written in the same languages we’ve been using for a long time. Unless there’s some seismic shift where all of these layers go away we’ll be maintaining these systems for a while.
In a way this is also a mathematical artifact — after all tokens are selected through beam searching or some random sampling of likely successor tokens.
"Computers are deterministic!"
If I wanted to plumb together badly documented black boxes, I'd have become an EE.
Underrated comment - computers can be decidedly non-deterministic.
https://en.wikipedia.org/wiki/Metastability_(electronics)
> People don’t understand how much is going to be lost if that goal is abandoned.
Ah but you see, imagine the shareholder value we can generate for a quarter or two in the meanwhile!
Editors note: please read that as absolutely dripping with disdain and sarcasm.
As pertaining to software development, I agree. I've been hearing accounting (online and from coworkers) of using LLMs to do deterministic stuff. And yet, instead of at least prompting once to "write a script to do X," they just keep prompting "do X" over and over again. Seems incredibly wasteful. It feels like there is this thought of "We are not making progress if we aren't getting the LLM to do everything. Having it write a script we can review and tweak is anti-progress." No one has said that outright, but it's a gut feeling (and it wouldn't surprise me if people have said this out loud).
I am one of these people. For my scripting needs, it would probably take me longer to find the script I saved rather than just asking it again and getting an answer in 15 seconds. I haven't saved a smallish script in a year.
I used to be one of those people, then I started saving these scripts in a folder and realized just how much time it saved me. Especially for nontrivial scripts which require a lot of typing or input specification.
This was me before LLMs. Had tons of scripts saved. I just don't see the point anymore "Take this list of users, remove from x group and reset their passwords" takes less time than trying to remember what I named that file and where it's saved. Anything that can be done in under 100 lines isn't worth saving anymore.
Honestly, one of the biggest gains for me with LLMs has been bash scripting. its so great.
especially if you have some saved in the same project and then you just tell it "look at the scripts in x and use that style"
Usually one would save this type of thing as a build tool task.
This is the 2025 equivalent of the people who once wrote 2000 word blog posts about how bad it was to use "cat" instead of just shell redirection.
These are hardly equivalent. One is someone preferring one deterministic way over another. The other is more akin to arguing that it's better to ask someone to manually complete a task for you instead of caching the instructions on your computer. Now if the LLM does caching then you have more of a point, I don't have enough experience there.
I would rather say it like this: Very good, very hardworking engineers spent years of their lives building the machine that raised us from the non-determinism of messy physical reality. The technology that brought us perfect, replicable, and reliable math from sending electrons through rocks has been deeply underappreciated in the "software revolution".
The engineers at TSMC, Intel, Global Foundries, Samsung, and others have done us an amazing service, and we are throwing all that hard work away.
'potatolicious says we're going forwards: https://news.ycombinator.com/item?id=44978319
That was an interesting take, but "probabilistic" is to me different from "random". In particular other field get error tolerances, LLMs give us nothing like that.
We're introducing chaos monkeys, not just variability.
Note that he's talking about the same nondeterminism in that post that we're talking about here.
From the linked comment
> Process engineers for example have to account for human error rates. [...] Designing systems to detect these errors (which are highly probabilistic!)
> Likewise even for regular mechanical engineers, there are probabilistic variances in manufacturing tolerances.
I read them as relatively confined, thus probabilistic. When a human pushes the wrong button, an elephant isn't raining from the sky. Same way tolerances are bounded.
When requesting a JSON array from an LLM, it could as well decide this time that JSON is a mythological Greek fighter.
I'm just saying, it's a cite to a thread about the implications of AI-based nondeterminism, just like this one. That's all.
adding to this, software deals with non-determinism all the time.
For example, web requests are non-deterministic. They depend, among other things, on the state of the network. They also depend on the load of the machine serving the request.
One way to think about this is: how easy is it for you to produce byte-for-byte deterministic builds of the software you're working on? If it's not trivial there's more non-determinism than is obvious.
Mostly the engineering part of software is dealing with non-determinism, by avoiding it or enforcing determinism. Take something like TCP, it's all about guaranteeing the determinism that either the message is sent and received or it is not. And we have a lot of algorithms that tries to guarantee consistency of information between the elements of a system.
But there is an underlying deterministic property in the TCP example. A message is either received within a timeout or not.
How can that be extralopated with LLMs? How does a system independently know that it's arrived at a correct answer within a timeout or not? Has the halting problem been solved?
You don't need to solve the halting problem in this situation, because you only need to accept a subset of valid, correct programs.
> How can that be extralopated with LLMs? How does a system independently know that it's arrived at a correct answer within a timeout or not?
That's the catch 22 with LLM. You're supposed to be both the asker and the verifier. Which in practice, it's not that great. LLMs will just find the snippets of code that matches somehow and just act on it (It's the "I'm feeling Lucky" button with extra steps)
In traditional programming, coding is a notation too more than anything. You supposed to have a solution before coding, but because of how the human brain works, it's more like a blackboard, aka an helper for thinking. You write what you think is correct, verify your assumptions, then store and forget about all of it when that's true. Once in a while, you revisit the design and make it more elegant (at least you hope you're allowed to).
LLM programming, when first started, was more about a direct english to finished code translation. Now, hope has scaled down and it's more about precise specs to diff proposal. Which frankly does not improve productivity as you can either have a generator that's faster and more precise (less costly too) or you will need to read the same amount of docs to verify everything as you would need to do to code the stuff in the first place (80% of the time spent coding).
So no determinism with LLMs. The input does not have any formal aspects, and the output is randomly determined. And the domain is very large. It is like trying to find a specific grain of sand on a beach while not fully sure it's there. I suspect most people are doing the equivalent of taking a handful of sand and saying that's what they wanted all along.
No? These kinds of analyses all seem to rely on the notion that the LLM-caller needs to accept whatever output the LLM provides. In practice, they discard all the outputs that don't compile, and then a further subset of the ones that don't --- those outputs that aren't instantly clear to the caller.
My intuition for the problem here is that people are fixated on the nondeterminism of the LLM itself, which is of limited importance to the actual problem domain of code generation. The LLM might spit out ancient Egyptian hieroglyphics! It's true! The LLM is completely nondeterministic. But nothing like that is ever going to get merged into `main`.
It's fine if you want to go on about how bad "vibe coding" is, with LLM-callers that don't bother to read LLM output, because they're not competent. But here we're assuming an otherwise competent developer. You can say the vibe coder is the more important phenomenon, but the viber doesn't implicate the halting problem.
Valid programs are almost infinite. Context free grammars (which describe valid programs) are generative. When you're programming, you are mostly restricting the set of valid program to include only the few that satisfy the specs. Adding an extra 0 to a number is valid, but put that in the context of money transactions, it's a "hell breaks loose" situation.
SO that's why "it compiles" is worthless in a business settings. Of course it should compile. That's the bare minimum of expectations. And even "it passes the tests" is not that great. That just means you have not mess things up. So review and quality (accountability for both) is paramount, so that the proper stuff get shipped (and fixed swiftly if there was a mistake).
I have LLMs generate Haskell. Having the code compile means everything type checks and is not worthless. That's a huge constraint on valid programs.
> Having the code compile means everything type checks and is not worthless. That's a huge constraint on valid programs.
In a business settings, what usually matter is getting something into prod and not have bug reports thrown back. And a maintainable code that is not riddled down with debts.
Compiled code is as basic as steering left and right for a F1 driver. Having the tests pass is like driving the car at slow speed and completing a lap with no other cars around. If you're struggling to do that, then you're still in the beginner phase and not a professional. The real deal is getting a change request from Product and and getting it to Production.
It feels like you stopped reading before "and then a further subset of those".
Again: my claim is simply that whatever else is going on, the halting problem doesn't enter into it, because the user in this scenario isn't obligated to prove arbitrary programs. Here, I can solve the halting problem right now: "only accept branchless programs with finite numbers of instructions". Where's my Field Medal? :)
It always feels like the "LLMs are nondeterministic" people are relying on the claim that it's impossible to tell whether an arbitrary program is branchless and finite. Obviously, no, that's not true.
> It feels like you stopped reading before "and then a further subset of those".
Pretty sure you've just edited to add that part.
No, I'd have indicated that in my comment if I had. Sorry, I think you just missed it.
I did add the last paragraph of the comment you just responded to (the one immediately above this) about 5 seconds after I submitted it, though. Doesn't change the thread.
> LLMs will just find the snippets of code that matches somehow
This suggests a huge gap in your understanding of LLMs if we are to take this literally.
> LLM programming, when first started, was more about a direct english to finished code translation
There is no direct english to finished code translation. A prompt like "write me a todo app" has infinitely many codes it maps to with different tradeoffs and which appeal to different people. Even if LLMs never made any coding mistakes, there is no function that maps a statement like that to specific pieces of code unless you're making completely arbitrary choices like the axiom of choice.
So we're left with the fact that we have to specify what we want. And at that LLMs do exceptionally well.
Right, you handle determinism by applying engineering. E.g. having fail safes, redundancies, creating robust processes, etc.
It’s not trivial largely because we didn’t bother to design deterministic builds because it didn’t seem to matter. There is not much about the actual problem that makes it difficult.
if you don't have deterministic builds then you can't tell whether the executable you're running comes from the source code you can see.
It is definitely non-trivial and large organizations spend money to try to make it happen.
Yes, I spent years working on exactly that. I personally was working with a compiler team to validate changes to eliminate non-determinism. That’s why I felt qualified to make the statement I did.
It’s non trivial because you have to back through decades of tools and fix all of them to remove non determinism because they weren’t designed with that in mind.
The hardest part was ensuring build environment uniformity but that’s a lot easier with docker and similar tooling.
It's uncertainty, not non-determinism. I don't design a rocket with no idea about the chain of events. I have inscrutable uncertainty everywhere, from manufacture to flight.
It was forward when Newton discovered the beautiful simple determinism of physics.
Was it going backwards when the probabilistic nature of quantum mechanics emerged?
Two words: many-world interpretation.
More seriously, this is not a fair comparison. Adding LLM output to your source code is not analogical to quantum physics; it is analogical to letting your 5 years old child transcribe the experimentally measured values without checking and accepting that many of them will be transcribed wrong.
The point is that determinism is an emergent phenomena and when you dig deeper it reduces to a ground which is nowhere near as stable. So as a heuristic, moving away from deterministic models isn’t necessarily the wrong direction.
Many worlds is, like you just said, an interpretation, and interestingly only one of many!
Hopefully even Newton already had some awareness of "deterministic chaos" (or whatever terms they would have used back then) ?
And on the side hand, no transistors without quantum mechanics.