You mean the English to code layer is stochastic while code to assembly is not?
This stochastic nature existed before AI. Humans are stochastic too. Two engineers convert specs to code in two different ways. I would argue the distribution is wider among humans than among agents.
As I said, "AI models don't replace compilers, they replace software developers" so yes, I agree that AI models are like humans.
while both LLMs and humans are not 100% deterministic, I don't think that's the interesting part. It's both true that there are many assembly language programs that implement a C program and that there are many valid programs (in any programming language) that correctly implement a natural language spec. But there's a significant qualitative difference between the two types of translation and that would be true regardless of whether the translations are stochastic.
The stement to which we are replying will (has?) launched a 1000 startups.
It would be interesting to state sane measurments that determine a "distance" between a spec and software that claims to implement it
I would argue the distribution is wider among humans than among agents.
Please go ahead and do.
> Two engineers convert specs to code in two different ways
So? Two compilers convert source to binary in two different ways.
The same compiler will convert source to binary in different ways! Hermetic builds are hard!
Even if we allow for hermetic builds in some cases in many many other cases your runtime will recompile your code based on its observations about your environment and that’s frequently what we want.
This fetishism of “determinism” seems crazy to anyone that has had to spend time really digging into modern JIT based compilation.
Arguing that two humans' distinct interpretations of a single specification is stochastic is patently absurd and requires, I think, some shred of evidence.
I would argue that, in the absence of such evidence, we cannot conclude that the English-to-code conversion process is stochastic; it is merely not deterministic, which is altogether different.
Would they write the same code, if given the same spec? No? How about over multiple trials? No? Even the individuals' own code would differ from one trial to the next?
Well, then, their interpretations of the spec are stochastic. Good enough for me, anyway.
That's not what "stochastic" means. You might want to look that up before you keep (mis)using it in conversations about actual stochastic processes.