In regulated industries none of those matter if the tool invents compliance issues or breaks compliance.
The only thought-ptovoking discussion should be "why the hell do we have this stochastic parrot anywhere near out codebase"
In regulated industries none of those matter if the tool invents compliance issues or breaks compliance.
The only thought-ptovoking discussion should be "why the hell do we have this stochastic parrot anywhere near out codebase"
I think that what technical people fail to understand is that a lot of the time, "compliance" is not the same as a binary compiles/does not compile. For a lot of rules/regulations, compliance means "making enough effort that legal is willing to back you up".
A system which will just randomly decide to give the legal team reasons to not back you up is:
* A system whose output will get brought up in lawsuits and make legal's job harder.
* A system that will make the dev team perpetually chase its tail while it oscillates between the several different valid interpretations of the rules.
Odd take. So if it identified 17 real gaps and helped fix them, the fact it was wrong about one gap, and the appropriate humans caught it and no harm was done, the whole thing is useless?
Not saying that is the situation, I don’t know. But if “one error is too many” is your point of view… do you think the humans in these orgs are 100% perfect 100% of the time?
> So if it identified 17 real gaps and helped fix them, the fact it was wrong about one gap, and the appropriate humans caught it and no harm was done
How many gaps have humans not caught?
> But if “one error is too many” is your point of view
Yes, in regulated industries "one error is too many" is the only right approach.
Yes, humans also make errors, and there you have a range of options: from tracing and finding the causes of the error (and tightening processes) to literally jailing those responsible. Your hallucination machine will happily "identify" 17 gaps, and create 34 more. And no, there are no processes to make it better. The "make no mistakes" incantation will happily be ignored for obvious reasons, regardless of how many forms of it you throw at it.