With programs large enough tests aren't going to ever be enough. Formal verification might work, but then who checks the specification for bugs?
With programs large enough tests aren't going to ever be enough. Formal verification might work, but then who checks the specification for bugs?
I really wonder where all of these people who believe that tests perfectly encapsulate the behaviour of software come from. Maybe it's because LLMs happen to work better when you give them acceptance criteria and people struggle to distinguish between "better" and "good"?
The real test is years in production. Over time your test suite grows when bugs are found and fixed, but not every bugfix necessarily gets a test, and it's very rare that a bugfix is exhaustively tested. Relying on the test suite as a directional indicator that your vibecoded rewrite functions something like the original is probably sensible. But it isn't "done" until you've run it in production for at least as long as the original. And that's where it all falls apart, because maintenance will be a nightmare. Nobody knows how the new thing works.
In the case of rewrites, the specification is the original behavior, no? bugs and all.
Deciding if two programs do the same thing is provably impossible in the strict mathematical sense.
There is often no spec, just the old code, copied with the old bugs and with new ones sprinkled on top.