> Programming has mainly been a career that requires the individual to understand English.

Disagree, programming is a career where in order to be good you can

1. Break down a big problem into smaller ones, creating abstractions

2. Implement those abstractions one by one to end up with a full program

3. Refactor those abstractions if requirements change or (better) reimplement an abstraction a different way

You do all of this to make complex software digestible by a human, so that they don't have to have the entire system 'in context'.

This prophesied view of software development will mean you end up with code that's likely only maintainable by the model itself.

I can't imagine the vendor lock in of that.... You have the source, but it is in such a state that no human can maintain it?

> I can't imagine the vendor lock in of that.... You have the source, but it is in such a state that no human can maintain it?

It’s much worse than that.

What happens when the erroneous output caused by model blind spots gets fed back into the model?

Those blind spots get reinforced.

Doesn’t matter how small that error rate is (and it’s not small). The errors will compound.

Vendor lock-in won’t matter because it will simply stop working/become totally unrecoverable.