The study compares ChatGPT use, search engine use, and no tool use.

The issues with moving from ASM to C/Fortran are different from using LLMs.

LLMs are automation, and general purpose automation at that. The Ironies of Automation came out in the 1980s, and we’ve known there are issues. Like Vigilance decrement that comes when you switch from operating a system to monitoring a system for rare errors.

On top of that, previous systems were largely deterministic, you didn’t have to worry that the instrumentation was going to invent new numbers on the dial.

So now automation will go from flight decks and assembly lines, to mom and pop stores. Regular to non-deterministic.