Years ago, long before the current "AI"/LLM-craze, I've heard someone saying that in the future programming will have less fixed, deterministic algorithms and more statistical/ML algorithms, because even in the case when writing a deterministic algorithm is possible, it is sometimes easier to collect training data and train a tiny model, then to write and maintain large codebase that does the same thing, but deterministically.
I wonder, if mass adoption of LLM code-generation will accelerate this process or not. On the one hand, it is now easier to "write" (LLM-generate) code then ever, so time/effort savings are not there anymore (probably, I'm not sure). On the other hand, now everyone uses LLMs so, I think, people are less deterred by non-determinism and statistical nature of ML models.
Let's imagine that you are making a CLI tool. What does a CLI tool do? Well, it accepts reads arguments, reads stdin, writes stdout and make syscalls. So, all possible inputs and outputs are very well defined. What if in the future it will be easier (and maybe even more natural) to ask an LLM to "imagine" tons of possible inputs and correct outputs for a tool that you are making and then train a tiny model, without writing or generating any code?
"the future programming will have less fixed, deterministic algorithms and more statistical/ML algorithms"
I think the direction this will be going depends in part on how hardware prices will develop. If we continue with "hardware is cheap, don't think about it" like we did in the past decades I can see this happen. On the other hand, if we continue down the path we've recently taken, people who know when to choose which approach to their advantage will have a good and prosperous life.
> What if in the future it will be easier (and maybe even more natural) to ask an LLM to "imagine" tons of possible inputs and correct outputs for a tool that you are making and then train a tiny model, without writing or generating any code?
This seems to assume you're okay with whatever you're building being a black box that will break in the future and require you to re-train the model for every scenario that comes up. I can't think of a problem I've solved recently where that would pass the bar for me. Maybe for one-off problems like 'I have all this data and I want to classify it' where you could hand-classify say 5-10% of your dataset and train a model to deal with the rest of it?
yes, it's easier to build something that doesn't work. More news at 11.