This is profound, and I think you have dove to the core of why LLMs are taking off for code.
They fix a problem: the mutual incompatibility between software. "Free software" is not the answer, as it's used today: for example, the canonical Free Software OS, Linux, is fragmented into many mutually incompatible ABIs.
We need to tear down barriers between software. Barriers between languages, frameworks and OS ABIs. I don't have a good plan , but think it is the true solution. LLMs paper over these incompatibilities, and in doing so, provide value. Imperfect, but good enough until we destroy the barriers properly.
Example: Why am I using an LLM for my rust code? Because it is taking code that solves my problem, but in Python, C, or papers, and translating in a way compatible with the rust dialect I use. The problem is solved multiple times, but not in a way I can use without a translation layer. LLMs provide that.
Yeah I think you're spot on with the compatibility issues. Obviously YMMV depending on the languages and libraries in question, but yeah there's no one size fits all solution, and writing boilerplate code is the most boring thing in software development.
That's the big thing that's warmed up to programming languages with good macro systems in the past few years (and I absolutely LOVE Zig's comptime paradigm). At the same time, I've got serious questions about the programmers who dismiss macros as overkill for their needs, but then immediately pull out the LLM.
Macros are harder to learn.
But I've wondered the same thing. On some level, the coding superpowers allegedly wrought by LLMs seem reminiscent of proficient use of macros. But most programmers don't use them, and most programming languages don't support them.