It's becoming increasingly clear to me that - at least right now - AI is only useful for 2 things:
1. Coding, with it being more useful the better you are at coding without AI
2. Any expert in their field asking questions about their field, who bother to fact check the output. E.g. "claude pls search these 1000 files and tell me if you find anywhere that they're discussing the settlement" and then the user checks the files/line numbers to make sure that it's correct - basically a turbocharged search that may have false negatives (content existed but I didn't find it) or false positives (content that I classified in a certain way but it was wrong). It takes an expert to tell the latter one in some cases.
I haven't found it that useful for doing any actual "agentic" coding at $DAYJOB with lots of legacy code it wasn't trained on (because proprietary). I do find it useful for summarizing sections of code that I am working on and asking for snippets that do very specific things. Also, it is pretty good at writing one-off short scripts with easily definable inputs and outputs.
I have come to the conclusion that people using AI for coding need to think about it as basically an automated version of the Docs -> Copy Paste -> Stack Overflow -> Copy Paste -> Compile Error -> Google -> Copy Paste -> New feature request from management -> Random internet blog -> Copy Paste loop that most of us do for a lot of the non-logic heavy portions (e.g. API interfacing) of our work with less randomness and more pattern matching or statistics or whatever guiding the process. Honestly, pretty useful, not knocking it.
I do think there is a killer application for AI, which it is already useful for, and that the industry doesn't really promote. That is basically taking a massive amount of unstructured data on a topic and allowing people an easy way to learn from that data without having to read through all of it (which may not even be possible for a single person in their lifetime). This would be a huge boon to humanity alone given the scale of data we produce. I think fundamentally, LLMs cannot take the data that they are so good at summarizing and use it in a creative way, it looks kinda like they can, because they are so good at "borrowing" other people's creative work, but in real-world scenarios where change is constant and the external forces of today are not understood by a model that was trained 3 months ago, they fall on their faces again and again.
I think AI companies know this but cannot admit that this ground breaking (I would argue) technology might only be transformative for one half of the observe->act workflow that would be necessary to replace humans as workers because.
1. It is possible the economics don't work out without replacing workers 2. If they admitted that the only value of their tech was in distilling value already present in other people's creative work, work that the LLMs cannot create on their own, a sane government might force them to pay for their inputs.