Four years ago LLMs were sometimes amazing, sometimes wrong, sometimes a huge time sink when it's almost there and you try to herd the tokens but it's like herding cats.
Just today I had the exact same experience. Every single testimonial is the same as I described above, just emphasizing a different bit to defend or attack LLMs or to make a case for nuance.
The two differences have been: (1) the 1.5 trillion dollar data center build out (2) everyone and their cats now has an opinion on "AI" and data centers. Software is not super amazing, nor are new useful features coming out super fast - It's about the same as 4 years ago plus 4 years of average long term progress as we've seen since 1990s,
They will always be wrong sometimes, but it’s becoming less wrong and wrong less often. Everyone will have their own opinion on “good enough”, but if you expect perfection you are bound for disappointment. No need for that when random chance and Murphy’s law will bring enough anyway.
May be getting harder to catch the mistakes but that makes them worse in my view. I'd much rather they make easy to spot mistakes because I don't expect factuality from them anyway just speedy transformation of information I already have available.
In fact it's the lossiest transformation tool I've ever used and it's still useful despite that. If it reaches one nine of reliability that would be huge but given the pace of growth in investment a first nine would cost an absurd amount of money, and the second and third nine would cost about the Earth's GDP
Or, the hard to catch mistakes were always there, and now we focus on them instead of the obvious ones that have been eliminated.
I still see obvious mistakes, so not eliminated.
The number of mistakes are also about the same.
But more mistakes are harder to catch. The output is more polished and convoluted and that make errors harder to catch. I don't want that.