To clarify, I didn't mean we should be using ancient models in production, I meant in R&D.
Anthropic says "do the simplest thing that works." If it works with the LLMs we had 3 years ago, doesn't that make it simpler?
The newer LLMs mostly seem to work around the poor system design. (Like spawning 50 subagents on a grep-spree because you forgot to tell it where anything is...) But then you get poor design in prod!
As an addendum... The base/text models which have fallen out of style, are also extremely worth learning and working with. Davinci is still online, I believe, although it is deprecated.
Another lost skill! Learning how things were done before instruct tuning forces you to structure things in such a way so the model can't do it wrong. Half a page of well crafted examples can beat 3 pages of confusing rules!
(They're also magical and amazing at writing, although they produce bizarre and horrifying output sometimes.)