I think you missed something...
> ... or there are underlying reasons for a language shift.
As to "best language" that is just as dogmatic as anything else... just look at the C/C++ vs. Rust divide in the Linux community.
I think you are overestimating the value of a best fit language for any given task, especially those where there are a half dozen popular languages that more people know well that can do the job good enough. Don't build for a sky scraper when all you need is a birdhouse.
Also, MOST engineers aren't particularly talented. If you're fortunate enough to be working for an organization where everyone is a rockstar, that's great... for those doing bog standard CRUD apps for business, you don't get rockstar money, and you aren't finding rockstar talent. You get what you get and make the pest of it.
In nearly three decades, I've once, only once worked a project where I didn't have to explain a relatively simple concept to someone, where everyone on the project delivers their pieces in time and all were talented. It was wonderful. Then new management gets stacked on top, all the job roles are reclassified to mid level developers and everyone rolls out of that group.
A lot of the actual experience is literally explaining public/private key usage to other developers who manage to (re)use the same keys from dev to all the production deployments. Or a pissing match with the "security expert" who doesn't understand that your app's use case is different than the in the box security script that is failing, because your /login route is a different app from / and the bogus query params don't matter.