I got a degree in statistics, and what I remember from those times is that we spent maybe 10% of our time learning how to build the model and 90% of the time correctly interpreting the model output. Of course, in the time I learned statistics it was almost exclusively applied in hard science, finance, and actuary. The wild west of machine learning hadn't gotten started. I wonder if they teach it differently now.
When we start using big datasets where n went into the tens of millions we kinda stopped bothering because margins were so small it was effectively certain you found the mean of some range. However I think in that time we forget what confidence scoring actually is and how quickly multiplied alpha can balloon on you.
From my time building ML models (2016-2023) evaluation and interpretation was a large chunk of it. I often used "brute force" interpretation methods but there were others like SHAP [1] and LIME [2]. The other large chunk of ML work focused on feature engineering and data prep. Today in the LLM agentics era this is out the window. There are "evals" but no one does it seriously, and interpretation is often propagated to the consumer.
[1] https://shap.readthedocs.io/en/latest/
[2] https://christophm.github.io/interpretable-ml-book/lime.html
I am happy to hear high quality evaluations are out of fashion. That will make the competition so much less hard in the EU where evaluation is a regulatory requirement.
On the other hand, it’s kind of expected that the hard part of the job in AI/ML engineering gets neglected over that part people can brag on social media about. Deeply concerning but sadly understandable.