Not really. I can obviously say something, like you learn which features the models are able to actually implement, and you learn how to phrase and approach trickier features to get the model too do what you want.
And that's not really explainable without exploring specific examples. And now we're in thousands of words of explanation territory, hence my decision to say it's hard to put it into words.
I think you’re handwaving away vague, ungrounded intuition and calling it learning.
For instance, if I say “I noticed I run better in my blue shoes than my red shoes” I did not learn anything. If I examine my shoes and notice that my blue shoes have a cushioned sole, while my red shoes are flat, I can combine that with thinking about how I run and learn that cushioned soles cause less fatigue to the muscles in my feet and ankles.
The reason the difference matters is because if I don’t do the learning step, when buy another pair of blue shoes but they’re flat soled, I’m back to square one.
Back to the real scenario, if you hold on to your ungrounded intuition re what tricks and phrasing work without understanding why, you may find those don’t work at all on a new model version or when forced to change to a different product due to price, insolvency, etc.
You're always free to stop at the level of abstraction at which you find a certain answer to be satisfying, but you can also keep digging. Why are flat shoes better? Well, it's to do with my gait. Ok, but why is my gait like that? Something-something musculoskeletal. Why is my body that way? Something-something genetic. OK, but why is that? And so on.
Pursued far enough, any line of thought will reach something non-deterministic - or, simply, That's The Way It Is - however unsatisfying that is to those of us who crave straightforward answers. Like it or not, our ground truth as human beings ultimately rests on intuition. (Feel free to say, "No, it's physics", or "No, it's maths", but I'll ask you if you're doing those calculations in your head as you run!)
It is very silly to treat zero grounding the same as accepting core, proven concepts. Your PoV here is no different than saying "It rains because god is sad and crying" is an appropriate thing to believe.
If you want to say "god is responsible for creating the precipitation cycle", sure. But we don't disregard understanding that exists to substitute intuition.