More than twenty years ago, I quit a program that taught math/cs/physics (the notorious French "classes préparatoires") ~almost precisely over this: I felt like I was being taught physics like it was an axiomatic system where the tricks should not be questioned, they just work so "shut up and calculate" (and you don't even need to be doing quantum mechanics for that).
I just felt like we never got to the heart of the matter of why the models work and how to approach developing them, it was all about learning a bag of tricks.
Meanwhile, math and CS being a lot more axiomatic by nature, they also made a lot more sense to me.
That being said, that specificity of physics, the unbridgeable gap between reality and the models we build to describe it, in retrospect, is what makes it more interesting to me today (it's not just a "closed" system in the sense that math is — of course the relationship between math and physics is itself fascinating but that's yet another topic), but I still feel like I haven't found the right pedagogical approach to make it fit my mindset.
Your issue with physics but not with math reminds me a little of Hume's law. The difference that has always made that difference "make sense" to me is that math rules, even the axiom we use, are entirely chosen by the people using them, but the rules of physics are only useful if they match/predict what happens in the real world. For math we get to pick the ones that happen to be useful at a given time for a given problem (my go-to example of "it's all made up and the points don't matter" is why 1 isn't considered prime). For physics we're constrained to pick what best describes the real world. It probably helped that nearly all the physics course I had in high school/university had lab components focused on experimentally validating those rules/using those rules to predict results.
I think what it boils down to is that in my experience physics education lacks a clear historical component about how the current state of the art is a gradual process of proposing new models and rejecting old ones and figuring out the gaps between reality and the model. Instead, it feels like a God-given set of equations (that lots of people consider "the truth" for some reason), that you apply to cookie-cutter problems you must learn by rote. Though I understand the practical concerns (but then let's call it "physics for engineering"), as far as I'm concerned, you couldn't treat physics in a worse way.
that (edit: the way you were taught) sounds like an altogether awful way to learn any hard science
The world just is, regardless of what we think about it. Physics is our best attempt so far to understand and predict it at a low level, but it will always be incomplete.
Maths (and especially compsci!) are constructions by and for humans.
Is it any wonder it is as you describe? It would be odd if it was any other way.
My point is precisely that I was often taught physics as if it was mathematics, where there is in fact a profound ontological difference between the two.
Also, physics (the discipline) is also a construction by and for humans.
To find tools applicable to reality. Not to construct reality.