Well, it's not exactly engineering, but physics often uses quite informal math. For a pretty modern example, the Dirac delta "function" was used long before it was formally described; and I have heard it said that even today String theory uses some math that is not fully formalized - though I can't say I know what specifically, so I may be wrong. Newton expressed calculus in terms of inifinitesimals (the dx notation was simply an infinitesimal delta x, not merely notation for derivation), which were not not formalized until much later, after they had already fallen out of favor and had been replaced in formal math by the delta-epsilon limits-based constructions.
Edit: one better example from modern physics - the path integral formulation, used in both string theory and other areas of QM/QFT, is not fully formalized and formally proven to work. Also, a more concrete example of a widely used but actually still unproven conjecture in string theory is the famous AdS/CFT correspondence.
Newton didn't use dx/dy. That's Leibniz' notation. Newton's notation for the derivative is just to pot a dot above the letter so ṙ would be Newton's symbol for speed (dr/dt) and two dots would be acceleration (d^2r/dt^2) in Leibniz' notation. Physicists still use Newton's notation but only for derivatives with respect to time these days.
The part of Newton's theory that was troublesome is his fluxions don't have the Archimedian property. It took until the 1960s before Newton's notion of fluxions became rigorously formalized with Non-standard analysis. https://en.wikipedia.org/wiki/Nonstandard_analysis
Oops, I confused some history, thank you for the corrections!
Physics famously has a rather cavalier attitude toward formal rigor, but that's because the ultimate test is comparison with experiment. String theory shows what happens when that bulwark against error is missing.