> its use in machine learning (or elsewhere, I suppose) is itself a form of empiricism in that what it is is essentially a repeated observe/measure error/adjust cycle.

"Empiricism" implies that the technique is based on observable, but not mathematically proven foundations. If a problem space is convex, gradient descent is guaranteed to converge to a global optimal solution, regardless of whether you know the exact formulation of the space.

Applying it when you don't understand if a space is convex is another question, but that's not a fault of gradient descent.