The trouble is: there is no deterministic algorithm that can do the things neural networks can do.

For many of these problems, I think it is likely that no deterministic algorithm can exist because the problems are fundamentally underspecified. E.g. a common task in computer vision is generating a 3D depth map from a 2D image. This is inverting a lossy projection, so any solution must be a least partially a hallucination.

I think we just have to accept this. It's a different type of algorithm, built out of statistics instead of logic, with different strengths and weaknesses compared to traditional software.