See this introduction to limits.[1]
If you allow infinite recursion, you soon get to Godel and undecidable problems. Finite deterministic systems are decidable, because you can in principle enumerate all the states. The halting problem is decidable for deterministic systems with finite memory. It may be exponentially hard for some programs, but that's quite different from being undecidable.
(This is too long a subject to discuss here, and I haven't worked on constructive mathematics in many years. It's more practical than it was decades ago. You need power tools, which we now have.)