I wonder though how a sensible architecture would look like. Sure it would involve HSMs but ultimately they need to support transactions and APIs.

Probably this draws the line to more traditional banks which are notorious for having so many layers of checks and firewalls.