You always need a long-lived key somewhere. Keeping it in an HSM is probably the safest, but also pretty expensive.

Yes, you'll probably need to buy a specific tool or product to have secure key storage. If you're running infrastructure that requires long-lived secrets, then you either need to have a place to keep them and a plan for secrets lifecycle management, or you need to accept the risk that those secrets could be compromised.