There's no secure way to do that. You have no guarantee that the computer won't copy your key or keylog your password.

You can mitigate it by using an MFA method that requires confirming on a separate device like a phone, but that's down to one layer of defense.

I use an SSH app on my phone for remote access, and I go over a VPN. SSH is not exposed to the public internet.