If your system is compromised, it doesn't matter what you are using: the attacker will get the password for whatever secure thing you store your secrets in.

Sure, but is your system compromised because someone left a key in plaintext? Attack surface is finite and can be meaningfully reduced.