I don't think that's actually true? That's like insisting all useful functionality would have to be moved to a smartcard/yubikey/bitcoin hardware wallet/TPM etc. The main reason this is an issue is to prevent emulated hardware tokens. If you can disable secure boot, you can emulate secure elements and then things that others (i.e. your bank, government, etc) believe are carefully controlled secrets are not.