This adds complexity and the added questions of how much you really trust vendors to handle it correctly.
After setting them up, I store one or more at various other locations. The core services people use them for rarely change, and adoption outside of those important services is slow. Even if you only kept one at home and one on your person at all times, this might mean a key would survive something like a house fire.
If given the choice between a hardware token and a passkey, I would prefer the former since it is almost impossible for it to be tampered with (especially without physical access to it).
I do see your point about HSMs and see why people would want such features (especially if there are multiple interested parties involved).
> The core services people use them for rarely change, and adoption outside of those important services is slow.
That's the opposite of what would be good for general security, though. I want people using these devices for as many services as possible. Heck, I even enjoy the FIDO workflow when I've used it in corporate settings (where I have a recovery method that doesn't involve begging a FAANG company to please, please, please give me back access to my stuff). I would love to use it for personal stuff instead of mucking with a password vault, TOTP, etc.
I guess the argument could be made for using these devices only w/ your "digital feudal lord" of choice (Google, Microsoft, Facebook, etc) with the expectation that all other services you use would federate authentication with those various fiefdoms. I don't see that happening with banks, for example. (Banks would be a great example of a place where I'd want to see token-based authentication adopted. I am so goddamned tired of SMS "authentication" being used w/ to arbitrate access to my money.)
I also find the idea of companies federating their authentication with the big digital feudal lords, to the exclusion of local authentication, repugnant.
My problem w/ tokens as they are now comes down to the "you must have both tokens in the same place to enroll new accounts" workflow being bogus. It creates make-work for people. Any technological "solution" that creates make-work for people is wrong, full stop. For me this workflow would discourage using the token in favor of technologies I can back-up (like passwords and TOTP seeds). For "normies" people are just going to lose their tokens and be in account recovery hell.
Here's my pitch for how I think it could work:
Token vendor sells a very simple embedded device to "brand" tokens. The device has no communication capability beyond a display and a USB port to connect tokens. This "branding iron" is enrolled w/ a vendor-signed certificate at the factory. The tokens are probably already enrolled w/ a vendor-signed certificate at the factory. This establishes a root of trust for the tokens and the "branding iron". I don't have to trust the vendor for anything more than a hardware root of trust.
The end user would use the "branding iron" to create a master key and enroll it onto their tokens. The ergonomics are very important but, functionally, it's just spinning a suitably large random key, showing it to the user so they can make an offline copy, and loading that key onto tokens. There are a ton of ergonomic decisions to A/B test (does it have a PIN pad, does it use a Bitcoin wallet-style key phrase presentation method, etc), but the functional purpose is simple.
The user would plug their tokens into the "branding iron", which would wipe them clean and, using the root-of-trust shared between the devices, load the master key into all the tokens.
This makes encrypted backups portable between tokens. The backups could be stored anywhere. (I like the idea, since a lot of tokens are HID keyboards, of just having the token "type" the backup into an email or a text file.)