One big problem I see with that is that, while almost all passports and EU ID cards now support ICAO cryptographic document validation standards, there's usually no publicly accessible revocation list for these.
Combine that with the absence of any built-in user verification (some national schemes have a PIN code, but the track record of that isn't great), and it becomes clear why these documents don't fully solve the problem of strong identity verification.