It's like a routing table on the way out and an ACL on the way in. Maybe an easier way to think of it.

Sure, but how does this differ from a routing table with RPF (which is default in Linux already)?

It's associated per-peer, so it assures a cryptographic mapping between src ip and public key.