There's a security problem (and hence legal liability) when two people know each other's IP addresses.

There's also a complex networking situation when people are behind NATs, firewalls, etc.