It's pretty easy, you get hit with a surcharge on your other transaction volume if you allow something that V/MC don't want you to allow, and if you keep doing it you could get cut off which effectively makes you irrelevant to many of your customers.

As others have said, it's about perceived brand risk (V/MC allowed X terrible transaction to take place) and regulatory risk.