Having a front door physically allows anyone on the street to come to knock on it. Having a "no soliciting" sign is an instruction clarifying that not everybody is welcome. Having a web site should operate in a similar fashion. The robots.txt is the equivalent of such a sign.
No soliciting signs are polite requests that no one has to follow, and door to door salesman regularly walk right past them.
No one is calling for the criminalization of door-to-door sales and no one is worried about how much door-to-door sales increases water consumption.
If a company was sending hundreds of salesmen to knock at a door one after the other, I'm pretty sure they could successfully get sued for harassment.
Can’t Americans literally shoot each other for trespassing?
Generally, legally, no, not just for ignoring a “no soliciting” sign.
But they’re presumably trespassing.
And, despite what ideas you may get from the media, mere trespass without imminent threat to life is not a justification for deadly force.
There are some states where the considerations for self defense do not include a duty to retreat if possible, either in general (“stand your ground" law) or specifically in the home (“castle doctrine"), but all the other requirements (imminent threat of certain kinds of serious harm, proportional force) for self-defense remain part of the law in those states, and trespassing by/while disregarding a ”no soliciting” would not, by itself, satisfy those requirements.
> door to door salesman regularly walk right past them.
Oh, now I understand why Americans can't see a problem here.
>No one is calling for the criminalization of door-to-door sales
Ok, I am, right now.
It seems like there are two sides here that are talking past one another: "people will do X and you accept it if you do not actively prevent it, if you can" and "X is bad behavior that should be stopped and shouldn't be the burden of individuals to stop". As someone who leans to the latter, the former just sounds like restating the problem being complained about.
> No one is calling for the criminalization of door-to-door sales
Door-to-door sales absolutely are banned in many jurisdictions.
And a no soliciting sign is no more cosmically binding than robots.txt. It's a request, not an enforceable command.
Tell me you work in an ethically bankrupt industry without telling me you work in an ethically bankrupt industry.