You can't trust everyone will be polite or follow "standards".

However, you can incentivize good behavior. Let's say there's a scraping agent, you could make a x402 compatible endpoint and offer them a discount or something.

Kinda like piracy; if you offer a good, simple, cheap service people will pay for it versus go through the hassle of pirating.