You're right, but even if I cut the water pipe right after the meter and run it for a month I might get a few thousand dollar charge.

You can ring up tens of thousands+ overnight with AWS. The scale of potential damages is nowhere even close.