There is no check on leaving, and our neighbor Mexico not only usually doesn't check, it's constitution* explicitly prohibits requiring passports for entry. And when you return US has to take you back, if citizen, passport or not. So you can easily go to and from Mexico without passport.

Virtually every time I go to Mexico, I'm one of very small % with passport going either direction.

* This is debated and some questionable court cases that actually protected rights of indigenous Mexicans to travel are cited as reasons why this constitutional provision isn't in effect (even though the case was clearly written to protect indigenous rather than justify overthrowing the constitution), but go to a land border with Mexico and 9 of 10 times I find they follow the constitution and don't check passport.