I'm surprised to see no mention of hierarchical rendezvous hashing in either the article or the comments here.

It is purpose-designed for exactly this type of proxy/cache load-balancing scenario!