Emacs's hexl-mode does this, incidentally, though annoyingly by default it makes all faces the same color. I never understood why it defines the faces but then doesn't customize them.

What exactly does it do? I'm looking at hexl-mode sources in my Emacs, and I see it defining only two faces - hexl-address-region and hexl-ascii-region.

That's correct, as far as I can tell: the first one is used for all hex values in the "main" are of the buffer, and the second one for the character representation of each byte in the right-hand side column.