Keep in mind that with U+258x characters you get a unique paired set by just inverting the foreground and background. This does work with some, but not all, of the U+259x characters as well.

You may very well be able to ditch colors using this technique with additional characters. Many characters are distinct with their inverted counterpart.