Have you ever explored the idea of shaped notes?
There's multiple different approaches with both 4-shape and 7-shape systems being common. But the point is that your color system seems largely correlated to it, and there has been research done on the shape note system.