I think that you are right. "Allow list" (or, in some contexts, "inclusion list") would be a better term, especially in contexts where colours are involved and it would be confusing.

The reasons that they usually actually have are not very good though, like you say, but nevertheless sometimes it can result in something better and sometimes not. But, banning words is not the solution, though.