No one presumes you 'should' work 9-5, but that is the way it is, and the bank/postal office/whatever employees don't have the option to work evenings. It is the way it is.
Now whether we could have a better system -- Sure! I'm all for a better society. I'm just saying it's not the way it is because of discrimination or some other conspiracy.