Working a job you don't like isn't the issue.
Working a job that makes the world a worse place because you need to survive is the issue.
It IS less evil to do nothing and be fed than to take up arms in a factory that produces produces that people want, but is poison (cigarettes, as an example). Paying people to prevent exploitation from plantation owners is a good thing.
A lot of people like cigarettes. The idea that no one would work in a cigarette factory because everyone would see it as morally objectionable and they could afford not to is preposterous.
Would you take a low paying factory job that creates an objectively addictive poison as a product willingly when you have other options? Do you REALLY know ANYONE that would? I'm not saying that number is in fact 0, there will always be outliers...