They certainly can say no in a lot of states, but that is often very much against the best interests of the prisoner because almost all work programs shorten your sentence.
I knew one guy who was doing a 30 day jail sentence for some misdemeanor and was told they would reduce it to 14 days if he worked in the kitchen. He took the job and lost most of his thumb in a very unsafe meat slicer. This put his 17 year career with UPS in jeopardy since the nerve damage made it hard for him to handle things.
This is very unrelated in most ways, but when I was 9 I managed to take a hatchet and remove about 1/4" off my left thumb (via cutting bailing twine against a tree, and apparently terrible, terrible aim).
I'm far older than 9 now, and the tip of my left thumb still gets very cold in the winter and if I directly bump it into something, it hurts a whole heck of a lot.
Rant is because while that moment sucked pretty hard (I immediately put my thumb in my mouth, eating the bit of thumb apparently..) it didn't take very long for me to realize that any lower and it would have certainly been a life changing event.
Bad aim, but in the best way possible.
I can only imagine the difference. Has to be harsh.