The residential buyer has one advantage though: they do not demand as much from those working. You can start this business today, and likely find enough friends and family willing to take a chance on you that you can get started and from there you have references. Commercial projects tend to be big enough that you cannot get into them without experience (which in turn you cannot get).
The crew that can build a driveway cannot build a parking lot, and they will never be able to break into that market because it requires experience that a driveway cannot give. From a parking lot there is then road building - the previous companies cannot break into them. Again there are just not enough projects of the size that sits in between to build experience.