I can't find the source, but one reason why SpaceX can reduce costs more than NASA, is that SpaceX is allowed to fail. NASA has to underpromise and overdeliver on everything. Else it will result in budget cuts for not meeting targets or wasting money. Having such constraints, results in much higher costs. If NASA could "fail-fast" without fear of cuts, it would look very different.
Another point is that NASA also works as a job program for a lot of states.
Don't forget the cost of oversight either. Someone, somewhere, slacked off while on a government job, so now we need someone to watch everyone work, and we need someone to watch them watch them work, and then we need a council to decide on team 1 and team 2's efficacy in case they slacked off ever, and then we need a division to determine that the council is doing their job as well!