I’m sorry but 20% of a retirement fund is a lot of money.

It may be a lot of money but it shouldn't matter because you don't need all of it right away.