You can't build a space elevator before getting to the orbit first.
A jet engine capable of leaving a deep gravitational well must have a big ratio of thrust to weight. If a chemical rocket is too weak, a nuclear jet engine is the only remaining option. Would you be comfortable running it in the thick atmosphere of a densely inhabited planet?
A civilization just a few decades ahead of us is (theoretically) almost unimaginable. Bio augmentation, true AIs, who knows what advances in fundamental physics knowledge... Just to start.
What I'm saying is that whatever engineering and environmental limitations we currently perceive are probably irrelevant.