Much of human innovation arises out of leisure interests where no immediate demand or need exists. Mathematics for example initially arised out of people with leisure time who found some hobby to do.
Think about it. Math is a hobby, practically useless. But it benefited humanity with side effects like engineering and computers.
Now to your point. Because math is initially a pretty useless endeavor it can only arise as a leisure hobby. A leisure hobby can only arise in people who have tons of leisure time and have no need to fight for survival. Thats why peace time is important.