In the 56 years since UTC was established, there have only been 37 leap seconds. At that rate it would take more than 5400 years before it would affect solar noon more than DST does. I'm more than okay kicking the can that far down the road in the name of avoiding all the ridiculous solutions that are needed to accommodate leap seconds. We've endured these headaches to potentially solve a problem for people who might not even still be using UTC.

Compare that to removing the leap day, where the start of seasons would be noticeably affected within just a few decades. Hundreds of years ago, a pretty insignificant headache was invented which is providing constant payoffs.