I agree. The difference in how much we're out of sync with the sun is tiny and also completely inconsequential in the vast majority of cases, especially compared to the trouble of the alternatives. By the time the difference is relevant or noticeable, I sure hope we have a better way of tracking time.

If anything, it's probably for the better to completely disconnect the measurement from it's old standard, the same way imperial measurements are now defined using SI units as a baseline, and how 1 AD isn't actually the year of Christ's birth. It's OK that the actual measurement is off from what it was once based of, so long as we all agree how we're defining our units.