> from an hour to 3 minutes
> It did introduce a bunch of bugs that I had to fix, but I also discovered some bugs in the original algorithm.
Seeing those big improvements and knowing you did more than make it faster is always really rewarding. I hope you do do more optimization :) Just remember Knuth's advice. Because IO is a common problem and Big O isn't going to tell you about that one haha
Yeah, I first want to know there's an actual performance issue to fix. That's basically what Knuth said, and that's what I live by.
> People get lulled into a false sense of security because tests pass but test just aren't enough.
Users weren't using a particular feature because they said they didn't understand it. So we explained it, again and again. Turns out that feature was incredibly buggy and basically worked the way we claimed it did, only when it was used in the specific configuration we tested for. Add another node somewhere and weird stuff starts happening.
The tests looked good, and code coverage was great, but the fact that the tests run through all the branches of the code doesn't mean you're really testing for all behaviour. So I added tests for all configurations I could think of. I think that revealed another bug.
So look at the actual behaviour you need to test, not merely the code and branch coverage.