I use KOReader a lot. After your comment, I realized file browser annoys me too. After some searching, I found an alternative view. Turns out, you can open the File browser tab, swipe down and switch "Display mode" to "Mosaic with cover images". Also seems like it's default for new versions now. This mosaic view still uses directories but it lists books as rectangles (covers) instead of lines (covers + titles), it seems more pleasant.

By the way, in the same settings I finally found how to set up KOReader to launch with a previously opened book. What a relief!