I think two factors are in play:
The first is that there is likely more diversity the deeper you go down the intellectual hole. You and I may read much more sophisticated books, but the books you read and the ones I read differ significantly. Thus, the list is biased towards the more popular (it is, after all, a popularity list).
Second is this:
> for engineers to be detached of liberal arts?
Most of us just haven't found value in the other types of books. It would help if you gave some examples of books that should be here. For me (perhaps as an engineer), I like books to kind of get to the point. When it comes to fiction, I'm a very firm believer that, although a given novel may give great commentary about a social/philosophical issue, its primary purpose is entertainment. If I wanted to understand the underlying social/philosophical issue, a more direct, nonfiction book will always do a better job.
I've yet to find someone "changed" because of fiction. Those I know who claim to already had the sentiments before they read that piece of fiction, and the story was merely preaching to the choir. What they are glorifying is how well the story depicted an issue.
>although a given novel may give great commentary about a social/philosophical issue, its primary purpose is entertainment. If I wanted to understand the underlying social/philosophical issue, a more direct, nonfiction book will always do a better job.
I think there is a fundamental misunderstanding there, if you think all the potential value of a fiction book is some commentary padded by a story.
Good fiction usually exercises the mind in ways a non-fiction book never would. You experience life through someone else's eyes, you try to understand someone's mind by the actions they take and the words they say, you wonder in a meta-plane what the author is trying to show, you see language being used in non-common ways to provoke emotions or express ideas, you wonder how you would have acted in someone's shoes... Saying the author could get to the point quicker is like saying that lifting weights in the gym is done faster with a forklift, the process is the point rather than the extracted output.
There is also a fundamental difference between being told 'pain is an unpleasant feeling that living beings take effort to avoid' and being punched in the face. Fiction gives you a fraction of the extra wisdom you get with the latter.
>It would help if you gave some examples of books that should be here.
That's the thing, there is not specific book I could recommend that is most likely change your life for the better, for the same reason there is no single specific equation I can mention that will make someone good at math if they solve it. Some exercises are better, some are pointless, but it's the act of engaging that counts in the long term.
My comment about this list had no implication that the books at the top of the list were less valuable than other hidden works; they're just a sign of a path not travelled quite far, if that makes sense.
And leaving aside the usefulness of it all, pleasant experiences not all amount to entertaining. You'd probably agree that having sex with the love of your life and watching TV are not equivalent experiences, even if you come out of both with roughly the same level of self-improvement.
> if you think all the potential value of a fiction book is some commentary padded by a story.
Actually, I'm flipping the two: The potential value of a fiction book is a good story - social commentary is purely optional. Fiction that has commentary padded by a story are valued only by those who are sympathetic to the commentary. Whereas I can easily love a good story even if I disagree with the commentary.
> Good fiction usually exercises the mind in ways a non-fiction book never would. You experience life through someone else's eyes, you try to understand someone's mind by the actions they take and the words they say, you wonder in a meta-plane what the author is trying to show, you see language being used in non-common ways to provoke emotions or express ideas, you wonder how you would have acted in someone's shoes... Saying the author could get to the point quicker is like saying that lifting weights in the gym is done faster with a forklift, the process is the point rather than the extracted output.
I don't think we're in disagreement. I'm merely saying that I've yet to see someone changed by a fiction book. If there was change, it was always "change in the same direction" (e.g. "a renewed appreciation of X").
I have seen plenty of folks changed by nonfiction, though.
Incidentally, most/all of what you wrote above can be done as effectively with nonfiction. Books like When Broken Glass Floats by Chanrithy Him are extremely powerful. As was Killers of the Flower Moon. I doubt any works of fiction dealing with the same topics would be more powerful. Both of these books could have been written (and read) as fiction, but knowing the events were true makes a huge difference in appreciation.
> There is also a fundamental difference between being told 'pain is an unpleasant feeling that living beings take effort to avoid' and being punched in the face. Fiction gives you a fraction of the extra wisdom you get with the latter.
This seems like a false dichotomy. You can have nonfiction do this very effectively without simply "telling" you.
Behold a super-satured solution. A crystal touches it and where before was a liquid, now hard unyielding crystals spring forward, hard enough to scratch diamond. Some books are seed crystals, changing the shapeless into eternal forms.
Let me know when you find such a book ;-)
All books can be seed crystals, provided they find an appropriate super saturated solution in their readers (that was my point, actually, as a counter argument to OP's "preaching to the choir". Although the super saturated solution is already there, it needs to seed crystal to (physically) transform).