> there is a relatively big agreement on not taking it literally by society.

I think you're seeing a relatively big agreement where there really isn't one. It's not just the U.S. or biblical literalism, it's also about most people not really being familiar with the notion that even the strangest religious doctrines might be "true in a non-literal (but still worthwhile!) sense". From that POV, the garden variety atheist's argument is raising that very point. You don't have to take the atheism literally to understand that it's hard to believe everything about religion in a literal sense.