I suspect some of these anecdotes are false, 15 mins of tale of two cities, 8 digit division in your head?
This gives a sense of embellishment by a clueless layman who think very smart people can memorize every single thing they ever do. Computers memorize well, very smart humans, it’s not clear if they memorize any better than a man of the street. I was struck by a fields medalist recent comment that he sometimes struggle remembering proofs he wrote himself in the past.
Memorizing books is absolutely possible. A great book about it is Moonwalking with Einstein (https://en.wikipedia.org/wiki/Moonwalking_with_Einstein), written by a journalist who was a skeptic but ended becoming a competitive mnemonicist himself. Some people intuitively learn the techniques early in life, and for some of them it becomes a habit.
Same thing applies to mental arithmetic. There are techniques for doing this. IIRC, ironically there's a famous case of someone who claimed to have memorized some hundreds of thousands digits of Pi but was suspected of using mental arithmetic techniques when reciting them.
Still, your point stands. People do embellish these kinds of stories based on stereotypes. It's certainly possible, but any particular anecdote may still be suspect.
Actors routinely memorize whole plays, so it is not implausible for someone to have memorized 15 min of A Tale of Two Cities.
The claim in the article is he remembered everything he read perfectly, which is wild. But another source claim he specifically memorized the Dickens book to train his English, which sounds more likely.