However if you go further back in time (50 million years or so) it seems possible that the Earth was hotter than today, modulo all uncertainties and approximations needed to get to these figures: https://en.m.wikipedia.org/wiki/Geologic_temperature_record

Well yes, and if you go back even further, to a few billion years ago, the entire surface of earth was glowing magma.

But I'm kind of thinking more recent history than that :)

Which may be fine for the Earth in general, but perhaps not for the human civilization and agriculture we have built over only the last few thousand years.

Absolutely!

Humans have existed for only 200k years and I'm not sure primates let alone hominids even existed 50 million years ago. So irrelevant to our survival.