Interesting, there are a handful of PDFs in the drop that appear to be an email with a Base64 encoded attachment—inline.

OCR is so bad of course that decoding the Base64 seems futile without a lot of effort.

Example: https://www.justice.gov/epstein/files/DataSet%2011/EFTA02609...

(More mentioned here: https://old.reddit.com/r/Epstein/comments/1qu9az2/theres_unr...)

Would a few byte errors break a binary so much as to make it undecodable ?

I think it's more than a few bytes error. (I believe this because I spent about 15 minutes on the linked document and came up empty.)