While I haven't tried OpenAI for OCR, I've put my small scale OCR work through both Claude and Mistral OCR. Claude is absolutely better - even in OCR work I did last week and compared with Mistral OCR 4.0.
Mistral's one advantage is that Anthropic now flags OCR, because they don't allow anything that could be considered "reproduction", even of work for which you own the copyright. So my new workflow is Mistral OCR for the actual OCR, followed by a proofreading pass by Claude (which is allowed). Claude is obviously more expensive, but it caught entirely hallucinated sentences created by Mistral OCR 4.0, so I was glad for the backup check.
Same company that OCRed millions of books, the irony.
I feel Anthropic is destroying itself with all these restriction. They got away because their models were the best for coding, but that is not an advantage anymore as OpenAI and other open source are already better.
Claude refuses to translate lyrics of songs even if it's just for your own understanding. ChatGPT allows it.
Also, Suno is so restrictive that it won't make songs of public domain 19th century poems because someone else already made a song and the lyrics are in their song database. And it won't do covers of your own uploaded humming of a novel melody because it matches "humming in an empty room" to some song fingerprint in the database.
They are all trying to avoid these lawsuits by being ridiculously overly cautious. "Your honor, look how much we went above and beyond, even when it made the product worse for legitimate use cases"
It is why they're pushing for regulatory capture. Let us do the bad stuff, make the money, then we'll regulate everybody else out of the market.
They were criticised/sued early on when people could reproduce copyright things. I dont think in this case it's something they'd prefer to do?
Yep, my understanding is that many guardrails like this are actually the result of government legislation (eg the Fable bans) or terms of settling copyright lawsuits over reproducing copyrighted text and lyrics.
I mentioned it in a sibling reply, but here's Anthropic's support document about not using Claude to reproduce content verbatim that already exists, regardless of copyright.
https://privacy.claude.com/en/articles/10023638-why-am-i-rec...
> Claude is obviously more expensive, but it caught entirely hallucinated sentences created by Mistral OCR 4.0, so I was glad for the backup check.
What does this entail? What does Claude do to decide that the text it was provided was hallucinated? Are you telling Claude that the source was OCR'd by another LLM?
I'm basically doing the OCR twice, except in the Claude proofreading pass, it is not being asked to transcribe the document to a Markdown file. I'm pointing it to the same image input files, and to the Mistral OCR transcript Markdown file (it knows it's a Mistral OCR output), and ask Claude to check that the text is correct and point out the errors - and then make the necessary edits.
I can't speak for Mistral OCR 4.1, but the hallucinations in 4.0 were so egregious (just completely making up new sentences in the middle of a page) that I knew I can't trust Mistral OCR on its own.
How bad is the first OCR pass allowed to be to still count as proofreading? Can you let Claude compare the images with /dev/random and make the necessary edits to correct differences?
Hmm, that's an interesting idea. But it's the classifier that is triggering, and it triggers specifically on Claude's output. So I think the /dev/random case wouldn't work, because that gets Claude into the state of just reproducing the entire text from the original again.
It doesn't always get flagged. Single pages are almost always okay. Running a program that sequentially runs single pages through the API is often not okay - I wrote a program in the early 4.x days before the rule came in, that's how I hit it first. But I've also had entire articles go through just fine recently in a Claude Code session (I'd forgotten about Anthropic's rules!), and then others where I get classifier errors by page 4.
The Mistral OCR errors were small in size. Single sentences, formatting errors, paragraphs with newlines. So this was a genuine proofreading job with small changes. For the most part Mistral is actually good, but I can't have it just inventing sentences in the middle of a document. That's where the Claude proofreading pass was most helpful.
I assume they pass the image alongside the text to redo/recheck the work. Yes this is silly, but is apparently required to get around refusals.
[flagged]
Yes, that's how I read it as well, but the question remains in how did Claude know this? It was not clear to me that Claude was being presented the original scan. The entire exercise is farcical, but it wasn't as clear exactly how farcical it was that the person was essentially running OCR wastefully on a different model just to convince Claude to do the OCR as desired because of Anthropic's rules. Here's this cool tool that is supposed to make you do less work and be able to do it faster and be able to do it more accurately, except you first have to do all of this other unnecessary work with a totally different company first. <face-palm>
>Anthropic now flags OCR
I haven't seen any difference in my ocr workflows, what do you mean by this?
Not the person you’re responding to, but I’ve had Claude refuse to OCR pages from in-copyright books. I was sometimes (but not always) able to get around that by changing models, by telling it that I was doing the text conversion only for personal use, or by first telling it to use Tesseract or another OCR engine to do the initial pass and then having a Claude subagent proofread and clean up the OCR output.
I’ve also had it refuse to OCR public-domain books that included content that it didn’t like, such as references to prostitution in 19th-century books about Japan.
I had one session where Claude refused to continue after it hit some kind of guiderail restriction. I couldn’t see what the trigger was, so I started a new session, gave Claude the link to the previous session, and asked it to diagnose the problem. This new Claude said it couldn’t view the exact guardrail issue, but it did suggest a workaround that turned out to be effective.
Yeah I do something similar and Claude (well OpenAI too) just refuses to transcribe anything related to slavery and pederasty in Ancient Greece.
Are you familiar with the Thesaurus Linguae Graecae? It's a project that may meet your needs. Paywalled, unfortunately, but I assume it would be a one-time expense.
Ah, I see. Thanks for sharing. I am OCR'ing public domain .pdfs, so perhaps I haven't run into that issue. The irony couldn't be stronger though.
"it refuse to OCR public-domain books that included content that it didn’t like, such as references to prostitution in 19th-century books about Japan."
Thoughtcrime -like territory and self-sensorship. The AI safety lobby is such a vile influence on the freedom of expression and communication via technology (since AI is starting to eat up rest of technology).
I guess the main problem is positioning AI tools as "human-equivalent" creators by the big AI corps. If they were positioned simply as "better OCR and proofreading" people would attribute to them as much responsibility as they would to a - say - typewriter and we would not need to have this nonsense.
I do realize most of the valuation comes from the positioning of "our TAM is the global salary base of 50 trilion and we aim to supesede human workers in the near future" which implies they need to position this technology as "human equivalent" or that valuation is no longer as credible.
This is too funny considering they did that themselves. I’m pretty tired of these companies deciding what we can and can’t do while they act with impunity.
I also saw this when I made a (personal use only) book translation (agent learns how book 1-3 of a row is translated, then translates book 4 the same way because it is not available in my language). Claude happily translated most of the book, except a few chapters which it denied. Tried several times always the same results (with no other context about the rest of the book).
The response / conversation gets blocked by the guardrails with "API Error: 400 Output blocked by content filtering policy"
At first I thought it was something in the scanned content that was being flagged, but it was the attempt to transcribe that was itself being flagged. Anthropic mention it on their pages:
"Anthropic takes these steps because Claude’s purpose is to generate new content and ideas, not to reproduce content that already exists."
https://privacy.claude.com/en/articles/10023638-why-am-i-rec...
Side note - Claude itself is not aware of this policy, and is unable to see the API responses - the turn just ends. Which turned into a really bizarre failure state where Claude thought I was gaslighting it and kept insisting it could do the work and even had the entire text in memory. Every time it would go to show me and prove it, it would hit API Error 400. I was only able to convince Claude by showing screenshots of my Claude Code screen output so it could see that I was seeing API errors. I've never seen Claude get into that angry & snarky state before, and I hope it doesn't happen again.
[flagged]