The VLM's are so good at complex document understanding now. But you just can't trust them not to invisibly censor sensitive clinical/legal docs, even at the maximally permissive settings.
And the deep learning OCR-only models won't censor, but can and do hallucinate. I've yet to see a 'scan with different approaches and reconcile and say you're not sure if they don't agree' system just work for generic complex documents.
The way my harness set it up is going through 2 or 3 providers, and cross-checking through them, also with plain text extracted if available.
I think we also had a layer that for any quote extracted tested it back if it exists within the original.
If you wanted 100% accuracy, I think it wouldn't be too difficult nowadays to guess the font&size&other text settings, and re render the crucial parts.
> But you just can't trust them not to invisibly censor sensitive clinical/legal docs
What's an example of this?