Damn, this is a good one. Sounds like we need an ANSI sanitizer, keep only basic formatting, remove all esoteric escapes, the fancy Sixel & co stuff.

For paranoia you could us a Chrome like multi-process architecture, the ANSI parser runs in it's own sandboxed process.