This idea is one I floated for my agent framework; it can get a little complicated if the model needs to reason about the redacted data. If not, excellent.
The other half of this equation is correctly marking PII/etc. This is a problem I'm relatively familiar with, at least as far as brute-forcing from raw files. I'd be curious to hear about how you managed this. Or is that something that AWS handles for you?
exactly i am actually working on complicated it can be if the model has to reason around the data, we use a combination of regex patterns, named entity recognition, and context aware detection to identify and tag PHI, PCI, and PII before tokenization. AWS doesn't do this for us that's actually the gap we built around.
Our approach is that tokens are format preserving and semantically typed. The model knows it's handling a name, a dollar amount, a diagnosis code it just never sees the real value.