Exactly. I don’t have the spare time but have been thinking that even a bit mask about provenance and policy could be prepended to the vector, then training could reinforce adherence, including having output tokens that indicate the provenance of the inputs used for the token.
How does that guarantee anything? I could definitely see it being better, but that doesn't make violating it impossible does it? Just... statistically less likely.
Looked at that way, there are no security guarantees anywhere. Root CA’s can be compromised, cosmic rays can flip bits, zero days can appear in your supply chain.
Perhaps “ensure to a level ~six orders of magnitude better than current practices” would be a better way to say it.