I think this is a very real concern. But I’m not sure of any way around it. Any stenographic system that you have the code for can be trivially defeated.
I wonder if this would be a good use for homeomorphic encryption. There might be a way to let anthropic check some text without actually giving them access to the source text. Any experts around? We could use your skills!
> Any stenographic system that you have the code for can be trivially defeated.
They're giving you an oracle regardless, which is almost as good. Take LLM output, make some modification, ask the detector if it's LLM output, repeat until you learn what kind of changes you have to make to defeat it.
Or don't even bother learning what to do, just make arbitrary changes until it says it's not, so when the person they're submitting to does the same check it says the same thing.
I assume this oracle will be behind 20 layers of anti-bot protection, CAPTCHAs and hardware attestation challenged. It will be incredibly painful to use. It won't stop the motivated attackers, but will make it too annoying for the average person.
> it says the same thing
Reference needed? I think it remains to be proven whether those detectors can be considered deterministic.
They could (..and probably will..) store that version and then refuse the check if this attack is detected, i.e. the version is too close to a known LLM output.
Alternatively they could also just keep saying "yes" if it's close enough to a version that was close enough.. Although that would enable the attack to allow arbitrary text to be "proven" AI, by slowly morphing close-enough generated material to the desired text. But perhaps this is not a problem they are not concerned with.
To satisfy the letter of the law I expect it's enough to just provide the oracle, without any mitigations.
Won't we just be able to fine tune OSS models to detect these patterns across providers? It will be cat-and-mouse but my bet is it converges to a central detector that isn't affiliated with any model provider.
> Won't we just be able to fine tune OSS models to detect these patterns across providers?
A good fingerprint should make use of cryptographic signatures. Without knowing the keys, the fingerprint should be indistinguishable from noise (or just random token selection)