IIRC Claude has some kind of mechanism where even though the system prompt is sent from the client, if it’s modified it can trip tamper checks and result in a ban.
IIRC Claude has some kind of mechanism where even though the system prompt is sent from the client, if it’s modified it can trip tamper checks and result in a ban.
Aren't they running a safety classifier model in parallel to the main user thread to catch and prevent 'banned' outputs?