I see two vectors here
- The bot giving out PII by accident. You ignore it and report it.
- You trying to fool the bot into giving you PII you're not supposed to have. But you've created an audit trail of your 100 failed prompt injections. The company fires you.
This isn't public facing, open to anyone. This is more like a shared printer in the office.