> I understand that such models can be used by malicious actors, but it’s fair to have it publicly available
I feel like there should be some mechanism to prove you own the code/app/site/whatever and it will remove the guardrails from the LLMs allowing them to find and fix these vulnerabilities.
This is a "they have guns so we need guns" scenario.
You can't guarantee everyone else will use a neutered model.
Isn’t this essentially what anthropic is doing, albeit in a manual fashion? They work with code owners to run mythos and find issues.
Only if you're some big corporation with deep pockets. They actually accepted me into their cyber program but Fable's still locked down.
OpenAI now makes it easy to join their verified security program. Took me 5 minutes, and I was able to get GPT to do a full end-to-end pen test
Impossible with source code, possible to bypass with app/site
Don't we already do this with services like Let's Encrypt, which is arguably more sensitive? If you had the codebase you could fake it, but it would still provide some amount of protection against abuse.
With Let's Encrypt, all the verification is done on their side with them controlling the connection between themselves and whatever they're trying to verify.
In this case, you can put whatever you want between the harness you're running (or modify the harness itself), and essentially "lie" to the model. Any verification technique would be fairly trivial to bypass, while you continue to run the harness locally.