But should developers be barred from asking an LLM to try secure their own app? Its not different from finding exploits...

That is a completely separate question and discussion.