There have been millions of trivially exploitable vulnerabilities out there for decades — many of which could be easily discovered by using simple scanning tools or manual probing. This is hardly a new situation and LLMs really aren't that impressive at pentesting — even with these simple exploits. Maybe they are if you're not a pentester, but then ZAP, Burp, Nessus, SQLMap, etc. are likely also impressive if you put a little effort into learning how to use them, but many AI-advocates aren't interested in learning skills themselves.

It's the same situation as with vibe coding. Everyone and their grandma can have an LLM spit out a web application without any programming experience, but if you're a programmer, you'll likely quickly see some issues with maintainability and further development of the code base.