What happens then if the security scanners say something is safe and it turns out not to be?

I don't think PyPI should be in the business of saying if a piece of software is safe to install or not.

Then it will be downloadable and then it's up to your own security scanners to catch it. If you find it, it should be reported to pypi and then the scanner should be improved to catch that kind of bypass the next time it comes around. In such a world I don't think pypi is acting negligent.

That's really not very different from what we have right now. PyPI works with scanners which catch a whole lot of malware and are getting better all the time.

I think PyPI suggesting that software is safe would be a step down from this because it make promises that PyPI can't keep, and would encourage a false sense of security.