IMO what's really wishful thinking is believing that society will necessarily adapt for the better in response to a deluge of AI spam/ads/propaganda.

You could have said the same about say, pre-AI deceptively edited/ragebait/made up content going viral on FB, "actually this is good because soon people will realize they are being tricked/lied to, they'll think extra-critically before sharing dubious content next time".

Which has not happened. I can only see AI videos/images making the problem worse as people are fed personalized, narrowly targeted content that seem to perfectly appeal to their own beliefs/biases/emotions/etc.

Also, if anything it seems like we will have to trust authoritative groups more thanks to GenAI. If I have to consider every video on the internet from e.g. Iran as fake, I'm going to turn to NYT or WSJ who can be relied on to (usually) share only original content, or highly vetted 3rd party content.

I agree that the solution we may find might not necessarily be for the better. In fact, there are a couple solutions I've seen that fall onto that category, like banning GenAI (does nothing to solve the underlying issue while control over economic production always requires increased authoritarianism).

I can't really provide a truly good solution, as this problem has large ramifications into philosophy and ethics, but I'd think it would involve solutions like attestation and certificates, and, primarily, thinking of shared media (text, images, videos, etc.) not as facts, but, strictly as allegations.