I have not encountered anything like that (but I am not in the US).

What I have encountered in the two countries I have lived in is the persistent expectation that the norm is for children to be with women, not with men. This makes suspicion of men with or near children more plausible.

I still find people are surprised a man is the primary parent. My daughter lives with me and people assume her mother must be dead. People have asked where mum is. Dads looking after their own kids are sometimes described a baby-sitting.

This is not as bad, but it does set things up for regarding men being around children as suspicious.