When we say "everyone in science", I think the part that people find scary is that it's hard to tell who is in science versus who is in 'science'.

Or in other words, it's hard to tell from the outside who really believes what you're stating and who believes it until it's inconvenient, or until it clashes with their personal ideology.