The point is not about "stifling EV adoption", it's reducing problematic levels of over-dependence on cars in general. EVs don't solve all the issues that cars raise.
Noise is a relevant factor in that discussion, not compared to internal combustion engines[1], but compared to fewer cars in general.
[1] The acronym for this did not age well
They don't solve all problems, but they do make solving the "too much CO2 cooks the planet" problem easier.
I'm all for fewer cars too!