I don't have kids. The homeless district is easy to avoid, and there's a k-12 school right on the edge of it with many kids walking, cycling and taking the bus to school. I've never heard of a serious incident involving homeless people.
Here's a fun fact for you: statistically cyclists live longer than non-cyclists, and that's in spite of all the hazards they have to deal with while out riding.