Unfortunately Craigslist pushed a precedent that puts scraping into risky territory.
https://en.m.wikipedia.org/wiki/Craigslist_Inc._v._3Taps_Inc.
Obviously this was a much different circumstance than a not-for-profit side project but still something to keep in mind.
All the AI companies are doing it so it must be fine. Right?
This but also with copyright.
Right, unironically.