That will irritate real users half the time while the bots won't care.

If they are a real user going on your site in 2025 then they have no alternative they are even interested in. They will blame their ISP and wait.

Meanwhile rate limiting the llm could potentially cost a lot of money in time and compute to people who don’t have our best interests at heart. Seems like a win to me.