But the tokens are IP-bound, and as the article mentions a crawler using a single IP is not so difficult to block.

A legitimate user solves it once, shows legitimate-use behavior, and retains their access for thousands of requests. A crawler solves a challenge, is determined to use bot-like behavior after a dozen requests, gets an IP ban, and needs to recycle their IP and start from scratch.

It is indeed worse for fly-by users, but with a website like Linux cgit those basically don't exist. It would be a completely different story for, say, someone's personal blog.