> It’s more economical at a compute level, but not at the developer level.

Developers and compute are interchangeable now.

Who's proompting the machine to do it differently without a developer there to ask the right questions?

You can have a high level prompt of: make our crawling cheaper and more reliable to run.