It only really makes sense for problems that are complex and require iterations that don't themselves require much review. E.g. if you want find, PoC, and patch bugs, the output can be reviewed without reading all the traces. Or if you want to write a custom tool that does some job using local LLMs, assembling that pipeline, tuning the prompts, etc takes a long time but reading the final tests + eval data + code is enough to get a lot of confidence that it works right. Model checkers can help too, for example I wanted multi-sink Bluetooth audio support in Gnome for my kids so I hooked the hardware up and robo-coded the core logic specifically to be checkable with Kani.
>don't themselves require much review
I'm still wary of any unreviewed code - though my area of work is not tolerant of defects.
Agree on targets / verifiable indications of progress or success being a prerequisite for this being useful - although that covers quite a lot of SWE work.