TLDR: We need agents that can read our minds.

I think we want two opposing things:

1. An agent that acts autonomously 2. An agent that acts like we would

The problem is that an agent can only act like we would if it would know our mind and all the bits and pieces we did not define but are obvious or clear to us.

The only real solution to get an agent to act like we would is to make it ask clarifying questions, breaking the first requirement we have. Until we have agents that can literally read our minds, we cannot have both.

Optimizing the harness/context is the best way to make it act like we would, but this of course isn't working perfectly.