I'm always skeptical of the whole "computer use" concept. It's like hiring someone and inviting him to your house and telling him to go ahead, feel free to sleep on the bed, use the toilet, eat whatever is in the fridge, watch the TV, and oh here are the combinations for the safe... and that someone you hire is a monkey.
I feel like I am taking crazy pills. Are we really having an AI fart around with a mouse and clicking on things to accomplish stuff because we're not capable of making one kind of software query and command another piece of software? It kind of boggles my mind.
There are several cases where this is not possible or desirable:
- you are using AI as an E2E testing tool and it's a requirement to not just test the API but also the workings of the UI
- you are using some third party tool and want to automate its usage
The writing was on the wall with the MCP->CLI jump. The promise to investors is that you're replacing people. People don't make API calls.
You are because that requires you to expose that API for every single piece of software ever
But think of how comfortable and productive the monkey will feel. It might not be that hard to just build temp housing for it while you have monkey business to do.
In fairness, you're hoping the monkey does all the monkey tasks you'd rather not do yourself