> Let the Agent use the CLI and if it guesses the wrong option, you make that the RIGHT option
This sounds backwards and presumes that the statistics machines which are LLMs are getting it right when they "average" out to the wrong command. No, fix the agents behavior, dont change the CLI to accommodate it.
the real solution is to simply provide hints in responses so that the model may self-correct, e.g., recommended next actions, describe commands to get schema definitions, etc.
I don’t remember exactly the specific examples off the top of my head (some are definitely ffmpeg commands) but I do know that when LLMs keep hallucinating command line flags that don’t exist for that specific command their “suggestion” is actually very reasonable and so many developers are adding support to their tools for common hallucinations.
Not to belabor my point, but I think "adding support to tools for common hallucinations" is a bad idea. Sounds like something a vibecoded project being spammed with issues by agents might do. Not so much a serious, mature project, though.
Well we will have to agree to disagree because my understanding of what has been generally the case is that the LLMs might vibe-coding spam, that’s true, but the interesting difference is generally speaking their “suggestions” are very reasonable and represent in hindsight useful changes that make the commands more useful for everyone, humans included.
If an option exists but it's got a poorly named flag, adding a flag alias is probably a good idea for usability in general. Most CLI tools probably don't report telemetry about failed executions, though, cuz that would be very creepy.