Both Codex and Claude Code have it, but they work slightly differently.
Claude Code uses Haiku to read through the transcript and decide if the goal has been completed. If not, Haiku injects a prompt back to the main model to indicate what still needs to be done.
In Codex, instead it's a tool available to the main model, plus some part of the surrounding harness that will re-prompt it if the tool calls haven't yet indicated that the goal is complete.
The issue that they are trying to solve is that sometimes models will stop before they have actually fully completed whatever task they were given; attention isn't perfect, and someitmes they'll complete part of it but not the whole task. Rather than making the user come back and re-prompt to keep going, they add a way to automatically do a bit more nudging to try to get the model to finish the task.
> Claude Code uses Haiku to read through the transcript and decide if the goal has been completed.
feels kinda odd to use a less capable model to determine if the goal is fully complete. Especially if the user is expecting /goal to thoroughly complete the task. A less capable model would be more likely to misclassify `isComplete?`
On Claude if you start with that, it won't stop until it achieves or exhausts your prompt. It feels like "here's your mission, go do it". I use it a few times a week.
> won't stop until it achieves or exhausts your prompt
This is overselling it. In all implementations of this that I’ve seen, a more correct phrasing is “won’t stop until it pinky-promises it achieved your goal”.
It usually relies on something like a hook that refuses to let the agent stop unless it includes a “promise sentinel”, which is basically the model having to include “I swear I’m done” in the response (usually as a Markdown comment so the user doesn’t see it). If that sentinel isn’t in the response, the hook rejects the completion and hands it back to the agent (sometimes it also restates the goal to keep it focused). It’s useful for sure, but it fails many times for obvious reasons.
More robust implementations use the hook to spawn another agent that verifies the goal is achieved, but that too isn’t prefect and sometimes performs worse.
Of course if you have a problem that can be deterministically verified, you could have the hook run this verification instead, but this is usually not the case (and even when it is, agents will often cheat the verification e.g. by deleting a test file).
The key thing is that normally Claude grades Claude's work, so you do get a"pinky promise, as all of Claude's training bias tells it to plunge ahead (even if that means "fixing" a test by commenting it out).
With /goal, a different Claude instance evaluates the goal. This is the robust version you described, and such an "adversarial Claude's" approach, with or without/goal, is exactly how you avoid pinky promises (from one Claude).
I'm way too afraid of a "paperclip problem"-style target overfitting to use it really; how could I even describe a goal so well it doesn't sacrifice other things that are important to me? Like, if my goal is to make an endpoint faster, will it create an over-engineered mess out of the clearly readable code I have to reach that goal?
/goal open a draft PR that makes <endpoint> ~200ms faster while still passing tests, and adhering to nearby style+pattern
It's obviously not perfect, but it's not like you're going to YOLO it into prod without evaluating it on whatever you care about.
If you want to be able to use it like that, you need to find a way to encode (most of) your concerns into something that can be programmatically verified.
That kind of emphasises the problem I was referring to; if I pick 200ms, will it stop before doing a proper optimisation because that would make it faster than that? Will it do something stupid to justify pushing from 195ms to >=200ms?
It might! In my prompt, I'm supposing you have some priors here about what's possible. Try saying something like "materially faster". But you need to also make sure it knows how to measure that endpoint's performance on its own, otherwise I wouldn't expect this to work well at all.
Is this useful? I feel like the problem is usually not that the model isn't capable of achieving what I give it, but the way it does it. Especially if originally I didn't 100% know how I would do it myself the model often takes weird paths through the code base, takes shortcuts that end up in weird feature interactions or pulls in a dependency without weighting if it could've been done without that.
I haven't really found a good way to solve this other than:
1. Produce an initial PR fulfilling all the requirements I knew at the start
2. Chat with the model about any weird snippets I notice and talk through alternatives
3. Simplify anything that I think is overengineered or plain unncessary
Sometimes I restart all over with more precise requirements but then it sometimes makes different mistakes/takes different shortcuts.
In practice the earlier I review the better the end result imo, so /goal seems very unproductive to me?
It's useful for things where it just needs to get through to completion. Long running tasks. I walk away and expect it to be done without pausing for input.
It's just a way of defining what "done" means, and passing off evaluation of whether "it's done" is true to a separate context and/or model. If false, it prompts the original context accordingly.
An agent is an llm running on a loop until it decides it should stop.
/goal is a gimmick where you run a "parent" agent on top that runs the agent on a loop until the it decides to stop, just prompting it "nope, not done yet, continue".
The article describes it.
Both Codex and Claude Code have it, but they work slightly differently.
Claude Code uses Haiku to read through the transcript and decide if the goal has been completed. If not, Haiku injects a prompt back to the main model to indicate what still needs to be done.
In Codex, instead it's a tool available to the main model, plus some part of the surrounding harness that will re-prompt it if the tool calls haven't yet indicated that the goal is complete.
The issue that they are trying to solve is that sometimes models will stop before they have actually fully completed whatever task they were given; attention isn't perfect, and someitmes they'll complete part of it but not the whole task. Rather than making the user come back and re-prompt to keep going, they add a way to automatically do a bit more nudging to try to get the model to finish the task.
> Claude Code uses Haiku to read through the transcript and decide if the goal has been completed.
feels kinda odd to use a less capable model to determine if the goal is fully complete. Especially if the user is expecting /goal to thoroughly complete the task. A less capable model would be more likely to misclassify `isComplete?`
> odd
There is nothing odd in the basic principle of Economics of using a "good enough" tool, optimal in other respects (e.g. cheap/er).
(You do not hire Nobel Prize winners as receptionists.)
The issue remains, whether the compromise to be adopted actually is "good enough".
On Claude if you start with that, it won't stop until it achieves or exhausts your prompt. It feels like "here's your mission, go do it". I use it a few times a week.
> won't stop until it achieves or exhausts your prompt
This is overselling it. In all implementations of this that I’ve seen, a more correct phrasing is “won’t stop until it pinky-promises it achieved your goal”.
It usually relies on something like a hook that refuses to let the agent stop unless it includes a “promise sentinel”, which is basically the model having to include “I swear I’m done” in the response (usually as a Markdown comment so the user doesn’t see it). If that sentinel isn’t in the response, the hook rejects the completion and hands it back to the agent (sometimes it also restates the goal to keep it focused). It’s useful for sure, but it fails many times for obvious reasons.
More robust implementations use the hook to spawn another agent that verifies the goal is achieved, but that too isn’t prefect and sometimes performs worse.
Of course if you have a problem that can be deterministically verified, you could have the hook run this verification instead, but this is usually not the case (and even when it is, agents will often cheat the verification e.g. by deleting a test file).
The key thing is that normally Claude grades Claude's work, so you do get a"pinky promise, as all of Claude's training bias tells it to plunge ahead (even if that means "fixing" a test by commenting it out).
With /goal, a different Claude instance evaluates the goal. This is the robust version you described, and such an "adversarial Claude's" approach, with or without/goal, is exactly how you avoid pinky promises (from one Claude).
I'm way too afraid of a "paperclip problem"-style target overfitting to use it really; how could I even describe a goal so well it doesn't sacrifice other things that are important to me? Like, if my goal is to make an endpoint faster, will it create an over-engineered mess out of the clearly readable code I have to reach that goal?
If you want to be able to use it like that, you need to find a way to encode (most of) your concerns into something that can be programmatically verified.
That kind of emphasises the problem I was referring to; if I pick 200ms, will it stop before doing a proper optimisation because that would make it faster than that? Will it do something stupid to justify pushing from 195ms to >=200ms?
It might! In my prompt, I'm supposing you have some priors here about what's possible. Try saying something like "materially faster". But you need to also make sure it knows how to measure that endpoint's performance on its own, otherwise I wouldn't expect this to work well at all.
Is this useful? I feel like the problem is usually not that the model isn't capable of achieving what I give it, but the way it does it. Especially if originally I didn't 100% know how I would do it myself the model often takes weird paths through the code base, takes shortcuts that end up in weird feature interactions or pulls in a dependency without weighting if it could've been done without that.
I haven't really found a good way to solve this other than:
1. Produce an initial PR fulfilling all the requirements I knew at the start
2. Chat with the model about any weird snippets I notice and talk through alternatives
3. Simplify anything that I think is overengineered or plain unncessary
Sometimes I restart all over with more precise requirements but then it sometimes makes different mistakes/takes different shortcuts.
In practice the earlier I review the better the end result imo, so /goal seems very unproductive to me?
For some frontier models like Fable 5 it doesn't matter, but for models less trained on long horizon tasks it very useful.
It's useful for things where it just needs to get through to completion. Long running tasks. I walk away and expect it to be done without pausing for input.
Can you give an example? And more curious about what you do with the resulting code afterwards I imagine its gonna be a big chunk then?
I feel like "do X untill tests are green" is sort of the prototypical /goal case.
Adding a new lint rule that a lot of code violates.
And that doesn't work with a simple prompt?
It's just a way of defining what "done" means, and passing off evaluation of whether "it's done" is true to a separate context and/or model. If false, it prompts the original context accordingly.
The llm runs in a loop until it meets a condition (the goal).
It loops until it thinks it has finished the goal
An agent is an llm running on a loop until it decides it should stop.
/goal is a gimmick where you run a "parent" agent on top that runs the agent on a loop until the it decides to stop, just prompting it "nope, not done yet, continue".
isn't this what they called a "ralph loop"?
[dead]