Skills don’t need to be preloaded, can be invoked by name on demand; and when you list available skills, a one-line description for each is enough.
Skills don’t need to be preloaded, can be invoked by name on demand; and when you list available skills, a one-line description for each is enough.
And all those "names and one-line descriptions" live in an .md file and of course pollute your context.
And then the model will randomly decide to load the .md file that is the "skill" randomly at any point. Again polluting the context.
There's no magic.
Most people seem to mean "Skills don't pollute the context window [with their entire contents]", they just elide the part in brackets.
There is no magic and people should be more precise when discussing skills.
Yes, but I expect that the elided part is less important than people assume it is.
Token count is a less important factor in context pollution than idea count. The worst of the rot factors are when models latch onto irrelevant information, or over-index on some vague idea/suggestion as if it was a hard direction, and then go off course.
The names + one-line descriptions of 10 tools can do as much (or more!) to distract the focus and intentionality of an agent than a 30k token exhaustive API documentation of some tool.
Is it so unusual to just ... not ... load the file with a description of them all automatically, but have different files listing different ones for different situations, and manually point to them in prompts?
Depends on how the harness injects those files. All harnesses these days auto-discover "skills" and read them anyway, and inject their descriptions, at the very least. Which leads to this anyway: https://news.ycombinator.com/item?id=49183906
But yeah. You could have files with skills outside of the auto-discovered skill tree, and invoke them manually.
I've seen implementations of a skill search, where instead of loading all descriptions into the initial prompt there's a tool the agent can call to search available skills and see if one might match their new task.
Search itself pollutes the context. With the need to maintain the tools, the search, the search results etc. in context. And wasting tokens while interpreting results.