You don’t want to do that for anything you want to be able to vary, but they do something similar with a “soul document” for things they always want to apply.
You don’t want to do that for anything you want to be able to vary, but they do something similar with a “soul document” for things they always want to apply.
In this token-mania frenzy that has taken hold of the industry, I guess solutions like "soul document" and "system prompts" will continue for a while, and once the industry matures a bit we'll go back to things like LoRA[1] and control vectors[2][3].
The other explanation may be that these AI labs may be expecting more government scrutiny, and "here's a document" would probably go better than "here's some vector representation of our values" when talking to politicians.
[1] https://arxiv.org/abs/2106.09685
[2] https://vgel.me/posts/representation-engineering/
[3] https://transformer-circuits.pub/2024/scaling-monosemanticit...
Is there a reason a document could not be converted to vectors via embedding, and you’d have both?
EDIT: I see, the control vectors operate more directly upon the model, in a way embedding vectors don’t quite have access to.
If it's a fine tuning step at the end, why is the need for it to vary a problem? Can't you run the fine tuning, test for regression, and deploy the weights in a day?
I think the more likely reason is it doesn't work as well as in context learning. Otherwise they would prefer to avoid polluting context and degrading performance.
Fine tuning isn't the same and doesn't have the same effect as selecting input tokens.
Does there exist a model X that behaves exactly as a model Y with context Z? Maybe, but it's not trivial to achieve and might possibly be convoluted and more expensive.