Even for older models I was (and am) of the opinion that a lot of the context-overstuffing cruft people have been wrapping around their LLM usage is more of an RNG-impacting (but not always for the better) lucky charm rather than it being universally helpful.
And a lot of people latched on to it as a form of self-soothing.
"I may not write much code anymore, but I can still be an expert prompt engineer!"
Not really. The big change in newer models is the amount of RL relative to pretraining. RL makes the models stable by collapsing distributions around desired outcomes. The old tweaks like giving the agent a role ("You are an expert XYZ engineer") or stuffing the context with engineering idioms have effectively been RL'd into negligence, but they still waste tokens.
You can see this visually in older image generation models. Stable Diffusion 1.5 would produce wildly different images based on slight variations in prompt and seed, but the latest image gen models are nearly seed indifferent and can tolerate a decent amount of prompt tweaking while staying "consistent"
> is more of an RNG-impacting (but not always for the better) lucky charm rather than it being universally helpful.
I wrote this last year, it's still true:
--- start quote ---
https://dmitriid.com/prompting-llms-is-not-engineering
In reality these are just shamanic rituals with outcomes based on faith, fear, or excitement. Engineering it is not.
--- end quote ---
There are some prompts useful for the user like brainstorming [1] but on the whole it's nothing but lucky charms
[1] brainstorming from superpowers: https://github.com/obra/superpowers