I think GP is using a different level of abstraction from yours in their metaphor.

You are saying:

Pre-Training == Everything you store in your memory throughout your life. Model weights == The lessons you learned

Context == whatever you're currently thinking about

One inference run == one thought

They are saying:

Pre-Training == building the DNA template of human brain through millions of years evolution. Model weights == Human DNA

Context == Everything you store in your memory throughout your life, plus whatever you're currently thinking about

One inference run == One human life. One instance == one human

Applying their metaphor, your sentence becomes:

> If a new physics breakthrough gets published today, no existing DNA structure will be able to fully integrate it - beyond an individual person. If I put the paper in my mind by learning it, and it isn't in yours, the DNA of human species stores nothing. It won't retain it past my lifetime.

> The human species is trained (through evolution), it doesn't learn.