I’ll throw another example.

Do the work of indexing, or summarization or compression vs just do it retrieval on the raw records.

It could be embeddings, or presorts or indices.

But I think the parent poster is correct. There’s an inherent write vs read tradeoff as well as memory vs compute. Maybe with some generalization vs specialization.

Idk if im explaining properly. But you can always front load more effort on a narrower problem space and then utilize it on that space downstream.

I think with your llm example there is no code/data split. The llm is caching a solution to a narrow problem by writing the code.