Is it possible to supplement the model with a diff for updates on modular memory, or would severely impact perf?

I imagine you could do something like a LORA

this design at 7 transistors per weight is 99.9% burnt in the silicon forever.