I guess you have to know which experts to keep “hot” in ram, which you can’t know beforehand, so there wouldn’t be much gain.
I guess you have to know which experts to keep “hot” in ram, which you can’t know beforehand, so there wouldn’t be much gain.
I do wonder if there are some experts that are more likely to be hit. So if the normal optimised setup runs in 12GiB an you have 4GiB extra to spare, you could say “promote the most used X experts to this stable (old gen in GC parlance) region and don’t swap it out.
Maybe you could even do something like profiling and remember over multiple sessions (per project/workspace) what the most used agents are and load those up before hand.