> With MoE it's low probability.

For even the sparsest MoE open models, having more than a handful of inferences in the batch is enough to make it more likely than not that you'll get some MoE weight reuse within any given layer. This assumes totally random sampling, ignoring any cross-request correlation that would push that probability even higher in many practical scenarios.

> With MTP it becomes _extremely_ low probability.

This is actually right, MTP is only ever worthwhile in very special cases involving either dense models or extremely wide batching of MoE ones that somehow still leaves unused room for parallelization (which AIUI would involve an assumption of very abundant compute with very limited memory bandwidth).