My vague understanding of the context window limitation is that it is largely a constraint of the model architecture. So maybe they have special extra long ctx, but it might just be a hard limit of the model itself.