Very little in terms of the layers they use. Calling it now, they are using global layers everywhere, making the model basically unusable due to high KV Cache use.