If I understand correctly the basic structure hasn't changed since Transformers were invented. It's just gotten bigger and the models have had better training and also some optimizations. I don't think this one has optimizations.