The BERT model, which uses the transformer architecture, was deployed by Google for every English language Google search by the end of 2019.

This is about concurrent with OpenAI's release of GPT-2. But GPT-2 was not really a product.