Thank you for not saying "language", but "text".

It's true, but it's also true that text is very expressive.

Programming languages (huge, formalized expressiveness), math and other formal notation, SQL, HTML, SVG, JSON/YAML, CSV, domain specific encoding ie. for DNA/protein sequences, for music, verilog/VHDL for hardware, DOT/Graphviz/Mermaid, OBJ for 3D, Terraform/Nix, Dockerfiles, git diffs/patches, URLs etc etc.

The scope is very wide and covers enough to be called generic especially if you include multi modalities that are already being blended in (images, videos, sound).

I'm cheering for Yann, hope he's right and I really like his approach to openness (hope he'll carry it over to his new company).

At the same time current architectures do exist now and do work, by far exceeding his or anybody's else expectations and continue doing so. It may also be true they're here to stay for long on text and other supported modalities as cheaper to train.