Simple version: In instances wherein the otherwise statistically chosen next word is a "toss-up", watermarking removes the randomness by imposing specific choices, determined by a key. This then becomes a detectable pattern when scanned with the key (stastically—detection itself is probabilistic).
>use it to store arbitrary information
No additional data is embedded. The range of available data is constrained by the text being generated (i.e. the sets of "next words" per text).
From what I’ve read, they won’t be imposing specific choices, but using a different (biased) RNG for those “toss-up” choices. With enough sampling, you could detect if the RNG was biased or not.