Fourier theory says that any sound, however complex, can be synthesized by summing sines and cosines. That's what an LLM does, if you twist the metaphor enough. It synthesizes complex outputs from simpler basis functions that are, or should be, uncopyrightable.
The fact that it inferred those basis functions from studying copyrighted works doesn't seem relevant. Nor does the fact that the "Fourier sums" sometimes coincide with larger fragments of works that are copyrighted. How weird would it be if that didn't happen?
Of course it's relevant. How copyright infringement happens doesn't actually matter, all that matters is that the infringement happened.
If I painstakingly recreate A New Hope frame by frame, pixel by pixel, that's infringement. Even if I technically used 0 content from the original.
Nobody is doing that, though. You might get a watermarked screenshot or stock photo now and then, or a couple of mostly-verbatim paragraphs from Harry Potter.
In any case, if the copyright mafia insists on butting heads with AI, they'll find that the fight doesn't quite play out the way it has in the past.