> but they can't keep getting better indefinitely

Maybe? We dont really know this right? People have been saying this for 5 years now and the models are still getting better. The companies running the frontier models have already scraped everything on the web, but the models are still getting better, even if it's only marginally better, with each release. Maybe eventually some company will actually achieve AGI/ASI, who knows..