Yeah, my impression is that even the best LLMs are pretty bad at analysing big data directly.

It's not even the hallucinations that are the biggest problem, it's more that they're so bad at managing their context windows that they end up ignoring huge chunks of the data without realising that they're doing it. You end up with a result that looks plausible but is often extremely misleading.

A better approach seems to be getting the LLM to write code to use more traditional analysis techniques (eg. iterative k-means, or whatever). That way you can at least be confident that you've looked at all the data rather than just a small slice of it.