Yes. Quantum mechanics for example is not something that could have been thought of even conceptually by anything “locked in a room”. Logically coherent structure space is so mind bogglingly big we will never come close to even the smallest fraction of it. Science recognizes that only experiments will bring structures like QM out of the infinite sea into our conceptual space. And as a byproduct of how experiments work, the concepts will match (model) the actual world fairly well. The armchair is quite limiting, and I don’t see how LLMs aren’t locked to it.

AGI won’t come from this set of tools. Sam Altman just wants to buy himself a few years of time to find their next product.