> I think that when the singularity occurs all of the problems in physics will solve, like in a vacuum, and physics will advance centuries if not millennia in a few pico-seconds
It doesn't matter how smart you are, you still need to run experiments to do physics. Experiments take nontrivial amounts of time to both run and set up (you can't tunnel a new CERN in picoseconds, again no matter how smart you are). Similarly, the speed of light (= the speed limit of information) and thermodynamics place fundamental limits on computation; I don't think there's any reason at all to believe that intelligence is unbounded.
The "singularity" can be decomposed into 2 mutually-supportive feedback loops - the digital and the physical.
With frontier LLM agents, the digital loop is happening now to an extent (on inference code, harnesses, etc), and that extent probably grows larger (research automation) soon.
Pertinent to your point, however, is the physical feedback loop of robots making better robots/factories/compute/energy. This is an aspect of singularity scenarios like ai-2027.
In these scenarios, these robots will be the control mechanism that the digital uses to bootstrap itself faster, through experimentation and exploration. The usual constraints of physical law still apply, but it feels "unbounded" relative to normal human constraints and timescales.
A separate point: there's also deductive exploration (pure math) as distinct from empirical exploration (physics), which is not bounded by any physical constraints except for those that bound computation itself.
Kind of, I mean you have to verify things experimentally but thought can go a very long way, no? And we're not talking about humans thinking about things, we're talking about an agent with internet access existing in a digital space, so what experiments it would do within that space are hard for us to imagine. Of course my post isn't meant to be taken seriously, it's more of a fun sci-fi idea. Also I'm implying not necessarily reaching the limits of the things you mentioned, but rather, just taking a massive step in a very short time window. Like, the time window from the discovery of fire to the discoveries of Quantum Mechanics but in a flash.
> what experiments it would do within that space are hard for us to imagine
The only thing you could do in a "digital space" (a.k.a. on a computer) is a simulation. Simulations are extremely useful and help significantly with designing and choosing experiments, but they cannot _replace_ real experiments.
> Like, the time window from the discovery of fire to the discoveries of Quantum Mechanics but in a flash.
And my point is that there's no good reason to think this is possible and many to think it isn't.
> it's more of a fun sci-fi idea
It's being presented as extremely serious possibility by people who stand to gain a _lot_ of money if other people think it's serious... that's the point of the linked post. Unfortunately, these AI boosters make it very difficult to discuss these ideas, even in a fun sci-fi way, without aggravating the social harms those people are causing.
You say that, but someone at CERN has spent at least ten minutes thinking about how they could expose the Haldron Colider as an MCP server.