> I can not fathom a LLM could conceive from scratch concepts like the real numbers by Dedekind.

What stops you? I can fathom it alright.

I am always suspicious of these arguments like "a machine won't do thing x which humans can". These always basically pre-suppose that humans aren't machines. If that's what you're arguing for, say it outright.