Is there anything that LLMs (or any software for that matter) could imaginably and theoretically do but do humans qualitatively cannot do?
I don't really know what I'm talking about but I imagined the difference was always quantitative (in a nutshell: they need less time).
Or almost any invention for that matter.
Washing machines can’t do anything humans do, they just remove labour. Trucks don’t do anything longboats can’t, it just need less labour and time/effort to build roads rather than canals. Computers can’t calculate anything humans can’t dry run by hand etc.
Everything in reality is about reducing time/effort/material/cost or achieving more with less resource.
Yes. I'd also argue there isn't anything humans couldn't do in theory, other than things strictly prohibited by known laws of physics. It's been pretty conclusively shown in the last 100 years, we're past tipping point of civilizational knowledge and scientific infrastructure.
What stops us from doing any specific thing is always allocation of resources - there's finite amount of time/effort/material/labor available, and past trivial amounts we need more and more people to agree on some allocation. Reduction of time/effort/material/labor costs of any thing is what moves it closer from "infeasible" to "feasible" for us to do. But again, short of violating laws of physics, it was never "impossible".
And then you realize AI was made by humans...
What I mean is that "can be done by humans" is ill defined. Are you allowed to use pencil and paper? That's not part of your body? Do you have to be strapped into a coffin for it to count as human-done? How about wheels, sticks and stones? Allowed to use those tools? How about an abacus? Or is electricity the line to draw?