If someone made an API to build a data center? Or made an API to keep their lights on? What then?

There are already such "APIs", which can be operated by a combination of textual communication and money. Or by illicit security vulnerabilities. You might notice that LLMs are pretty good at that now.

We're building something that has the capabilities of humans. There is no X for which it's persistently safe to assume humans can X and AI cannot X.