Yes, this problem space has inherent complexity, but no, this inherent complexity does not require Github's insanely insecure defaults and incoherent security model.

As a practical step, one could try using webhooks to integrate their github repo with literally any other CI provider. This would at least give you a single, low-coupling primitive to build your workflows on. It would not, in any way, eliminate the domain's inherent complexity (secrets, 3rd party contributions, trusted publishing, etc.), but it starts out safe because by default it doesn't do anything - it's just an HTTP call that gets fired under certain conditions.