Are you IoT devices ALSO used by humans directly, where they would be forced to have some admin permission to do their work if there was no MDM system?

MDM are clearly a possible SPOF for certain attack vectors, but are also the only defense against others (unless you want to hire a legion of IT helpdesk specialists)