The people OP mentioned about "just want AI" .

The pain point they raised is this is too complicated for people who just want to get started, that is not true anymore.

It is certainly fun to fine-tune and setup if you like do something like that, however the need to do it hardly is a barrier for those who don't want complexity as OP imagines.

Lower level API/interfaces should not be a barrier for people if they are apply framing that way. More and more people are thinking agent native so this is not really a issue.