At this point AI basically means "we didn't know how to solve the problem so we just threw a black box at it".

I disagree. More often than not is "We know how to solve the problem, and the solution is some linear algebra"

I disagree with both of you.

It's not about linear algebra (which is just used as a way to represent arbitrary functions), it's about data. When your problem is better specified from data than from first principles, it's time to use an ML model.

I think what you're expressing is also known as "the Bitter Lesson".