the set of all matrices of a fixed size are a vector space because matrix addition and scalar multiplication are well-defined and follow all vector space axioms.

But be careful of the map–territory relation.

If you can find a model that is a vector space, that you can extend to an Inner product space and extend that to a Hilbert space; nice things happen.

Really the amazing part is finding a map (model) that works within the superpowers of algorithms, which often depends upon finding many to one reductions.

Get stuck with a hay in the haystack problem and math as we know it now can be intractable.

Vector spaces are nice and you can map them to abstract algebra, categories, or topos and see why.

I encourage you to dig into the above.