I don't think that argument holds water, because the scope of what can be reasonably approximated by matrix multiplication is extremely broad (especially when the matrix multiplication is also paired with a source of randomness).