> Mathematicians like to see their matrices laid out on paper this way (with the array indices increasing down the columns instead of across the rows as a programmer would usually write them).
Could a mathematician please confirm of disconfirm this?
I think that different branches of mathematics have different rules about this, which is why careful writers make it explicit.
Not a mathematician, just an engineer that used matrix a lot (and even worked for MathWorks at one point), I would say that most mathematicians don't care. Matrix is 2D, they don't have a good way to be laid out in 1D (which is what is done here, by giving them linear indices). They should not be represented in 1D.
The only type of mathematicians that actually care are: - the one that use software where using one or the other and the "incorrect" algorithm may impact the performance significantly. Or worse, the one that would use software that don't use the same arbitrary choice (column major vs row major). And when I say that they care, it's probably a pain for them to think about it. - the one that write these kind of software (they may describe themselves as software engineer, but some may still call themselves mathematicians, applied mathematicians, or other things like that).
Now maybe what the author wanted to say is that some language "favored by mathematician" (Fortran, MATLAB, Julia, R) are column major, while language "favored by computer scientist" (C, C++) are row major
Languages that don't have multidimensional arrays tend to have "arrays of arrays" instead, and that naturally leads to a layout where the last subscript varies fastest. Languages that do have multidimensional arrays can of course lay them out in any order.
ah, so you can get row vectors with a type cast in C but not column ones. While in Fortran and friends is the converse (if they casted). Yep that is more mathy. Linear map evaluation is linear combination of columns.
EDIT: type cast or just single square bracket application in C
This is one of those things that's perennially annoying in computer graphics. Depending on the API you can have different conventions for:
- data layout (row-major vs. column major)
- pre-multiplication vs. post-multiplication of matrices
Switching either of these conventions results in a transposition of the data in memory, but knowing which one you're switching is important to getting the implementation of the math right.
And then on top of that for even more fun you have:
- Left- vs. right-handed coordinate systems
- Y-up or Z-up
- Winding order for inside/outside
There are lots of small parity things like this where you can get it right by accident but then have weird issues down the line if you aren't scrupulous. (I once had a very tedious time tracking down some inconsistencies in inside/outside determination in a production rendering system.)
What I suspect he really means is that FORTRAN lays out its arrays column-major, whilst C choose row-major. Historically most math software was written in the former, including the de facto standard BLAS and LAPACK APIs used for most linear algebra. Mix-and-matching memory layouts is a recipe for confusion and bugs, so "mathematicians" (which I'll read as people writing a lot of non-ML matrix-related code) tend to prefer to stick with column major.
Of course things have moved on since then and a lot of software these days is written in languages that inherited their array ordering from C, leading to much fun and confusion.
The other gotcha with a lot of these APIs is of course 0 vs 1-based array numbering.
The MKL blas/lapack implementation also provides the “cblas” interface (I’m sure most blas implementations do, I’m just familiar with MKL—BLIS seems quite willing to provide additional interfaces to I bet they provide it as well) which explicitly accepts arguments for row or column ordering.
Internally the matrix is tiled out anyway (for gemm at least) so column vs row ordering is probably a little less important nowadays (which isn’t to say it never matters).
Not a mathematician, but programmers definitely don't agree on whether matrices should be row-major or column-major.
I'm surprised we even agree that they should be top-down.
At this point might as well make them match the x/y convention, with first index increasing to the right, and second index increasing from bottom to top.
Programmers don’t agree on the x/y convention.
Mathematician here. I never heard that.
(In many branches the idea is that you care about the abstract linear transformation and properties instead of the dirty coefficients that depend on the specific base. I don't expect a mathematician to have an strong opinion on the order. All are equivalent via isomorphism.)
I'm an applied mathematician and this is the most common layout for dense matrices due to BLAS and LAPACK. Note, many of these routines have a flag to denote when working with a transpose, which can be used to cheat a different memory layout in a pinch. There are also parameters for increments in memory, which can help when computing across a row as opposed to down a column, which can also be co-opted. Unless there's a reason not to, I personally default to column major ordering for all matrices and tensors and use explicit indexing functions, which tends to avoid headaches since my codes are consistent with most others.
Abstractly, there's no such thing as memory layout, so it doesn't matter for things like proofs, normally.
I'm a mathematician. It's kind of a strange statement since, if we are talking about a matrix, it has two indices not one. Even if we do flatten the matrix to a vector, rows then columns are an almost universal ordering of those two indices and the natural lexicographic ordering would stride down the rows.
Yes. I think what all mathematicians can agree on is that the layout (and the starting index! :-) is like this:
Recently graduated math student here. The definition of the "vec" operator which turns a matrix into a vector works like this, stacking up columns rather than rows.
https://en.wikipedia.org/wiki/Vectorization_(mathematics)
Most fields of math that use matrices don't number each element of the matrix separately, and if they do there will usually be two subscripts (one for the row number and one for the column number).
Generally, matrices would be thought in terms of the vectors that make up each row or column.