I mean I mostly agree but the part that I'm warning against is "you can comfortably deal with higher dimensions [visualizations]". You should always feel a bit uncomfortable in high dimensions least you be lulled into a false sense of security. In fact, this is frequently a pain point for people as they advance. They over leveraged the utility of visualization and hit a wall abstracting those same concepts to domains where you can't do any visualization. Just look at how many people claim to be able to visualize the 4th dimension and point at a tesseract.
I'm not disagreeing imagination and visualization aren't critical tools. They are. But read my comment again like a mathematician. If you misread you'll overfit the wrong concept.
You are just placing too much emphasis on caveats and limitations rather than focusing on the usefulness of the technique itself. Also techniques like Dimensionality Reduction exist to help tame higher dimensions.
None of your cautions are really that big of an impediment in practice and i would not want students to take away the wrong message from these comments. Mathematicians/Practitioners know when to switch from simple geometry to algebra and that is taught as a matter of course.
As an example, the farin and hansford book Practical Linear Algebra: A Geometry Toolbox that i mention at https://news.ycombinator.com/item?id=45060305 precisely takes this approach starting with visualizing vectors in 2D/3D geometry and then moving on to higher-dimensions. Its 1st edition was actually named The Geometry Toolbox: For Graphics and Modeling.
I highly encourage you to talk to a mathematician about this stuff. In high dimensions everything is all about caveats and edge cases. Your {1,2,3}D intuition leads you astray rather than helps you closer to the answer. In fact, math is specifically an overly pedantic because nuances are so critical.
I'll use an example that we can reason from low dimensions but that many people make the wrong connections. What is the probability that in N dimensions two i.i.d randomly drawn vectors are orthogonal? We can easily intuit in low dimensions that this is a pretty unlikely event. But in high dimensions exactly the opposite is true. An easy mistake to make is to not think about this question from a continuous perspective, by reframing the question to ask what the probability that the angle between to i.i.d drawn vectors are within some epsilon bound. If we don't, then the answer is trivial in all cases: 0. A good student would be able to reason through this from looking at the 2D case and 3D case but this is only a sample of 2 and it would be naive to generalize such a notion without doing the proof.
So let's look at something a little harder. A classic problem is looking at the volume of a n-ball. Our 2D and 3D understanding make this problem look easy. The volume of the 3D ball is much larger than the 2D ball. We can even verify this my analytically solving the 4D case. Same with the 5D case! But that's when things go awry. For the unit n-ball, 5D is where the maximum volume is. By 10 dimensions you have less volume than our 2D case and by 20 dimensions your volume is quickly converging to 0.
Now let's look at something wild. Consider a hypersphere with radius r inscribed within a hypercube with sides of length 2r, what proportion of the hypersphere resides within the hypercube for a given dimension D? This sounds like an absolutely dumb question if we use or 2D and 3D logic. What amount of the sphere is inside the cube? 100%! It's right there in the problem description! Right? But this is entirely wrong. As dimensionality increases, very quickly, ~0% of the hypersphere resides within the hypercube and this is all due to the fact that hypercubes and hyperspheres have very different representations in high dimensions. Yes, there is still visual understanding we can draw to help reason, but again, this requires us to look at the problem from a different vantage point (the key thing I stressed in my original post btw). We have to look at the dual of the problem and instead ask "what is the ratio of the volume of a hypersphere to the volume of the hypercube it is inscribed within?" This will help but there's also a reason I put this question after the previous one because we know that there is not a clean linear relationship here and that at least with the sphere the volume increases and then decreases. Our visual intuition only helps if we can recognize that the relationship has everything to do with the corners of the cube.
It is easy to look at these 3 cases and see how visual intuition from low dimensions can help, but be careful about post hoc trivialization. It's easy when you know these things and after they have been told to you but they are not so clear when you're being presented with the problem. All 3 of these problems are deeply related to the nearest neighbor problem and the concentration of measure. Where we run into the problem that it becomes nearly impossible to distinguish the nearest point to the furthest point. Leveraging our lower dimensional visualization can provide help but it is important to remember that they are a small part of the much larger story here. Which of course should make sense because as the dimensionality grows our 2D/3D slices of those are a smaller and smaller portion.
This is why I referenced the blind man and the elephant. Not because you can't use visualization to aid you, but because to see the elephant there you have to sample that elephant at many different places. And just like the blind man, you don't know how big that elephant is and where things are changing.
But this stuff isn't going to help with even more abstract concepts. Like how there is no division algebra in 3D and how the largest is represented via the Octonion. How by moving to quaternions we lose commutativity and how moving to octonions we lose associativity. These are very fundamental features of math that we are losing while moving to higher dimensions.
So I'm not sure why you're pointing to Farin and Hansford's book. Their highly concentrated on low dimensions and only briefly discuss generalization at the very end. They miss many important concepts. Not because the book is poorly written but because they are far out of scope from what is being taught. My caution about high dimensional spaces does not mean I don't absolutely love Needham's books on Visual Differential Geometry and Forums or his book Visual Complex Analysis or even Carter's inspired Visual Group Theory. These are masterpieces and books I highly recommend. But it is about what is being communicated. We work frequently in lower dimensions and there is a wealth of information there. But this does not mean higher dimensions are not absolutely littered with pitfalls and paradoxes. The true study of high dimensional spaces is relatively new and drove people like Cantor and Boltzmann insane. Every small mistake that is easy to make in low dimensions becomes exponentially more important in high dimensions. Yes, there are dimensional reduction techniques and these help with varying degrees (utility being critically dependent upon the latent dimensionality of the data) but every single one of these comes with major concessions. There is often even very bad science performed due to this lack of understanding. Techniques like t-SNE and UMAP are routinely unknowingly abused to draw inaccurate conclusions about these spaces, as it is possible to perform nearly any clustering you wish (see Lior Pachter's Picasso). Even PCA is frequently abused in this area due to there being subtle assumptions being invalid on different problems.
My point is: you absolutely should warn people that there are many pitfalls when dealing with high dimensions. Anyone brushing these differences off as inconsequential nuances is just ignorant of this rich space. I don't blame anyone for being unaware of these things because it requires some very complex mathematics to truly understand, and we're talking about math that the average scientist is never going to be introduced to. You likely won't even see this in a given science PhD. Not because it doesn't matter, but because it is just very difficult math. And we learn math through a game of telephone that is often hyper focused (not a great way to learn it tbh)
I am not sure what you are trying to argue with this gish gallop of a comment filled with strawmen. The usefulness of Geometry in visualizing and building intuition in Mathematics is a given. Equally, its limitations when it comes to higher dimensions is also known. But for a beginning student (the context of this thread) it is the former which must be emphasized before he has developed the requisite mathematical maturity to deal with the latter. You do not want to scare away students from the important practical subject of Linear Algebra by focusing too much on its complexity in the initial stages of learning.
I highly recommend that you talk to/read the works of some mathematicians who are also involved with the research in the teaching of mathematics. There has been a lot of studies/research done on how to teach students build mathematical intuition using Geometry (and other graphical means) and in particular w.r.t. Linear Algebra. Some relevant references are given below for your edification; those by Harel are in particular, noteworthy.
The farin and hansford book mentioned earlier is one of the best introductory texts for beginners since it starts with building geometrical intuition for the study of the linear algebra. Pair it with some more rigorous text like for example; Linear Algebra Pure & Applied by Edgar Goodaire and the student can comfortably learn a pure mathematical approach with a clear understanding of what he is doing. Goodaire is a "pure mathematician" who specifically starts with 2D/3D graphs and then transitions to n-space with all necessary definitions/theorems/proofs included in a very accessible writing style. This is the ideal approach to the teaching and learning of Linear Algebra i.e. use geometry (and other graphical means) in conjunction with algebra, and where applicable, include caveats/limitations, but always with the aim and focus of teaching concepts and not mere computation.
References:
1) Intuition in Science and Mathematics: An Educational Approach by Efraim Fischbein - https://link.springer.com/book/10.1007/0-306-47237-6
2) The efficiency of visualization through geometry at mathematics education: a theoretical framework by Sefa Dundar et al. - pdf at https://www.sciencedirect.com/science/article/pii/S187704281...
3) Should we teach linear algebra through geometry? by Ghislaine Gueudet-Chartier - pdf at https://www.sciencedirect.com/science/article/pii/S002437950...
4) Geometric thinking in a n-space by Ghislaine Gueudet - pdf at https://shs.hal.science/hal-00529609/
5) Students' understanding of proofs: a historical analysis and implications for the teaching of geometry and linear algebra by Guershon Harel - pdf at https://www.sciencedirect.com/science/article/pii/S002437959...
6) Promoting Linear Algebraic Reasoning among Students: Affordances and Challenges by Guershon Harel - pdf at https://www.tandfonline.com/doi/full/10.1080/10511970.2024.2...