I’ve been craving some Physics courses since it’s been about a decade since I was in school. I picked up a Classical Mechanics book to get back into the swing of things and of course it went through some basic linear algebra. It’s been a while since I’ve thought about the dot product of two vectors.
You know what blew me away though? Not one textbook I looked in mentioned “why” the dot product is important; that is it’s useful for determining the similarity of two vectors. They all focused on the mechanical details of computing the dot product, but never spelled out the reason it can be useful. I went through a few other resources before I broke down and had a little chat with ChatGPT to discuss the meaning behind it and it makes perfect sense after that.
In comparison to when I was in college, things are much slower paced so I can take the time I need to ensure I have a full grasp of a concept before moving forward. I guess all of this is to say that as I’ve continued forward through more concepts I keep finding that the books I’m reading offer a mechanical view instead of a holistic view of the material. This feels like the biggest issue with most math books I’ve read and it makes me wonder where books that offer more semantic meaning of concepts instead of recipes exist.
It's not just the books it is the whole method of teaching. I remember learning the steps to calculate an eigenvector without a single comment on why one would ever want to do that. I think it is done so that the educator can claim "this course teaches all of calculus and linear algebra and quantum mechanics". To actually explain things properly would require more modest course goals.
I think I only learned linear algebra about 3 or 4 years after I graduated. I learned how to do the computations during the course but the teacher was so bad I had no idea what anything was for. Could've been an IQ test course for all it mattered. Here transpose this matrix now. Ok.
I still would argue that method of teaching is perfectly fine.
You cannot simply explain to someone complex stuff - best way is to let people grind through to build their own understanding.
Parent poster wrote that "it’s useful for determining the similarity of two vectors" - now I would ask why do I need to determine similarity of two vectors as it does not mean much to me - if I would be grinding through math problems I would most likely find out why, but there is no way I could understand and retain it when someone would just tell me.
I was taught linear algebra and multivariate calculus as a business major. They could hardly justify why they were teaching it in that context - they were weeder courses - but I always wished they had at least tried to give us a hint of applications. Nothing, it was all algebra for the sake of algebra. Atrocious.
A long time ago, when I was in high school, we had an introductory course to differentials and integral calculus. When I asked what the purpose of integration was, the teacher shouted that I should save the stupid questions for my parents ... She was a shit teacher for various reasons, but that was the day that I lost my drive for maths.
It wasn't until years later that I found that it was all about "the area under the curve" and why that would be useful. At no point in those high school classes did we ever work a practical example. I was pissed off all over again when I found out how useful that stuff could be, and how much I'd missed out on.
I'm sure most teachers mean well, and I'm sure most of them try. But by god there are some truly awful twats out there who should never set foot in a classroom again.
> When I asked what the purpose of integration was, the teacher shouted that I should save the stupid questions for my parents
What an awful person. Chances are she was getting defensive and covering for her own lack of understanding. If I were a parent, I would confront her about that, not least of all her contempt for students and for learning, but toward parents.
Teachers don't know everything, and when they don't know, they should be able to admit that without hesitation or defensiveness. This sets a good example in general, of humility, instead of inculcating the notion that life is about having all the answers, or rather, pretending to have all the answers. All this does is set up people to become imposters. Of course, if you're teaching calculus, you should have at least a basic grasp of the material, and if you don't, you should say so, so that you've not put in a position where you have to teach it.
> I'm sure most teachers mean well, and I'm sure most of them try.
I think it is generally accepted that primary education isn't exactly packed with the best candidates, both from the point of view of pedagogical ability as well as mastery of the material.
We had a good math teacher. There was a formula he just told us to memorize, the class asked how it worked but he just said we don't need to know why or how, just like we don't know how a calculator works. What he didn't know was that the class last week in electronics was about how calculators work.
He had to confess he didn't know why or how either of them works, he just uses them :-)
I took both applied math and pure math (+ comp. sci) at uni, and was similarly frustrated at the applied math, which was all triple integrals and partial derivatives etc, and exactly zero examples given of where this stuff would be useful. I just stopped attending the class, since the downside of doing so was minimal. To this day I still don't know where one might apply triple integrals, although no doubt there are uses.
Perhaps a bit oddly I didn't have any problem with the pure math prof. also not offering any use cases, although of course it also does!
In high school trigonometry I am sure I was clear that sine and cosine formed the circle. How could I not?
But that fact’s significance and too obvious simplicity, with all its ramifications, only hit me deeply and profoundly when a year later I realized I could use those functions to draw a circle on a screen.
Before that they were abstractions related to other abstractions that I had to memorize to pass a course.
To this day I am frustrated when reading papers about abstract algebraic relations and other such concepts, without even a sentence or two discussing any intuitive way to think about them. Just their symbolic relations.
I appreciate that in the game of math that view becomes natural. But most of us learn math with additional motivations and are interested in any perspective that highlights potential usefulness or connection to the real world. Many of us mentally organize our knowledge teleologically.
Yet even when usefulness is known to exist, it is often neither mentioned or referenced. Or even considered relevant.
Edit: the same goes for not showing a single concrete example of an abstract concept. A kind of communication that would unlock many mathematical papers to a much larger audience of intelligent and relevant readers.
I’ve been teaching mathematics for over 30 years at the community college level. Most people at the time of taking a course don’t have a sophisticated enough understanding of math to really appreciate “intuitive explanations” because they don’t have intuition.
Take parametric curves. I explain that they generalize the concept of a function. Every function can be parametrized in a trivial way. They don’t really understand this concept. They have a hard time parametrizing a function and do so only becuase of a formula.
The fact is most people need to go through the mechanical process of doin g before they can get to a point of understanding. It takes almost the entire semester for me to convince beginning algebra students that the reason that 2x + 3x is 5x is because of the distributive property. And when they do understand it they don’t understand why that is important.
Later on when things click for someone they will often say things like, “Why didn’t they just tell this when we took the course?” Usually we did. You just didn’t have a sophisticated enough understanding of things to grok it at the time you took the course.
(oddly the Calculus book was published second, so I guess I'll need to re-read it after I finish the trigonometry book)
Hopefully, this will provide me with a sufficient grounding in conic sections that I can solve my next CNC project with a reasonably efficient set of calculations (trying to do it using my rudimentary understanding of triangles from trigonometry had me 4 or 5 triangles deep, barely half-way to the final point I needed, and OpenSCAD badly bogged down performance-wise).
> To this day I am frustrated when reading papers about abstract algebraic relations and other such concepts, without even a sentence or two discussing any intuitive way to think about them. Just their symbolic relations.
If you’re talking about research papers, that’s just because they’re written for domain experts and aren’t really for giving you intuition. They’re written in a deliberately terse (one might say elegant) style to convey the research findings in formal mathematical language and nothing much else. If you want to gain an intuitive grasp of things, read a proper textbook in detail or play around with the ideas on paper. Or both!
I guess the reason is that once you’ve acquired the intuition, having the literature cluttered up with the same explanations again and again becomes clunky and increases the volume of material to be sifted through when you’re just looking for a result you need in your research and don’t need all the extra chatter. It’s just cleaner that way. But to an outsider it does look more opaque. It’s a trade off.
Not teaching the Why is such a sin!
I didn't understand calculus properly at all until I read Steven Strogatz' brilliant book Inifinte Powers, which not only explained the why but the history of why. 10/10 book for me.
Modern education is grounded in a different worldview than the classical liberal arts[0]. The classical liberal arts are so-called because they are freed from the burden of having to be practical or economic in nature (which is not to say they could not or did not incidentally have practical application), intended to produce a free man. Here, too, by "free" we mean free to be good, that is, more fully human, not what we mean by freedom today as doing whatever you happen to feel like doing, a recipe for enslavement, misery, and despair, and therefore directly opposed to the good and to becoming more human.
Opposed to the liberal arts were the illiberal or servile arts. These are necessary and good, of course, but necessarily inferior to the liberal arts because their end is not truth or formation; they are instead practical, concerned with effecting some kind of economic end. The point here is not to disparage, but to understand how all of these are related and ranked according to a "for the sake of" relation. A human being doesn't exist to eat, he eats to exist, for instance.
Modern education is very much oriented toward the servile arts, and what passes for the liberal arts today is anything but the classical notion.
The point is that modern education is less interested in leading to understanding, realizing virtuous habits, and leading to freedom, and more interested in churning out workers. Workers don't ask "why" (though we can agree that those who do can, guided by prudence, contribute more economically). Indeed, that is perhaps the key difference between classical science and modern science: the emphasis of the former is truth, while that of the latter is control of nature. Of course, it isn't that you must choose absolutely between understanding and effectiveness, and the classical tradition does not claim either that study precludes work. Study often requires work, for sake of preparing the way for truth. Rather, it is that the end of the modern educational tradition is different from that of classical education, and this end determines the form of the pedagogical methodology. It is a difference in anthropology, of the vision of man.
All men work, but what do they work for? Do they work for work's sake, or perhaps to make money to satiate their base appetites (modern view)? Or do they work in order to be free to pursue higher ends[1]?
That seems pretty surprising to me. The lower level/physics books I've seen introduce the dot product with both a geometric and algebraic definition, and show they're equivalent in 2-3 dimensions. The "how" is the algebraic definition and the "why" is the geometric definition.
It's not really that it measures similarity. Physics isn't interested in that. It's that it tells you lengths and angles, which you need in all sorts of calculations. In more advanced settings, a dot product is generally taken as the definition of lengths and angles in more abstract spaces (e.g. the angle between two functions).
In machine learning applications, you want a definition of similarity, and one that you could use is that the angle between them is small, so that's where that notion comes in. A more traditional measure of similarity would be the length of the difference (i.e. the distance), which is also calculated using a dot product.
Angle is a measure of similarity (well, distance/nearness).
In physics, the dot product is used to losslessly project a vector onto an orthonormal basis, and the angle measures how much of the vector's magnitude is distributed to each bases vector.
The angle can be defined in terms of the dot product, because you don't need the angle (as in a uniform measure of rotation) in order to compute important physical results.
This is why 1b3b is so popular. Instead of teaching the mechanics he teaches the intuition.
With that being said, I do remember my math and physics teachers in high school spend lots of time talking about the why and intuitions and let the books state the how.
> Not one textbook I looked in mentioned “why” the dot product is important; that is it’s useful for determining the similarity of two vectors.
Most textbooks motivate it by the angle between the vectors or as projections (e.g., for hyperplanes). Numerics-focused ones will further emphasize how great it it is that you can compute this information so efficiently, parallelizable etc..
Later on it will be about Hilbert space theory or Riemannian geometry and how having a scalar product available gives you lots of structure.
> This feels like the biggest issue with most math books I’ve read and it makes me wonder where books that offer more semantic meaning of concepts instead of recipes exist.
All of the good ones do both. They first give the motivation and intuition and then make matters precise (because intuition can be wrong).
The sheer amount of material a student needs to digest in order to become conversant as even a pseudo-professional is enormous, which I think excuses, to some degree, the strange style of text books. I personally find that education is a process of emanations: first one digests the jargon and the mechanical activity of some subject (taking a dot product, in this case) and then one revisits the concepts with the distracting unfamiliarity of the technical accoutrements diminished by previous exposure. Thus able to digest the concepts better, the student can revisit the technical material again with a deeper appreciation of what is happening. The process repeats ad-infinitum until you ask yourself "what even IS quantum field theory?"
Colleges often have multiple classes on the same math subject, one made for physics and ME/EE people, one made for psych people, and one for CS. Some people don't realize that they accidentally picked up a textbook meant for a specific college pathway they don't care about.
Understandably college courses & textbooks meant for CS people will be more focused on computation, while a math major who is taking Linear Algebra will get a more theoretically motivated course. Gilbert Strang is an example of an engineering-focused text while Sheldon Axler or Katznelson & Katznelson is an example of what a math major would experience.
There are two ways to see every operation: a mathematical way and a physical way. The mathematical view of the dot product is an operation on vectors that adds their multiplied components, a·b = a_x b_x + a_y b_y + a_z b_z. The physical view of the dot product is what you said, comparing two vectors for similarity, or, in alternatively, multiplying their parallel components like scalars. The difference between these perspectives is in what is regarded as the defining property of the operation, which affects what you keep "fixed" as you vary aspects of the theory you're working in. For instance, when switching to spherical coordinates, the mathematical version of the dot product could still look the same, but the physical version has to change to preserve the underlying concept, which means its form becomes quite messy: (a_r, a_θ, a_φ)·(b_r, b_θ, b_φ) = a_r b_r (sin (a_θ) sin (b_θ) cos (a_φ - b_φ) + cos a_θ cos b_θ.
The difference in pedagogy seems to be which of these perspectives is treated as fundamental. Math education tends to treat the mathematical operations as fundamental. Physics treats the concept as fundamental and regards the operation as an implementation detail. It is very similar to how in software development you (for the most part) treat an API's interface as more fundamental than its implementation.
Unfortunately even physics books don't go over the intuition for underlying math very well, to their detriment. They seem to just assume everyone already perfectly understands multivariable calculus and linear algebra. I think it's because by the time you've gotten through a physics PhD you have to be completely fluent in those and the authors forget what it was like to find them confusing.
You might really enjoy working through the Art of Problem solving series. They’re early math -> calc books for kids that are getting into math competitions, and they explain so much in so much detail and really get to the root of why while also developing intuition. Get the e-book version. The explainers are incredible.
Maybe the books assume that the geometrical interpretations of the dot product are already known by the reader? I think they (both the projection interpretation and relation to angle between vectors) were taught in high school at the latest. There's also a lot of interpretations and uses for the dot product, some of which aren't necessarily that useful for classical mechanics.
But in general, literature using and/or teaching mathematics does tend to be too algebraic/mechanistic. Languge models can be a very good aide here!
I remember being shocked in the first year of college that introductory physics and introductory derivatives and integrations were not taught together. The calculus class never explains why these methods are useful, and the physics class expects rote memorization of the final algebraic equations.
It might be because you weren't in a Physics or an Engineering program.
Colleges tend to have two tracks for physics, one that's closer to high school physics, which is as you described. A collection of algebraic equations that you have to either remember or, if your professor was kind, given a crib sheet of.
The other is the "Engineering" or "Calculus" based physics track where, as you can imagine, you're taking Calc 1 and Physics 1 at the same time.
I have seen some, kinder, programs where you take Calc 1 in your first semester and start the Physics classes in your second semester.
I've seen a lot of comments, in this thread and others, to the effect of: "I didn't get math until I looked at it in a different way, with a lengthy span of time in between." Maybe just the two different looks and the time span by themselves are beneficial.
I think you hit the nailed on why most the textbooks are lousy at best on providing the 'why', apparently they are focusing more on the mechanical aspects and repeating exercises for scoring the exams, as a filler to the 1000 pages at US$100 textbook.
One of the best books on Electronics according to HN crowd is The Art of Electronics, and it is filled with pages over pages of how-to of designing circuits of more than 1000 pages. But if you want to know why a Colpitts oscillator is the best for your design, all the best for that.
Even the textbooks produced by professors from the best engineering schools (e.g MIT, Stanford, etc) are not spared of this issue. One of my former lecturers (not MIT) for linear algebra and numerical analysis courses claimed that he worked and consulted for NASA, but how I wished that he had cover some of the motivations of doing a dot product. For the ChatGPT responses of the reasons of doing dot product for two vectors see ChatGPT 4 prompt below. I think once ChatGPT/Gemini/etc can operate properly on math equations at least as good they can for NLP, I think we are going to have a wonderful treat even for the understanding the higly complex concepts for examples thermodynamics, electromagnetics and general relativity.
Another classic example is try to find inside your favorite DSP book on why analytic signal in complex number generated by Hilbert Transform is necessary for communication and there is a very high probability that that's not a single explanation for that but there are many equations and graphs related to analytic signal. If we understand why or the raison d'etre, perhaps we can further ask why stop at analytic signal in complex number form, can we use quaternion number and what is the benefit of having more elaborate quaternion number form rather than complex form. If you are trying to do that and check this fact with most of DSP textbooks on this particular important issue then you will probably end up dissapointed even if you have bookshelf that contains majority of DSP books that are published since the invention of FFT.
I have also included the ChatGPT prompt for the dot product, and the responses are delightful to say the least.
>User prompt: Why you want to perform dot product of two vectors
>ChatGPT 4:
The dot product of two vectors, also known as the scalar product, is a fundamental operation in mathematics and physics with various applications, including:
Computing angles between vectors: The dot product can be used to find the cosine of the angle between two vectors. This is particularly useful in geometry and 3D modeling to determine orientations and angles.
Projecting vectors: The dot product is used in the projection of one vector onto another. This is crucial in physics for resolving a vector into components parallel and perpendicular to another vector, and in computer graphics for rendering and shading calculations.
Checking orthogonality: Two vectors are orthogonal (i.e., the angle between them is 90 degrees) if their dot product is zero. This property is vital in many areas including computer graphics, signal processing, and linear algebra.
Work calculation in physics: In physics, the work done by a force (a vector) moving an object through a displacement (another vector) is calculated using the dot product. This is important for understanding how much energy is being used or transferred in a system.
Machine learning and data analysis: The dot product is used in algorithms such as those for finding similarities between data points, calculating vector norms, or in the training processes like those of neural networks where weights and input signals are combined.
Each of these applications demonstrates why the dot product is a key tool in both theoretical and applied disciplines, linking geometry, algebra, and practical computation.
After 20 years dealing with calculus in school, professional career and spare time there is always joy and inevitably a smile when I see writings like this. The feeling is that the intuition that took me years to build it is a matter of minutes when properly developed.
Things like:
> Then (dx)^2 will mean a little bit of a little bit of x;
is one of the pillars for one of my last struggles that has taken tens of my hours just to have a basic understanding of stochastic calculus and why this actually matters in this specific case.
When I see things like this makes me think that humanity is progressing as new generations having access to this information will make them learn faster. Thanks :)
The problem isn't that resources like this didn't exist in the past (Calculus Made Easy was written in 1910 after all), problem is that they aren't wellknown - and that is unlikely to change even today.
I think the notation and conceptualization is needlessly confusing. Some of it relates to old, often philosophical and even theological, debates on ontological status of infinitesimals.
The difference quantinent ended up as the Official Blessed Formulation of differential calculus, but it's very rarely used in practice, even though that's how calculus is used. And in practice calculus is still done using ad-hoc infinitesimal notations, but they are some weird thing with rules of their own which very few actually know (at least I don't).
Nonstandard calculus allows using infinitesimals in algebra with more or less the usual rules. Not sure if it's not more popular due to some fundamental technical or philosophical problems, or if it's just conservatism.
Stochastic calculus is quite bizarre indeed. Never understood e.g. the "proper" formulation of continuous time Kalman filters. Just limiting the timestep to zero seems to make sense and produces the right result with some massaging, but I've understood it's not really formally correct.
As someone who's taking a university entrance course in Calculus I find these kind of "calculus made easy" pamphlets irritatingly trite.
The hard part isn't the highest level concepts, which are actually fairly easy to grasp and somewhat intuitive.
The hard part is all the foundational knowledge required to solve actual math problems with Calculus.
The most difficult parts of Calculus (for me at least) are:
1. Having a very thorough grasp of the groundwork / assumed knowledge. Good enough that you can correctly solve an unexpected problem, from completing the square to long division of polynomials to an equation involving differentials.
2. Understanding and correctly applying the notation and graphing techniques, from Leibniz notation to sketching curves.
This is why large books and courses exist covering only introductory Calculus, not even beginning to scrape the surface of more advanced math.
The link is not a pamphlet (unless you read only the linked HTML page). It is an entire book, published in 1910 by Silvanus P. Thompson, and sufficiently well-regarded that it was re-edited in 1998 by Martin Gardner, and (independently) lovingly re-typeset in TeX by volunteers (and also turned into this website). Clearly it serves a need, and is not merely a “trite” pamphlet.
(The edition by Gardner is actually recommended against by some, who see in it a clash of two strong personalities, individually delightful.)
It's a great book, and one my father recommended to me to get me through the concepts when I was having trouble with the standardised teaching of the day.
It comes down to Leibnitz Vs Newton, and the world has standardised on the notation of one (I forget which). However the notation is a destination when learning it all, and the foundational ideas behind calculus were best explained taking ideas from both of them.
That's what this book does. It takes you through with every simple jumps in logic allowing you to discover calculus yourself and you therefore have the foundations to reason about it yourself. You don't just have to learn the final answers by rote.
Your issues seem to be algebra. I recommend Khan academy personally and just working through all of the highschool math that he goes over. I found his stuff when he was still just a guy on youtube back when I was in the same position as you. Studying calculus, did fine in high school, but my school was not good and totally unprepared me for actually studying math, skipping over a lot of those fundamentals. So often I would have a professor or TA take a complex equation, show an "obvious trick" that we "knew from algebra" and it would be the first time I ever saw that in my life. There is really no other solution than to study and relearn algebra, geometry and trig yourself as you learn calculus.
Most of your first point is … algebra? Yes if your algebra is weak you will not be able to cope with solving calc equations. The solution to that problem is not to be found in a calculus made easy. It would be found in algebra made easy.
Math isn't like programming. In programming you can often solve a problem using a library, framework, language facility, etc. without entirely understanding why it works all the way down to the binary level.
In math you can't often solve a more advanced problem such as Calculus problem without understanding the more foundational math such as algebra, fractions, etc.
If "information hiding" / layers of abstraction was possible in math, I would have completed my university entrance course months ago, but here I am still struggling.
Sure, we could have Algebra made easy and also Trigonometry made easy, Fractions made easy, Functions made easy, etc. etc.
I just find it personally irritating that all this foundational knowledge is brushed aside when it's really core to someone's actual competence dealing with actual math problems.
Maybe it's just assumed that people went to a good high school or had a private math tutor and already learned the foundations very well, but I think at least that assumption would be coming from a place of privilege.
It's similar to telling someone to take a Bootcamp in React and that will be enough for them to succeed as a software engineer. But to solve the kind of problems they are going to face in reality they will eventually have to learn at least some foundational Javascript and maybe a little about algorithms and data structures.
Algebra is the simple part. I’d say it’s more about math maturity. At least 1/3rd of my classmates had a hard time grasping the epsilon-delta definition of limit, let alone the deeper definitions like Cauchy sequence or those used in the proof that R is dense(and we were in an elite university’s competitive program). Among the survivors of single-variable calculus, at least 1/3 could barely get by the multi-variable calculus. I saw too many of my friends struggle with different integrals, and got massacred by Green’s equation.
My guess is that most people hit a wall of abstraction at certain point.
I feel the opposite. In high school I was pretty good at solving calculus problems but had little understanding what "limit" actually is. When in college I finally understood the definition of limit and all the foundational theorems arised from it, I was blown away.
For most people—who won't solve complex math problems daily at work—the takeaway from learning math is not their mechanical ability at solving math problems. The takeaway is their understanding of math concepts and ideas, which will shape their thinking skills in general.
> For most people—who won't solve complex math problems daily at work—the takeaway from learning math is not their mechanical ability at solving math problems. The takeaway is their understanding of math concepts and ideas, which will shape their thinking skills in general.
Agree, but for some others, there are real world consequences, e.g. whether they get accepted into a university or whether they can read and properly understand an academic paper.
For me, the biggest stumbling block in understanding the usual ε/δ limit definition in high school was teachers reading |x - a| as "the absolute value of x minus a" rather than "the distance between x and a".
The later reading suggests a more intuitive (to me) definition: a limit f(x)→q as x→p exists if, for every open interval Y containing q, an open interval X containing p exists such that f(x)∊Y for all x≠p in X (and then if f(p)=q, f is also continuous at p).
Another nice property of the above definition: replace "interval" with "ball" or "neighborhood" for analogous definitions for functions between metric and topological spaces, respectively.
So you want to do calculus ? You need algebra. What parts of algebra ? Go figure !
this is one big hurdle in learning math backwards. You discover new missing pieces at every corner. Each missing piece leading to another missing piece.
Learning math from the basics to advanced (as recommended by most) is very frustrating at how slowly you actually develop the math muscle.
At a deeper level, conceptual grasp does not make you good at math, its not enough. You may fool yourself into thinking you "get it" till you try to solve a few exercises. You need to repeat the lower levels enough to make it into muscle memory (which some people refer to as math intuition or groundwork) before embarking onto higher levels that build on it.
So working your way bottom up is slow and frustrating, top down is slow and frustrating. What do you do?
Just keep at it. One key observation for me was that at some point the misery and rabbit hole nature diminishes, quite rapidly. The groundwork of solving all those exercises repeatedly pays off and the next set becomes a little easier. Getting to calculus after spending ridiculous amount of time on algebra is the only way I have known to work.
And this is true for learning progamming too. knowing the concept of loops is essential but, you still can't write efficient code to sort an array. You need to get the syntax and write enough loops and then progress to exercising writing specific sorting algorithms repeatedly to get them into muscle memory.
But there is an inflection point beyond which the same concepts repeat but in different variations and they take progressively lesser time to get a grasp on.
thats just how I've learned math and programming. Also why a large percentage of people just give up hope and accept they just don't have the math gene. Meh.
> this is one big hurdle in learning math backwards. You discover new missing pieces at every corner. Each missing piece leading to another missing piece.
Yes this is exactly what happening to me.
E.g. I got up to Week 2 of the course and suddenly made the big (to me) discovery that sqrt(a/b) = sqrt(a)/sqrt(b).
It seems trivial I know when you see it written like that, but the problem is to recognise and apply that principle in the context of a broader problem such as factoring.
> Just keep at it.
Thanks, this gives me confidence that I'm not wasting my time haha
I am beginning to get better at it, to the point that I can often work out why I got a question wrong on my own without referring to the answer.
I "knew" enough calculus to get good grades in math courses through high school and college as an engineer.
But I didn't really "know" calculus until I read a book not too dissimilar from this one, "A Course of Pure Mathematics" from 1908 (!), which constructs calculus up from number theory (I think the fundamental theorem of calculus comes halfway through the book?). From that point it's impossible to forget.
As for why it's not taught this way today, I'd blame our testing regime and large classrooms, which incentivizes temporary memorization of key formulae and knowing where to mindlessly apply them, over deep/lasting/semantic understanding. I'd also blame the fact that we have a different maths teacher every year, so students come in with heterogenous understandings of the pre-requisites for the next year's material, so the first part of a section is spent reviewing + consolidating.
It takes maybe 10-20% more time to get a rich understanding of the subject that lasts a lifetime. But we value compression and instantly-measurable results more than actual learning. :/
Encountering material like this makes you really happy, but it's kind of bittersweet because it makes you realize that the modal state of modern pedagogy is pretty abysmal.
The last couple of months, I have been studying the fundamentals of algebra using Professor Leonard's YT Channel[0]. My goal is to fill in the gaps in my knowledge before I refresh my Calculus. It takes a while to go through all this stuff, if you do it right. But man, I have so much more confidence in my skills now than I had before, which to me is in itself rewarding and motivating. I had no idea how big my knowledge gaps in algebra were before I started going through his playlists.
My end goal is to be able to follow Andrej Karpathy's "Neural Networks: Zero to Hero"[1] without any big problems
So starting basically from "zero" in order to learn the prerequisites before learning what you actually want to learn on your own can feel daunting at times. But I think taking shortcuts will result in frustration. So, here I am taking algebra courses on YT with 38 years.
Readit News
You know what blew me away though? Not one textbook I looked in mentioned “why” the dot product is important; that is it’s useful for determining the similarity of two vectors. They all focused on the mechanical details of computing the dot product, but never spelled out the reason it can be useful. I went through a few other resources before I broke down and had a little chat with ChatGPT to discuss the meaning behind it and it makes perfect sense after that.
In comparison to when I was in college, things are much slower paced so I can take the time I need to ensure I have a full grasp of a concept before moving forward. I guess all of this is to say that as I’ve continued forward through more concepts I keep finding that the books I’m reading offer a mechanical view instead of a holistic view of the material. This feels like the biggest issue with most math books I’ve read and it makes me wonder where books that offer more semantic meaning of concepts instead of recipes exist.
You cannot simply explain to someone complex stuff - best way is to let people grind through to build their own understanding.
Parent poster wrote that "it’s useful for determining the similarity of two vectors" - now I would ask why do I need to determine similarity of two vectors as it does not mean much to me - if I would be grinding through math problems I would most likely find out why, but there is no way I could understand and retain it when someone would just tell me.
It wasn't until years later that I found that it was all about "the area under the curve" and why that would be useful. At no point in those high school classes did we ever work a practical example. I was pissed off all over again when I found out how useful that stuff could be, and how much I'd missed out on.
I'm sure most teachers mean well, and I'm sure most of them try. But by god there are some truly awful twats out there who should never set foot in a classroom again.
What an awful person. Chances are she was getting defensive and covering for her own lack of understanding. If I were a parent, I would confront her about that, not least of all her contempt for students and for learning, but toward parents.
Teachers don't know everything, and when they don't know, they should be able to admit that without hesitation or defensiveness. This sets a good example in general, of humility, instead of inculcating the notion that life is about having all the answers, or rather, pretending to have all the answers. All this does is set up people to become imposters. Of course, if you're teaching calculus, you should have at least a basic grasp of the material, and if you don't, you should say so, so that you've not put in a position where you have to teach it.
> I'm sure most teachers mean well, and I'm sure most of them try.
I think it is generally accepted that primary education isn't exactly packed with the best candidates, both from the point of view of pedagogical ability as well as mastery of the material.
He had to confess he didn't know why or how either of them works, he just uses them :-)
Perhaps a bit oddly I didn't have any problem with the pure math prof. also not offering any use cases, although of course it also does!
https://betterexplained.com/articles/a-calculus-analogy-inte...
But that fact’s significance and too obvious simplicity, with all its ramifications, only hit me deeply and profoundly when a year later I realized I could use those functions to draw a circle on a screen.
Before that they were abstractions related to other abstractions that I had to memorize to pass a course.
To this day I am frustrated when reading papers about abstract algebraic relations and other such concepts, without even a sentence or two discussing any intuitive way to think about them. Just their symbolic relations.
I appreciate that in the game of math that view becomes natural. But most of us learn math with additional motivations and are interested in any perspective that highlights potential usefulness or connection to the real world. Many of us mentally organize our knowledge teleologically.
Yet even when usefulness is known to exist, it is often neither mentioned or referenced. Or even considered relevant.
Edit: the same goes for not showing a single concrete example of an abstract concept. A kind of communication that would unlock many mathematical papers to a much larger audience of intelligent and relevant readers.
Take parametric curves. I explain that they generalize the concept of a function. Every function can be parametrized in a trivial way. They don’t really understand this concept. They have a hard time parametrizing a function and do so only becuase of a formula.
The fact is most people need to go through the mechanical process of doin g before they can get to a point of understanding. It takes almost the entire semester for me to convince beginning algebra students that the reason that 2x + 3x is 5x is because of the distributive property. And when they do understand it they don’t understand why that is important.
Later on when things click for someone they will often say things like, “Why didn’t they just tell this when we took the course?” Usually we did. You just didn’t have a sophisticated enough understanding of things to grok it at the time you took the course.
_Make: Geometry: Learn by coding, 3D printing and building_ https://www.goodreads.com/book/show/58059196-make
_Make: Trigonometry: Build your way from triangles to analytic geometry_ https://www.goodreads.com/book/show/123127774-make
_Make: Calculus: Build models to learn, visualize, and explore_ https://www.goodreads.com/book/show/61739368-make
(oddly the Calculus book was published second, so I guess I'll need to re-read it after I finish the trigonometry book)
Hopefully, this will provide me with a sufficient grounding in conic sections that I can solve my next CNC project with a reasonably efficient set of calculations (trying to do it using my rudimentary understanding of triangles from trigonometry had me 4 or 5 triangles deep, barely half-way to the final point I needed, and OpenSCAD badly bogged down performance-wise).
If you’re talking about research papers, that’s just because they’re written for domain experts and aren’t really for giving you intuition. They’re written in a deliberately terse (one might say elegant) style to convey the research findings in formal mathematical language and nothing much else. If you want to gain an intuitive grasp of things, read a proper textbook in detail or play around with the ideas on paper. Or both!
I guess the reason is that once you’ve acquired the intuition, having the literature cluttered up with the same explanations again and again becomes clunky and increases the volume of material to be sifted through when you’re just looking for a result you need in your research and don’t need all the extra chatter. It’s just cleaner that way. But to an outsider it does look more opaque. It’s a trade off.
https://www.stevenstrogatz.com/books/infinite-powers
Opposed to the liberal arts were the illiberal or servile arts. These are necessary and good, of course, but necessarily inferior to the liberal arts because their end is not truth or formation; they are instead practical, concerned with effecting some kind of economic end. The point here is not to disparage, but to understand how all of these are related and ranked according to a "for the sake of" relation. A human being doesn't exist to eat, he eats to exist, for instance.
Modern education is very much oriented toward the servile arts, and what passes for the liberal arts today is anything but the classical notion.
The point is that modern education is less interested in leading to understanding, realizing virtuous habits, and leading to freedom, and more interested in churning out workers. Workers don't ask "why" (though we can agree that those who do can, guided by prudence, contribute more economically). Indeed, that is perhaps the key difference between classical science and modern science: the emphasis of the former is truth, while that of the latter is control of nature. Of course, it isn't that you must choose absolutely between understanding and effectiveness, and the classical tradition does not claim either that study precludes work. Study often requires work, for sake of preparing the way for truth. Rather, it is that the end of the modern educational tradition is different from that of classical education, and this end determines the form of the pedagogical methodology. It is a difference in anthropology, of the vision of man.
All men work, but what do they work for? Do they work for work's sake, or perhaps to make money to satiate their base appetites (modern view)? Or do they work in order to be free to pursue higher ends[1]?
[0] https://www.newadvent.org/cathen/01760a.htm
[1] https://a.co/d/hE5830i
It's not really that it measures similarity. Physics isn't interested in that. It's that it tells you lengths and angles, which you need in all sorts of calculations. In more advanced settings, a dot product is generally taken as the definition of lengths and angles in more abstract spaces (e.g. the angle between two functions).
In machine learning applications, you want a definition of similarity, and one that you could use is that the angle between them is small, so that's where that notion comes in. A more traditional measure of similarity would be the length of the difference (i.e. the distance), which is also calculated using a dot product.
In physics, the dot product is used to losslessly project a vector onto an orthonormal basis, and the angle measures how much of the vector's magnitude is distributed to each bases vector.
The angle can be defined in terms of the dot product, because you don't need the angle (as in a uniform measure of rotation) in order to compute important physical results.
With that being said, I do remember my math and physics teachers in high school spend lots of time talking about the why and intuitions and let the books state the how.
Most textbooks motivate it by the angle between the vectors or as projections (e.g., for hyperplanes). Numerics-focused ones will further emphasize how great it it is that you can compute this information so efficiently, parallelizable etc.. Later on it will be about Hilbert space theory or Riemannian geometry and how having a scalar product available gives you lots of structure.
> This feels like the biggest issue with most math books I’ve read and it makes me wonder where books that offer more semantic meaning of concepts instead of recipes exist.
All of the good ones do both. They first give the motivation and intuition and then make matters precise (because intuition can be wrong).
Understandably college courses & textbooks meant for CS people will be more focused on computation, while a math major who is taking Linear Algebra will get a more theoretically motivated course. Gilbert Strang is an example of an engineering-focused text while Sheldon Axler or Katznelson & Katznelson is an example of what a math major would experience.
The difference in pedagogy seems to be which of these perspectives is treated as fundamental. Math education tends to treat the mathematical operations as fundamental. Physics treats the concept as fundamental and regards the operation as an implementation detail. It is very similar to how in software development you (for the most part) treat an API's interface as more fundamental than its implementation.
Unfortunately even physics books don't go over the intuition for underlying math very well, to their detriment. They seem to just assume everyone already perfectly understands multivariable calculus and linear algebra. I think it's because by the time you've gotten through a physics PhD you have to be completely fluent in those and the authors forget what it was like to find them confusing.
But in general, literature using and/or teaching mathematics does tend to be too algebraic/mechanistic. Languge models can be a very good aide here!
I guess if you want to learn math, only a math textbook will actually care.
Colleges tend to have two tracks for physics, one that's closer to high school physics, which is as you described. A collection of algebraic equations that you have to either remember or, if your professor was kind, given a crib sheet of.
The other is the "Engineering" or "Calculus" based physics track where, as you can imagine, you're taking Calc 1 and Physics 1 at the same time.
I have seen some, kinder, programs where you take Calc 1 in your first semester and start the Physics classes in your second semester.
Deleted Comment
One of the best books on Electronics according to HN crowd is The Art of Electronics, and it is filled with pages over pages of how-to of designing circuits of more than 1000 pages. But if you want to know why a Colpitts oscillator is the best for your design, all the best for that.
Even the textbooks produced by professors from the best engineering schools (e.g MIT, Stanford, etc) are not spared of this issue. One of my former lecturers (not MIT) for linear algebra and numerical analysis courses claimed that he worked and consulted for NASA, but how I wished that he had cover some of the motivations of doing a dot product. For the ChatGPT responses of the reasons of doing dot product for two vectors see ChatGPT 4 prompt below. I think once ChatGPT/Gemini/etc can operate properly on math equations at least as good they can for NLP, I think we are going to have a wonderful treat even for the understanding the higly complex concepts for examples thermodynamics, electromagnetics and general relativity.
Another classic example is try to find inside your favorite DSP book on why analytic signal in complex number generated by Hilbert Transform is necessary for communication and there is a very high probability that that's not a single explanation for that but there are many equations and graphs related to analytic signal. If we understand why or the raison d'etre, perhaps we can further ask why stop at analytic signal in complex number form, can we use quaternion number and what is the benefit of having more elaborate quaternion number form rather than complex form. If you are trying to do that and check this fact with most of DSP textbooks on this particular important issue then you will probably end up dissapointed even if you have bookshelf that contains majority of DSP books that are published since the invention of FFT.
I have also included the ChatGPT prompt for the dot product, and the responses are delightful to say the least.
>User prompt: Why you want to perform dot product of two vectors
>ChatGPT 4:
The dot product of two vectors, also known as the scalar product, is a fundamental operation in mathematics and physics with various applications, including:
Computing angles between vectors: The dot product can be used to find the cosine of the angle between two vectors. This is particularly useful in geometry and 3D modeling to determine orientations and angles.
Projecting vectors: The dot product is used in the projection of one vector onto another. This is crucial in physics for resolving a vector into components parallel and perpendicular to another vector, and in computer graphics for rendering and shading calculations.
Checking orthogonality: Two vectors are orthogonal (i.e., the angle between them is 90 degrees) if their dot product is zero. This property is vital in many areas including computer graphics, signal processing, and linear algebra.
Work calculation in physics: In physics, the work done by a force (a vector) moving an object through a displacement (another vector) is calculated using the dot product. This is important for understanding how much energy is being used or transferred in a system.
Machine learning and data analysis: The dot product is used in algorithms such as those for finding similarities between data points, calculating vector norms, or in the training processes like those of neural networks where weights and input signals are combined.
Each of these applications demonstrates why the dot product is a key tool in both theoretical and applied disciplines, linking geometry, algebra, and practical computation.
Things like:
> Then (dx)^2 will mean a little bit of a little bit of x;
is one of the pillars for one of my last struggles that has taken tens of my hours just to have a basic understanding of stochastic calculus and why this actually matters in this specific case.
When I see things like this makes me think that humanity is progressing as new generations having access to this information will make them learn faster. Thanks :)
The difference quantinent ended up as the Official Blessed Formulation of differential calculus, but it's very rarely used in practice, even though that's how calculus is used. And in practice calculus is still done using ad-hoc infinitesimal notations, but they are some weird thing with rules of their own which very few actually know (at least I don't).
Nonstandard calculus allows using infinitesimals in algebra with more or less the usual rules. Not sure if it's not more popular due to some fundamental technical or philosophical problems, or if it's just conservatism.
Stochastic calculus is quite bizarre indeed. Never understood e.g. the "proper" formulation of continuous time Kalman filters. Just limiting the timestep to zero seems to make sense and produces the right result with some massaging, but I've understood it's not really formally correct.
The hard part isn't the highest level concepts, which are actually fairly easy to grasp and somewhat intuitive.
The hard part is all the foundational knowledge required to solve actual math problems with Calculus.
The most difficult parts of Calculus (for me at least) are:
1. Having a very thorough grasp of the groundwork / assumed knowledge. Good enough that you can correctly solve an unexpected problem, from completing the square to long division of polynomials to an equation involving differentials.
2. Understanding and correctly applying the notation and graphing techniques, from Leibniz notation to sketching curves.
This is why large books and courses exist covering only introductory Calculus, not even beginning to scrape the surface of more advanced math.
The link is not a pamphlet (unless you read only the linked HTML page). It is an entire book, published in 1910 by Silvanus P. Thompson, and sufficiently well-regarded that it was re-edited in 1998 by Martin Gardner, and (independently) lovingly re-typeset in TeX by volunteers (and also turned into this website). Clearly it serves a need, and is not merely a “trite” pamphlet.
(The edition by Gardner is actually recommended against by some, who see in it a clash of two strong personalities, individually delightful.)
It comes down to Leibnitz Vs Newton, and the world has standardised on the notation of one (I forget which). However the notation is a destination when learning it all, and the foundational ideas behind calculus were best explained taking ideas from both of them.
That's what this book does. It takes you through with every simple jumps in logic allowing you to discover calculus yourself and you therefore have the foundations to reason about it yourself. You don't just have to learn the final answers by rote.
(As someone who's surprisingly bad at math and trying to undertake a Calculus pre-req to get into university!)
Math isn't like programming. In programming you can often solve a problem using a library, framework, language facility, etc. without entirely understanding why it works all the way down to the binary level.
In math you can't often solve a more advanced problem such as Calculus problem without understanding the more foundational math such as algebra, fractions, etc.
If "information hiding" / layers of abstraction was possible in math, I would have completed my university entrance course months ago, but here I am still struggling.
Sure, we could have Algebra made easy and also Trigonometry made easy, Fractions made easy, Functions made easy, etc. etc.
I just find it personally irritating that all this foundational knowledge is brushed aside when it's really core to someone's actual competence dealing with actual math problems.
Maybe it's just assumed that people went to a good high school or had a private math tutor and already learned the foundations very well, but I think at least that assumption would be coming from a place of privilege.
It's similar to telling someone to take a Bootcamp in React and that will be enough for them to succeed as a software engineer. But to solve the kind of problems they are going to face in reality they will eventually have to learn at least some foundational Javascript and maybe a little about algorithms and data structures.
My guess is that most people hit a wall of abstraction at certain point.
Dead Comment
For most people—who won't solve complex math problems daily at work—the takeaway from learning math is not their mechanical ability at solving math problems. The takeaway is their understanding of math concepts and ideas, which will shape their thinking skills in general.
Agree, but for some others, there are real world consequences, e.g. whether they get accepted into a university or whether they can read and properly understand an academic paper.
The later reading suggests a more intuitive (to me) definition: a limit f(x)→q as x→p exists if, for every open interval Y containing q, an open interval X containing p exists such that f(x)∊Y for all x≠p in X (and then if f(p)=q, f is also continuous at p).
Another nice property of the above definition: replace "interval" with "ball" or "neighborhood" for analogous definitions for functions between metric and topological spaces, respectively.
this is one big hurdle in learning math backwards. You discover new missing pieces at every corner. Each missing piece leading to another missing piece.
Learning math from the basics to advanced (as recommended by most) is very frustrating at how slowly you actually develop the math muscle.
At a deeper level, conceptual grasp does not make you good at math, its not enough. You may fool yourself into thinking you "get it" till you try to solve a few exercises. You need to repeat the lower levels enough to make it into muscle memory (which some people refer to as math intuition or groundwork) before embarking onto higher levels that build on it.
So working your way bottom up is slow and frustrating, top down is slow and frustrating. What do you do?
Just keep at it. One key observation for me was that at some point the misery and rabbit hole nature diminishes, quite rapidly. The groundwork of solving all those exercises repeatedly pays off and the next set becomes a little easier. Getting to calculus after spending ridiculous amount of time on algebra is the only way I have known to work.
And this is true for learning progamming too. knowing the concept of loops is essential but, you still can't write efficient code to sort an array. You need to get the syntax and write enough loops and then progress to exercising writing specific sorting algorithms repeatedly to get them into muscle memory.
But there is an inflection point beyond which the same concepts repeat but in different variations and they take progressively lesser time to get a grasp on.
thats just how I've learned math and programming. Also why a large percentage of people just give up hope and accept they just don't have the math gene. Meh.
Yes this is exactly what happening to me.
E.g. I got up to Week 2 of the course and suddenly made the big (to me) discovery that sqrt(a/b) = sqrt(a)/sqrt(b).
It seems trivial I know when you see it written like that, but the problem is to recognise and apply that principle in the context of a broader problem such as factoring.
> Just keep at it.
Thanks, this gives me confidence that I'm not wasting my time haha
I am beginning to get better at it, to the point that I can often work out why I got a question wrong on my own without referring to the answer.
> all of it really
Deleted Comment
Deleted Comment
https://press.uchicago.edu/ucp/books/book/chicago/C/bo548572...
Genetic
But I didn't really "know" calculus until I read a book not too dissimilar from this one, "A Course of Pure Mathematics" from 1908 (!), which constructs calculus up from number theory (I think the fundamental theorem of calculus comes halfway through the book?). From that point it's impossible to forget.
As for why it's not taught this way today, I'd blame our testing regime and large classrooms, which incentivizes temporary memorization of key formulae and knowing where to mindlessly apply them, over deep/lasting/semantic understanding. I'd also blame the fact that we have a different maths teacher every year, so students come in with heterogenous understandings of the pre-requisites for the next year's material, so the first part of a section is spent reviewing + consolidating.
It takes maybe 10-20% more time to get a rich understanding of the subject that lasts a lifetime. But we value compression and instantly-measurable results more than actual learning. :/
Encountering material like this makes you really happy, but it's kind of bittersweet because it makes you realize that the modal state of modern pedagogy is pretty abysmal.
My end goal is to be able to follow Andrej Karpathy's "Neural Networks: Zero to Hero"[1] without any big problems So starting basically from "zero" in order to learn the prerequisites before learning what you actually want to learn on your own can feel daunting at times. But I think taking shortcuts will result in frustration. So, here I am taking algebra courses on YT with 38 years.
[0] https://youtube.com/@ProfessorLeonard?si=0kiGvmbZv4b9Sgf9
[1] https://youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThsA9Gv...
https://archive.org/details/calulusforthepra000526mbp
[1] https://en.wikipedia.org/wiki/Calculus_Made_Easy
The link isn’t to the front page with his name.