I failed in university at math. Why? Because the tutors had not the time to help me. My level of math was not the same as the other students as i was not in the math part of a gymnasium.
I struggled and wasted a lot of time and energy to even find good explanations.
And when i had a math group, one girl was super nice but knew so muchmore than i did because of her math in gym. Professors asumed so much knowledge and no one cared to try to help people.
Best help were people from india on youtube with bad english.
And the most ridiculous part: Every year around the globe people teach this level of university math to probably millions of students. We should have the perfect free educational platform which teaches everyone perfectly already because so many tutors and professors lecture on the same topics over and over and over again. Our educational system is a joke.
THEN SAID A teacher, Speak to us of Teaching.
And he said:
No man can reveal to you aught but that which already lies half asleep in the dawning of your knowledge.
The teacher who walks in the shadow of the temple, among his followers, gives not of his wisdom but rather of his faith and his lovingness.
If he is indeed wise he does not bid you enter the house of his wisdom, but rather leads you to the threshold of your own mind.
The astronomer may speak to you of his understanding of space, but he cannot give you his understanding.
The musician may sing to you of the rhythm which is in all space, but he cannot give you the ear which arrests the rhythm nor the voice that echoes it.
And he who is versed in the science of numbers can tell of the regions of weight and measure, but he cannot conduct you thither.
For the vision of one man lends not its wings to another man.
And even as each one of you stands alone in God’s knowledge, so must each one of you be alone in his knowledge of God and in his understanding of the earth.
It's high school, and parent is likely from Denmark, where high school was split into language or math, although it is odd that someone from language would be admitted to a bachelor in math without taking extra classes for about a year between the two.
I took a course in numerical analysis and the prof was hopeless as a teacher - not only a bad teacher but proud to be a bad teacher.
The syllabus had all the methods we had to know, and I learned them all through youtube on a channel called NumericalMethodsGuy. I stopped attending class and just went to labs (which were really just matlab assignments in numerical methods) and turned in assignments, and wrote exams. I got an A.
I just finished a mathematics degree (BMath). Not a single one of my professors was a teacher in the sense of a primary/secondary school teacher. They lectured for an hour, three times per week, and assigned weekly or biweekly coursework. They set midterms and final exams and they assigned all the grading to TAs.
Key takeaways:
* Mathematics is hard. Much harder than most other subjects (except physics which is mostly hard because of all the math involved).
* University is not like primary/secondary school. It is a place where you need to learn how to take responsibility for your own learning. Ideally, you learn how to become an adult.
Every one of my classmates began their degrees from a different place. They excelled in some areas and struggled in others. Many dropped out. This led me to believe that most secondary schools do not fully prepare their students to study mathematics. Having said that, those of us who did make it weren't exactly geniuses. Just people who got used to it.
"Young man, in mathematics you don't understand things. You just get used to them." -- John von Neumann
I've been tutoring high school students in math (and other subjects) for 8 years now. One thing I'd like to add is that I can tell who will succeed at mathematics and who will struggle just by watching them work from across the room.
The students who succeed are the ones who can sit there and focus for hours at a time. The ones who struggle do so because they can't focus for more than five minutes and then start socializing. I think one of the biggest issues for people studying math is that they either can't focus (due to ADHD) or they have math anxiety which fills them with dread any time they try to study. This dread can be so overwhelming that they will do anything they can to avoid it, so they stop studying and do something else.
When I was studying math I was spending upwards of 40 hours per week working on homework. Although this is normal for anyone with a full-time job, it's an unfathomable amount of time to be studying math for those who struggle. This is really what it takes though. An unrelenting drive to figure things out.
For myself, with ADHD, the key was not doing it alone. Working through coursework and homework on a literal blackboard with colleagues from the same class. Just thought I'd add in case anyone felt dissuaded or needed something to try.
It really was the savior of me. I can focus for hours when I get into something, but its the friction of starting up that I found other people really helped with. There are formal names for it, like "body doubling", but I didnt know of that, I just knew it was critical to work together with others to get stuff done, which as the above correctly writes, absolutely must be done.
If you have ADHD, you need to understand things in mathematics, because merely memorizing them becomes too difficult.
Understanding things in math is definitely possible, and whoever says otherwise probably sucks as a teacher. Yes, it is possible to just say a lot of random stuff, and the best students succeed to figure it out anyway. But if you take care to actually explain how it works, the proportion of successful students increases dramatically.
Conversational dialogue seems like a fascinating distraction.
Many people who have (unsuccessfully) attempted to apply AI to education have focused too much on the "explanation" part and not enough on scaffolding, navigating, and managing the entire learning process. It’s easy to go on a wild goose chase building an explanation AI.
You fall in love with the idea of AI having conversational dialogue with students, and then you get lost in the weeds of complexity. You solve just enough of the problem to produce a cool demo, yet you're still hopelessly far away from self-service learning in real life.
I don't think conversational dialogue is even necessary.
What we do at mathacademy.com is hard-code explanations and break them up into bite-size pieces that are served at just the right moment. And we close the feedback loop by having students solve problems, which they need to do anyway. (The student's "response" is whether they got the problem correct.)
Sure, hard-coding explanations feels tedious, takes a lot of work, and isn't "sexy" like an AI that generates responses from scratch – but at least it's not a pipe dream. It's a practical solution that lets us move on to other components of the AI that are just as important.
What are those other components? A handful off the top of my head:
* After a minimum effective dose of explanation, the AI needs to switch over to active problem-solving. Students should begin with simple cases and then climb up the ladder of difficulty, covering all cases that they could reasonably be expected to solve on a future assessment.
* Assessments should be frequent and broad in coverage, and students should be assigned personalized remedial reviews based on what they answered incorrectly.
* Students should progress through the curriculum in a personalized mastery-based manner, only being presented with new topics when they have (as individuals, not just as a group) demonstrated mastery of the prerequisite material.
* After a student has learned a topic, they should periodically review it using spaced repetition, a systematic way of reviewing previously-learned material to retain it indefinitely into the future.
* If a student ever struggles, the system should not lower the bar for success on the learning task (e.g., by giving away hints). Rather, it should strengthen a student’s area of weakness so that they can clear the bar fully and independently on their next attempt.
I find myself endlessly frustrated by these discussions around the teaching of mathematics. People seem to want to make it more conversational, more interactive, more engaging. It doesn't work that way! Learning mathematics is like learning to play a musical instrument. No amount of 1-on-1 discussion with a teacher will get you to mastery.
You just need to practice. For hours and hours and hours.
The teacher's job is to help guide you to what you might want to study next. The teacher cannot replace individual practice time.
Agree. A common theme in the science of learning is that effective learning centers on deliberate practice, where activities are done entirely for the purpose of pushing one's limits and improving performance. Of course, these activities tend to be more effortful and less enjoyable.
Classroom activities that are enjoyable, collaborative, and non-repetitive (such as group discussions) can sometimes be useful for increasing student motivation and softening the discomfort associated with deliberate practice. However, these activities are only supplements, not substitutes, for deliberate practice. Unlike deliberate practice, they do not directly move the needle on student performance – rather, they "grease the wheels" and reduce psychological friction during the process of deliberate practice.
The need for practice notwithstanding, suspect both practice and narrative/conversational exposition are needed. I’ve worked with students who become very good at applied procedural problem solving through practice but who can only get just so far because they lack foundational constructs to guide them. More time in discussion with an expert could potentially fill in those gaps. An expert who understands the material well enough to adapt it to the learner in front of them.
> I find myself endlessly frustrated by these discussions around the teaching of mathematics. People seem to want to make it more conversational, more interactive, more engaging. It doesn't work that way! Learning mathematics is like learning to play a musical instrument. No amount of 1-on-1 discussion with a teacher will get you to mastery.
That's flat out untrue. I speak from experience of teaching math all the way from grade school to teaching undergrads and grad students to teaching adults.
I can take a failing student who is completely confused and get them on track in a few hours over a few weeks. 1-on-1 discussions are not just really important, they're astronomically more effective than anything else.
Making math engaging is the number one thing that matters. Being an engaging teacher matters. Adapting to a specific student and their needs matters. Someone can practice something they hate for 10 hours and get nowhere. Someone else can practice something they love for 10 minutes and make amazing progress. You remember things that are engaging, you hardly remember anything about things that are boring.
This is what all of these AI tools get wrong. Even if they understood the problem domain and could explain it --- which they cannot.
I've had this experience with adults switching jobs into programming too. So many avoided STEM because they hated math. After like two hours together they not only got math, they started to enjoy it. Happens all the time.
> You just need to practice. For hours and hours and hours.
This is probably the worst brain damage that Gladwell and others have done with the whole nonsense of 10,000 hours of practice.
No. No amount of poor quality practice where you're hitting your head against the wall will be productive.
You need to learn how to practice math. Practice is worth it after you've gained some mathematical sophistication and some understanding of the problem domain. Most people need peers or a teacher or a tutor to help get them unstuck and point you in the right direction.
> The teacher cannot replace individual practice time.
No amount of practice for 99% of people can replace a tutor. Yes, people with mathematical aptitude don't need one; I didn't. But most people need to have some guidance and then they can practice.
As someone whose parents were Bay Area workaholics and was left to teach myself, was yelled at for asking questions and not knowing how to do it, and physically punished if homework wasn't done by the time they came home, 1 on 1 discussion would have been incredibly helpful and probably life changing in more areas than just math (homework instilling fear doesn't translate to dealing well with tasks later in life).
I can't imagine how nice it would have been not to have to hide out of fear that I was having problems figuring out calculus on my own.
I agree in principle, but I must stress that conceptual understanding of what the problem or functional area of math is and how you perform the task to solve and know that you are solving it correctly should be understood.
Simply being shown it once on a whiteboard or in a book isn’t working for the majority of humans.
Really? I struggled with calc in school because lectures were dry proofs (without ever formally teaching how proofs work!!) and then they just told us to read the textbook. Then I found mit ocw and ripped through the material because it was much more engaging. Never would have passed calc I-III, linear algebra, and diff eq without those lecture recordings.
Hard-coding explanations does seem like the way to go here. It's what researchers at Carnegie Mellon did back in 1984 with demonstrated success. See: Intelligent tutoring Systems: The ACT Project https://www.youtube.com/watch?v=boDH_pW14B0
My question is, if we've had intelligent math tutoring demonstrating success in real classrooms since 1984, then why haven't we been actually using them to help teach math?
I don't know enough about the ACT Project to make any hard claims regarding why it didn't go mainstream, but I imagine one big source of friction was that students didn't have their own school-issued Chromebooks like is so common nowadays. Remember that in 1984, computers were way, way more expensive than they are now.
As long as there are enough people working on this problem, I don't mind how many approaches there are. Education systems around the world just aren't good enough.
The irony is that the world where AI could reasonably explain and teach any level of math is the same world where most of such graduates would be unemployed anyways.
You’re already seeing oversupply of educated folks in India and China. Graduate students working as baristas in USA, etc. Sadly inevitable.
In any case it’s good to have the resources I suppose.
I’m also fundamentally skeptical of these stats around math struggling. What percentage of kids who would be very likely to use Khan Academy are struggling with math? And what percentage who are not would even use khan academy to begin with?
Most of the problems with students doing poorly are sadly societal - not to say that this isn’t useful, though.
I disagree. There is no such thing as a societal oversupply of educated people. There might be a market oversupply, but a well educated society is a stupid thing to avoid. I would also argue that we are seeing an oversupply of people with education credentials, rather than an education. Even in high end universities in China cheating is rampant (this isn't isolated to China, but my experience is that Chinese students are VERY open about it). For many people in University, the goal isn't to get an education, it is to get a degree.
As to my own experience: I am, according to most standardized tests, very apt at quantitative reasoning, but I never progressed far in math in school. Why? because I was placed on the standard track in math in a public school with a bunch of students who didn't care, and teachers who didn't have time to care, and to be honest, the attitude rubbed off. I once got in trouble because I programmed a python script to do my problem sets when I was 13 because it was faster than doing it by hand for me. In retrospect, that form of "cheating" was a sign that my teachers should have picked up on.
Quite simply, I never had access to a good math instructor throughout my schooling.
Now, decades later, I am intensely interested in a lot of subjects that require a background in math that I don't have, and I am becoming interested in Math for Math's sake. I have been using open access textbooks, and an AI assistant of my own creation to help me learn.
I'm similar to you in many regards. I had no desire to learn math early in school due to the education system, how it is taught and lack of meaning for math portrayed early on. I coasted through high school like a zombie without meaning until 11th grade when I took Physics. Suddenly everything clicked, I magically became good at math despite not performing well at it before in my math classes. I ended up studying engineering and working in the EV industry. Now, I am studying pure math for the sake of my own curiosity and I'm passionate about developing AI tools to help people learn and see the why behind math as early as possible. I think I would have accomplished much more if I was exposed to Physics in elementary or middle school or at least a "History of Math" philosophy based class.
> I disagree. There is no such thing as a societal oversupply of educated people.
I disagree with this assessment. You don’t need everyone to have a phd. There’s education and there’s Education. With respect to the article, sure it’s great if everyone is good at math, but not everyone needs a math PhD. Additionally there are diminishing returns on the time spent learning more and more math vs. other things.
In any case there have already been tons of math resources available for a while now.
A better example would be chemistry. Should everyone be spending time learning chemistry, why or why not?
Intellectual obesity is a thing - simply knowing more things isn’t inherently useful.
Also, writing a computer program because you are slow at solving math problems is a sign that you may need more math practice. It's not like you were handing my in your homework with solutions your wrote for harder problems that you wrote.
Yes, the studies tend to exclude the kids who aren't using the tools "enough," which can end up being 95% of the students! [0]
On the other hand, I know there are people who enjoy math but struggle to pay attention and learn from a "sit quietly and listen" lecture-based model. (My husband is in this camp.) I can see how the back-and-forth conversational approach of these AI tutors could be really helpful for students like this.
They're still a small minority of the student population, but maybe a different minority from the students who are already excelling academically. If Khanmigo helps only 5% but it's a slightly different 5% from the students who are best served by the current system, I see that as a win, even if it's not the educational revolution that is being hyped.
> What percentage of kids who would be very likely to use Khan Academy are struggling with math?
A lot. A lot, lot. There are entire states full of people in India who don't have access to good teaching. Who travel absurd distances and stay away from their family for years to get access to good education. In the last few years as kids in my extended family grew up, I am amazed by how much they just learn from YouTube. One of the biggest educational startups started from a free education YT channel.
Interesting - from my time in India most kids don’t even have reliable access to internet and electricity and are very unlikely to be using khan academy to begin with
> You’re already unemployed if you focus on history or literature but we still learn it.
Most people are still employed if they focus on history or literature. They just might not be making a lot of money or working in history or literature, though some do. Plenty of people in highly "employable" degrees also find themselves unemployed for a long time once they graduate.
The reality is that getting a college degree, regardless of the major, increases your chances of employment afterwards and potential lifelong earnings enormously over only a HS degree.
But as you implied, most of the worst trends in higher-ed, a real corruption of its original intentions and values, have come from a obsession with the idea that higher-ed is solely about employee training.
AI won't ever be able to teach math because AI cannot reason. It handles information very differently than humans do. It is at best a flawed oracle that may lie through omission without you being able to tell.
This is supported by the fact that there are hard limits to computation, in terms of both Computability Theory, and Complexity Theory that are often disregarded by the novice and magical thinker alike despite being largely solved fields by experts in Computer Science, at least in terms of Computability Theory.
Most of this work was completed in the 1950s.
This is not a difficult a subject, but most people online simply are unwilling to do the work to rationally learn this and instead choose to try and hide their own ignorance because they feel inadequate, or perhaps they are engaging in something more malevolent. The reasoning (false justification) doesn't really matter.
It is, however; quite telling when expert's have validated these things, and yet any mention contrary to a narrative provokes downvotes in a public forum to remove it from view. A perfect example of intentional actions done by third-parties trying to misinform others about the risks (by those parties actions removing legitimate and valid information).
For those with the cajones to actually learn this stuff. Here is a link. It seems very jaron laden but it provides a true understanding of how computers actually work which is sorely lacking in the youth of today.
The only net benefit AI has towards society is towards destructive ends because there are far more destructive people whose efforts scale more in general when compared to the good people today. These people often do things which cannot be undone (salting the earth and burning the bridge).
Simply not having a sufficient background, or failing to perform basic due dilligence often will place most people in that destructive cohort regardless of their own personal beliefs.
Outcomes matter more than intentions, and the devil is always in the details.
A perfect example of this potential landmine of a field would be anything dealing with the underlying mechanisms involved in human to human communication.
There is an uncanny valley, and distorted reflected appraisal will occur inevitably in any human to AI interface seeking to mimic human communication.
Case studies from torture during the Korean Conflict (1950s) show that distorting reflected appraisal leads to either psychotic or dissociative behavior that progresses as exposure increases, it can permanently break people to the point where they cannot recover. Often, the first thing to go is rational thought.
Multiple experts cover this and confirm the findings (Lifton, Meerloo).
So what do you suppose will happen when AI distorts reflected appraisal (because it can't be tested sufficiently to the contrary) and its then used on the next generation of children? (who are inherently vulnerable to permanent change at that stage of development).
If as a result, they end up either killing themselves or others, or become people incapable of rational thought (which impulse control is correlated with), do you think they'll be alive or survive very long? What indicators would there be that this is happening (none, other than increasing chaotic violence)
Who do you think will be responsible for crippling them if that were to come to pass? Would any kind of justification ever be able to justify that outcome? Could we even recover from intentional crazy-making as a society? (likely not, insanity cannot be cured).
Other posters have told you that computability has nothing to do with what AI can and can't do relative to humans and I agree with them, but its not exactly something universally agreed upon.
However, I think its reasonable to say that if you think that some fundamental thing keeps computers from doing what people do then you believe that people are somehow magic. This isn't a super unusual perspective among humans, but its certainly not a particularly common scientific one.
>The moment that any element has more than one underlying meaning the problem class exceeds what is capable by a computer (also known as a deterministic finite automata).
This is complete bullshit, and computability theory has nothing to say about which (if any) cognitive tasks people can do that AI cannot.
Maybe I’m reading too much into it but the roadmap mentioning switching from GPT4 Turbo to 4-o and hoping for better math performance feels like they are betting on a significant near term reliability improvement in LLMs without any other real plans. That magic jump is starting to look more and more doubtful by the day.
> Khanmigo now uses a calculator to solve numerical problems instead of using AI’s predictive capabilities. If you’ve been using Khanmigo recently, you may have seen that it will sometimes say it is “doing math.” This is when the math problem is running through the calculator behind the scenes.
> We’ve upgraded parts of Khanmigo to a more capable large language model, which is the software that generates human language. The more capable large language model is called GPT-4 Turbo. Our internal testing shows an improvement in math after we made the switch.
> We are beginning to test the capabilities of a new large language model called GPT-4o, and we’re evaluating other models too to see if they are stronger at math.
> We’ve improved the way AI “thinks” during a tutoring session before responding to a student. We have instructed the AI to write out all the ways in which the student may have arrived at their answer. This approach mimics how a tutor in real life works with a student. We’ve found it significantly improves the quality of math interactions.
> We’ve built new tools to track our progress on math.
> We’re sharing math examples and learnings with others in our field so that we can learn from each other.
> We’re studying the latest research papers on math performance.
Sounds like most of what they're doing is related to prompting, chain-of-thought reasoning and similar, on top of a 'vanilla' foundation model. Sounds like something an ambitious student could replicate / improve upon, so given their mission, it'd be cool if they published the exact techniques they're using and their benchmark results.
I sometimes wish if money, which Facebook, et.al invested in Indian edtech ecosystem, was given to Khan Academy, it would have done justice for the whole world's education system.
Given its 4$ per month[1] and they are a non-profit with just 55M$ revenue. How is GPU cost and hence their with OpenAI for this going to work if it becomes really-good and 100's of millions of kids start using it?
Renting a 24GB VRAM Runpod is ~0.5$ per hour. How can the math work out unless you have to have a non-profit energy company and a server farm attached?
The math seems about right to me. Using a chatbot for an hour might only use a minute or two of computer time, I would be genuinely surprised if a human could hit ten minutes of compute time/hour of interaction. Most responses in my technical chats are under 5 seconds, so using a full hour of GPU time is going to take A LOT of questions.
I heavily use ChatGPT via a desktop client and my own key, and I am at $3.90 for the month after making a mistake where I accidentally was sending a 30k token prompt repeatedly, that cost me about $2. Using an optimized approach, it is VERY hard to incur a $4/month bill.
I imagine that Khan academy has a much better bulk-pricing agreement with openAI, and that they can probably have a caching layer that can look for previously asked questions.
I've taught math at every level. From volunteering to teach it to grade school kids, to getting paid for highschool tutoring, and at the university level both for undergrads and now with my own grad students.
I tried Khanmigo. It is counterproductive junk.
It simply doesn't understand what a student knows and it has to idea how to give examples or provide perspective shifts that help students. That's what a good tutor does. It gets stuck explaining the same things in the same ways but with different words over and over again.
It has no idea how to think geometrically vs algebraically and how to switch between the two. And it can't carry out even simple proofs.
More people can afford phones than can afford tutors (or even books). We shouldn't get rid of tutors anymore than we should throw out Khanmigo because it isn't perfect.
The fun thing about getting math help from the ai is that okay, sure it can explain how fractions work to a third grader, but it can also explain how fiber bundles work to a graduate student. Maybe it gets the details wrong, but like even if it's wrong sometimes, it's far better than googling or wikipedia or even a text book sometimes, because you can interrogate it interactively and ask for clarifications. Yes, a tutor or a teacher is going to be better, but not everyone has access to a math expert.
> Maybe it gets the details wrong, but like even if it's wrong sometimes, it's far better than googling or wikipedia or even a text book sometimes, because you can interrogate it interactively and ask for clarifications
this to me is an insane way to think. I do not consider something better because I can ask it a question, something is better because it is more correct.
Wikipedia and textbooks are not necessarily correct, either and if you don't understand the topic, you really have _no way of knowing if they're correct_. You have to always check multiple sources and chase down references if you want to really know if something is true or not.
> Maybe it gets the details wrong, but like even if it's wrong sometimes, it's far better than googling or wikipedia or even a text book sometimes
Are people consulting only one book/source on any given subject? I remember learning Python and C, and it was such a mixture of books/articles/forums/wiki, I can't even remember the process. I do remember understanding pointers easily because of a chapter of another book about memory models in OS. Learning is an iterative process and the expert is just there to shorten the way, not to carry you through it.
I've found that at least chatgpt still produces complete nonsense with graduate level material (category theory). It's slightly better than the mLab[0] as a reference. When you try to guide it back onto something meaningful, it just ignores what you told it. In fact, I had a similar experience with programming where I asked it to explain a diff, and when it got it wrong and I told it so, it still just kept repeating the wrong information. Maybe the paid version is better, but the public demo seems like a useless toy to me.
I've found Wikipedia to be an excellent resource for math/physics once you're ~halfway through a math degree to have a good baseline level of knowledge to understand it.
The trouble with math is one of the important goals is to teach people not to produce nonsense and to be able to tell the difference between and argument that sounds good vs. an argument that's sound and good. A nonsense generator (presented as authoritative) is perhaps not the best way to do that. Obviously its abilities could change in the span of a couple years, so it's probably still worthwhile to explore how we would use it if it were useful.
Readit News
I failed in university at math. Why? Because the tutors had not the time to help me. My level of math was not the same as the other students as i was not in the math part of a gymnasium.
I struggled and wasted a lot of time and energy to even find good explanations.
And when i had a math group, one girl was super nice but knew so muchmore than i did because of her math in gym. Professors asumed so much knowledge and no one cared to try to help people.
Best help were people from india on youtube with bad english.
And the most ridiculous part: Every year around the globe people teach this level of university math to probably millions of students. We should have the perfect free educational platform which teaches everyone perfectly already because so many tutors and professors lecture on the same topics over and over and over again. Our educational system is a joke.
That's called "zone of proximal development" in pedagogy.
However, "that which already lies half asleep in the dawning of your knowledge" is often a consequence of things that were revealed to you previously.
When you learn, you could probably make any specific step alone, but you cannot make all the steps alone.
The syllabus had all the methods we had to know, and I learned them all through youtube on a channel called NumericalMethodsGuy. I stopped attending class and just went to labs (which were really just matlab assignments in numerical methods) and turned in assignments, and wrote exams. I got an A.
Key takeaways:
* Mathematics is hard. Much harder than most other subjects (except physics which is mostly hard because of all the math involved).
* University is not like primary/secondary school. It is a place where you need to learn how to take responsibility for your own learning. Ideally, you learn how to become an adult.
Every one of my classmates began their degrees from a different place. They excelled in some areas and struggled in others. Many dropped out. This led me to believe that most secondary schools do not fully prepare their students to study mathematics. Having said that, those of us who did make it weren't exactly geniuses. Just people who got used to it.
"Young man, in mathematics you don't understand things. You just get used to them." -- John von Neumann
I've been tutoring high school students in math (and other subjects) for 8 years now. One thing I'd like to add is that I can tell who will succeed at mathematics and who will struggle just by watching them work from across the room.
The students who succeed are the ones who can sit there and focus for hours at a time. The ones who struggle do so because they can't focus for more than five minutes and then start socializing. I think one of the biggest issues for people studying math is that they either can't focus (due to ADHD) or they have math anxiety which fills them with dread any time they try to study. This dread can be so overwhelming that they will do anything they can to avoid it, so they stop studying and do something else.
When I was studying math I was spending upwards of 40 hours per week working on homework. Although this is normal for anyone with a full-time job, it's an unfathomable amount of time to be studying math for those who struggle. This is really what it takes though. An unrelenting drive to figure things out.
It really was the savior of me. I can focus for hours when I get into something, but its the friction of starting up that I found other people really helped with. There are formal names for it, like "body doubling", but I didnt know of that, I just knew it was critical to work together with others to get stuff done, which as the above correctly writes, absolutely must be done.
Understanding things in math is definitely possible, and whoever says otherwise probably sucks as a teacher. Yes, it is possible to just say a lot of random stuff, and the best students succeed to figure it out anyway. But if you take care to actually explain how it works, the proportion of successful students increases dramatically.
I learned complex math concepts before the university like math i needed for 3d graphics without issues.
If we as a society think its okay to have such an entry barrier, i disagree and therefore really like having an AI tutor.
Many people who have (unsuccessfully) attempted to apply AI to education have focused too much on the "explanation" part and not enough on scaffolding, navigating, and managing the entire learning process. It’s easy to go on a wild goose chase building an explanation AI.
You fall in love with the idea of AI having conversational dialogue with students, and then you get lost in the weeds of complexity. You solve just enough of the problem to produce a cool demo, yet you're still hopelessly far away from self-service learning in real life.
I don't think conversational dialogue is even necessary.
What we do at mathacademy.com is hard-code explanations and break them up into bite-size pieces that are served at just the right moment. And we close the feedback loop by having students solve problems, which they need to do anyway. (The student's "response" is whether they got the problem correct.)
Sure, hard-coding explanations feels tedious, takes a lot of work, and isn't "sexy" like an AI that generates responses from scratch – but at least it's not a pipe dream. It's a practical solution that lets us move on to other components of the AI that are just as important.
What are those other components? A handful off the top of my head:
* After a minimum effective dose of explanation, the AI needs to switch over to active problem-solving. Students should begin with simple cases and then climb up the ladder of difficulty, covering all cases that they could reasonably be expected to solve on a future assessment.
* Assessments should be frequent and broad in coverage, and students should be assigned personalized remedial reviews based on what they answered incorrectly.
* Students should progress through the curriculum in a personalized mastery-based manner, only being presented with new topics when they have (as individuals, not just as a group) demonstrated mastery of the prerequisite material.
* After a student has learned a topic, they should periodically review it using spaced repetition, a systematic way of reviewing previously-learned material to retain it indefinitely into the future.
* If a student ever struggles, the system should not lower the bar for success on the learning task (e.g., by giving away hints). Rather, it should strengthen a student’s area of weakness so that they can clear the bar fully and independently on their next attempt.
You just need to practice. For hours and hours and hours.
The teacher's job is to help guide you to what you might want to study next. The teacher cannot replace individual practice time.
Classroom activities that are enjoyable, collaborative, and non-repetitive (such as group discussions) can sometimes be useful for increasing student motivation and softening the discomfort associated with deliberate practice. However, these activities are only supplements, not substitutes, for deliberate practice. Unlike deliberate practice, they do not directly move the needle on student performance – rather, they "grease the wheels" and reduce psychological friction during the process of deliberate practice.
The need for practice notwithstanding, suspect both practice and narrative/conversational exposition are needed. I’ve worked with students who become very good at applied procedural problem solving through practice but who can only get just so far because they lack foundational constructs to guide them. More time in discussion with an expert could potentially fill in those gaps. An expert who understands the material well enough to adapt it to the learner in front of them.
That's flat out untrue. I speak from experience of teaching math all the way from grade school to teaching undergrads and grad students to teaching adults.
I can take a failing student who is completely confused and get them on track in a few hours over a few weeks. 1-on-1 discussions are not just really important, they're astronomically more effective than anything else.
Making math engaging is the number one thing that matters. Being an engaging teacher matters. Adapting to a specific student and their needs matters. Someone can practice something they hate for 10 hours and get nowhere. Someone else can practice something they love for 10 minutes and make amazing progress. You remember things that are engaging, you hardly remember anything about things that are boring.
This is what all of these AI tools get wrong. Even if they understood the problem domain and could explain it --- which they cannot.
I've had this experience with adults switching jobs into programming too. So many avoided STEM because they hated math. After like two hours together they not only got math, they started to enjoy it. Happens all the time.
> You just need to practice. For hours and hours and hours.
This is probably the worst brain damage that Gladwell and others have done with the whole nonsense of 10,000 hours of practice.
No. No amount of poor quality practice where you're hitting your head against the wall will be productive.
You need to learn how to practice math. Practice is worth it after you've gained some mathematical sophistication and some understanding of the problem domain. Most people need peers or a teacher or a tutor to help get them unstuck and point you in the right direction.
> The teacher cannot replace individual practice time.
No amount of practice for 99% of people can replace a tutor. Yes, people with mathematical aptitude don't need one; I didn't. But most people need to have some guidance and then they can practice.
Deleted Comment
I can't imagine how nice it would have been not to have to hide out of fear that I was having problems figuring out calculus on my own.
Simply being shown it once on a whiteboard or in a book isn’t working for the majority of humans.
Then yes. Practice, practice
Deleted Comment
You’re already seeing oversupply of educated folks in India and China. Graduate students working as baristas in USA, etc. Sadly inevitable.
In any case it’s good to have the resources I suppose.
I’m also fundamentally skeptical of these stats around math struggling. What percentage of kids who would be very likely to use Khan Academy are struggling with math? And what percentage who are not would even use khan academy to begin with?
Most of the problems with students doing poorly are sadly societal - not to say that this isn’t useful, though.
As to my own experience: I am, according to most standardized tests, very apt at quantitative reasoning, but I never progressed far in math in school. Why? because I was placed on the standard track in math in a public school with a bunch of students who didn't care, and teachers who didn't have time to care, and to be honest, the attitude rubbed off. I once got in trouble because I programmed a python script to do my problem sets when I was 13 because it was faster than doing it by hand for me. In retrospect, that form of "cheating" was a sign that my teachers should have picked up on.
Quite simply, I never had access to a good math instructor throughout my schooling.
Now, decades later, I am intensely interested in a lot of subjects that require a background in math that I don't have, and I am becoming interested in Math for Math's sake. I have been using open access textbooks, and an AI assistant of my own creation to help me learn.
I disagree with this assessment. You don’t need everyone to have a phd. There’s education and there’s Education. With respect to the article, sure it’s great if everyone is good at math, but not everyone needs a math PhD. Additionally there are diminishing returns on the time spent learning more and more math vs. other things.
In any case there have already been tons of math resources available for a while now.
A better example would be chemistry. Should everyone be spending time learning chemistry, why or why not?
Intellectual obesity is a thing - simply knowing more things isn’t inherently useful.
Also, writing a computer program because you are slow at solving math problems is a sign that you may need more math practice. It's not like you were handing my in your homework with solutions your wrote for harder problems that you wrote.
On the other hand, I know there are people who enjoy math but struggle to pay attention and learn from a "sit quietly and listen" lecture-based model. (My husband is in this camp.) I can see how the back-and-forth conversational approach of these AI tutors could be really helpful for students like this.
They're still a small minority of the student population, but maybe a different minority from the students who are already excelling academically. If Khanmigo helps only 5% but it's a slightly different 5% from the students who are best served by the current system, I see that as a win, even if it's not the educational revolution that is being hyped.
[0] https://danmeyer.substack.com/p/the-kids-that-edtech-writes-...
A lot. A lot, lot. There are entire states full of people in India who don't have access to good teaching. Who travel absurd distances and stay away from their family for years to get access to good education. In the last few years as kids in my extended family grew up, I am amazed by how much they just learn from YouTube. One of the biggest educational startups started from a free education YT channel.
It teaches you how to think and how to learn which is valuable no matter what.
Most people are still employed if they focus on history or literature. They just might not be making a lot of money or working in history or literature, though some do. Plenty of people in highly "employable" degrees also find themselves unemployed for a long time once they graduate.
The reality is that getting a college degree, regardless of the major, increases your chances of employment afterwards and potential lifelong earnings enormously over only a HS degree.
But as you implied, most of the worst trends in higher-ed, a real corruption of its original intentions and values, have come from a obsession with the idea that higher-ed is solely about employee training.
This is supported by the fact that there are hard limits to computation, in terms of both Computability Theory, and Complexity Theory that are often disregarded by the novice and magical thinker alike despite being largely solved fields by experts in Computer Science, at least in terms of Computability Theory.
Most of this work was completed in the 1950s.
This is not a difficult a subject, but most people online simply are unwilling to do the work to rationally learn this and instead choose to try and hide their own ignorance because they feel inadequate, or perhaps they are engaging in something more malevolent. The reasoning (false justification) doesn't really matter.
It is, however; quite telling when expert's have validated these things, and yet any mention contrary to a narrative provokes downvotes in a public forum to remove it from view. A perfect example of intentional actions done by third-parties trying to misinform others about the risks (by those parties actions removing legitimate and valid information).
For those with the cajones to actually learn this stuff. Here is a link. It seems very jaron laden but it provides a true understanding of how computers actually work which is sorely lacking in the youth of today.
MIT 18.404J Theory of Computation https://www.youtube.com/watch?v=9syvZr-9xwk&list=PLUl4u3cNGP...
The only net benefit AI has towards society is towards destructive ends because there are far more destructive people whose efforts scale more in general when compared to the good people today. These people often do things which cannot be undone (salting the earth and burning the bridge).
Simply not having a sufficient background, or failing to perform basic due dilligence often will place most people in that destructive cohort regardless of their own personal beliefs.
Outcomes matter more than intentions, and the devil is always in the details.
A perfect example of this potential landmine of a field would be anything dealing with the underlying mechanisms involved in human to human communication.
There is an uncanny valley, and distorted reflected appraisal will occur inevitably in any human to AI interface seeking to mimic human communication.
Case studies from torture during the Korean Conflict (1950s) show that distorting reflected appraisal leads to either psychotic or dissociative behavior that progresses as exposure increases, it can permanently break people to the point where they cannot recover. Often, the first thing to go is rational thought.
Multiple experts cover this and confirm the findings (Lifton, Meerloo).
So what do you suppose will happen when AI distorts reflected appraisal (because it can't be tested sufficiently to the contrary) and its then used on the next generation of children? (who are inherently vulnerable to permanent change at that stage of development).
If as a result, they end up either killing themselves or others, or become people incapable of rational thought (which impulse control is correlated with), do you think they'll be alive or survive very long? What indicators would there be that this is happening (none, other than increasing chaotic violence)
Who do you think will be responsible for crippling them if that were to come to pass? Would any kind of justification ever be able to justify that outcome? Could we even recover from intentional crazy-making as a society? (likely not, insanity cannot be cured).
These things are not toys. Outcomes matter.
However, I think its reasonable to say that if you think that some fundamental thing keeps computers from doing what people do then you believe that people are somehow magic. This isn't a super unusual perspective among humans, but its certainly not a particularly common scientific one.
This is complete bullshit, and computability theory has nothing to say about which (if any) cognitive tasks people can do that AI cannot.
> We’ve upgraded parts of Khanmigo to a more capable large language model, which is the software that generates human language. The more capable large language model is called GPT-4 Turbo. Our internal testing shows an improvement in math after we made the switch.
> We are beginning to test the capabilities of a new large language model called GPT-4o, and we’re evaluating other models too to see if they are stronger at math.
> We’ve improved the way AI “thinks” during a tutoring session before responding to a student. We have instructed the AI to write out all the ways in which the student may have arrived at their answer. This approach mimics how a tutor in real life works with a student. We’ve found it significantly improves the quality of math interactions.
> We’ve built new tools to track our progress on math.
> We’re sharing math examples and learnings with others in our field so that we can learn from each other.
> We’re studying the latest research papers on math performance.
Sounds like most of what they're doing is related to prompting, chain-of-thought reasoning and similar, on top of a 'vanilla' foundation model. Sounds like something an ambitious student could replicate / improve upon, so given their mission, it'd be cool if they published the exact techniques they're using and their benchmark results.
This was not their objective though.
Renting a 24GB VRAM Runpod is ~0.5$ per hour. How can the math work out unless you have to have a non-profit energy company and a server farm attached?
[1] https://osboncapital.com/khanmigo-ai-public-vs-private/ [2] https://www.runpod.io/gpu-instance/pricing
I heavily use ChatGPT via a desktop client and my own key, and I am at $3.90 for the month after making a mistake where I accidentally was sending a 30k token prompt repeatedly, that cost me about $2. Using an optimized approach, it is VERY hard to incur a $4/month bill.
I imagine that Khan academy has a much better bulk-pricing agreement with openAI, and that they can probably have a caching layer that can look for previously asked questions.
If it actually works, this is the ideal outcome.
I tried Khanmigo. It is counterproductive junk.
It simply doesn't understand what a student knows and it has to idea how to give examples or provide perspective shifts that help students. That's what a good tutor does. It gets stuck explaining the same things in the same ways but with different words over and over again.
It has no idea how to think geometrically vs algebraically and how to switch between the two. And it can't carry out even simple proofs.
this to me is an insane way to think. I do not consider something better because I can ask it a question, something is better because it is more correct.
Are people consulting only one book/source on any given subject? I remember learning Python and C, and it was such a mixture of books/articles/forums/wiki, I can't even remember the process. I do remember understanding pointers easily because of a chapter of another book about memory models in OS. Learning is an iterative process and the expert is just there to shorten the way, not to carry you through it.
I've found Wikipedia to be an excellent resource for math/physics once you're ~halfway through a math degree to have a good baseline level of knowledge to understand it.
The trouble with math is one of the important goals is to teach people not to produce nonsense and to be able to tell the difference between and argument that sounds good vs. an argument that's sound and good. A nonsense generator (presented as authoritative) is perhaps not the best way to do that. Obviously its abilities could change in the span of a couple years, so it's probably still worthwhile to explore how we would use it if it were useful.
[0] https://cemulate.github.io/the-mlab/