So, Khan Academy is experimenting with providing every student with (what they call) a "personal tutor".
Examples given are: in coding and math, identifying potential mistakes in partial solutions, and suggesting how the student might get unstuck; in literature, an AI impersonates characters in a novel so the student can ask them questions (e.g. "Mr. Gatsby, why do you keep staring at that green light?"); in writing, the AI writes with the student, rather than for students.
The flip side is teachers saving time on lesson planning, grading, etc, all the work that's adjacent to actually teaching.
A lot of this is a feature preview for Khan Academy, and a big part of it is just the ReACT pattern[1] (have people settled on this name?).
To me, even all of the above is just an efficiency increase, which means workloads will increase to fill the available time, and we might see student to teacher ratios in the order of tens of thousands to one. How much of a future teacher's career will be about maintaining their AI systems? How much of learning will be like that too?
I don’t think it’s ever safe to procnosticate on the basis that something is “Just an efficiency increase”. Plenty of tech that is just an efficiency increase has ended up having profound impact on society.
>just an efficiency increase has ended up having profound impact on society.
truer words haven't been spoken. one big one off the top of my mind was the us governments decision to allow civillians to access gps at the same resolution as the arms. with that, all sorts of applications that require realtime location tracking. uber, lyft, rappi, gojeck off the top of my head.
And efficiency is everything. People defending bad education system using "gov has no money" "country has no teacher" excuses while the higher education is pumping out unemployed graduates every year. They can act more actively to make teacher as an job option to people. They just don't like problem being solved. Now GPT4 personal tutor exist, make them no excuse to not provide better education to people.
Do teachers as we know them even exist in a world where that idea of tutoring goes to scale ?
I don't think they do. It sounds more like a world, where in person testing and certification is extremely important when a human skill needs to be evaluated.
Do teachers as we know them even exist in a world where that idea of tutoring goes to scale ?
Do students as we know them even exist in such a world ? I'm thinking if AI can effectively teach (AI asks questions to the student, gives feedback and leads the lesson, reacts to situational events, maintains discipline etc.) then AI can also do. Aside from the romantic take of learning for the pleasure of learning why would we assume that such AI teachers aren't also going to be AI employees. Why would humans even have the drive to learn stuff on a mass scale if the main or one of the main motivators for education is the market (or is it ?).
The tutors scale up,
but so can the testing. Instead of standardized testing, every student can get interviewed as if the teacher had only them as a student and be given an in-depth assessment of their skills and knowledge, rather than being simplified to a letter grade.
Pedagogy is an area of study outside of my expertise, but I do know there are institutions that didn't give grades.
One of the biggest factors impacting education outcomes is students per teacher. Students tutored one-on-one learn better than students in small classes, and students in small classes learn better than students in large classes. Because if there are fewer students per teacher, the teacher can devote more time and attention to each student. AI promises to put a decent one-on-one tutor in the hands of every student for a fraction of the cost of human tuition, potentially bringing about a huge leap in educational attainment. No longer will private tutors be restricted to the rich only.
>No longer will private tutors be restricted to the rich only.
This is probably the number one thing that excites me about AI. Throughout all of history, when you look back at famous scientists, there's a clearly obvious pattern. Almost without exception they were children of enormous privilege who were given the opportunities to study and be tutored individually throughout their life. Unlocking that for every child on earth is going to accelerate human progress faster than anything else.
You are either being dishonest or you're in for a major disappointment. There's a lot more involved in that "opportunity" than a mechanism for low cost feedback. One's material circumstances predominates their ability to take advantage of such mechanisms, hence why strictly increasing school budgets or teacher quality has never succeeded in significantly improving overall outcomes for children living in poverty. I suppose maybe you have to experience it to grasp the extent to which the havoc and insecurity that living in poverty in brutally individualized Western countries ensures, but it's been shown over and over again that these shortcuts simply don't work. You have to address the poverty itself.
That's one factor. But the way I've learned best, is through stories. My teachers told all sorts of stories, and it made me very engaged in my classes. How can the same be said with AI? How can it make an engaging learning experience?
Even if you want it to, how could it make a very engaging environment when deep down inside you know it's fake?
Decent? These current LLMs have more knowledge than every teacher on the planet combined. I’m very hopeful to see what finetuned models come out on top of the base gpt4 or lama
That's a little like saying Wikipedia has more knowledge than every teacher on the planet combined. It's being able to bring the relevant knowledge to bear on the student's understandings that matters, and that depends not only on the prompts the student provides the teacher/tutor/LLM, but on the teacher/tutor/LLM's understanding of the domain, the student's point on the learning curve, the student's level of frustration and receptivity, how people learn, and probably a dozen things that aren't coded into how LLMs respond.
But is it just knowledge or is it the human connection that makes a difference?
Students have access to all the information they need on Calculus through Google and other resources, for instance, but actually sitting down with a living, breathing human being and having them help you through your troubles...to me, that cannot be replaced with an AI.
If access to knowledge was all we needed to acquire and maintain intelligence in a specific area, we would all be a lot smarter...
'Disproven' is too strong, it's trivially true at the extreme edge cases. But smaller class sizes fail to eliminate all the performance gaps between schools, so the idea of class size (and/or school funding) being the explanation for all school performance gaps has been disproven. Particularly, smaller class sizes can't eliminate a performance gap caused by different conditions at home, whether students bully each other for trying to learn, etc.
But there's really no question that a small class where teachers can know each individual student is better than a large class of hundreds of students or more, where the teachers can't realistically remember who is who.
Jesus, no! Not all of my teachers were great, and most of the knowledge I got, I earned on my own. But they were part of a world that had nothing to do with the world of my parents--my teachers were learned, where my immediate family was not. My teachers were, if nothing else, an example to follow, a light, a guide.
>> No longer will private tutors be restricted to the rich only.
It's not as simple. I've met plenty of poor people who leave well-tutored rich folk to bite the dust, and the other way around too. That's neither here nor there. A kid with all the tutors but with parents or an environment that doesn't favor learning, gets nowhere. That I have seen too, more times than I can count. And, let's face it, we have had the Internet for a few decades now, bursting to the seams with free knowledge. What difference will it make an AI system of dubious trustworthiness?
Just ignore the AI hype herd. People who are pulling predictions out of their ass, have never had to deal day to day with the bugs these systems produce. You will never see a single person who works on production AI(not toy ai or research ai) making predictions cuz they are too busy staring at exploding bug lists.
Maybe AI is going to save education, but the next few years are going to be very rough. I built a plugin for Chrome that helps detect cheating [1] so I've been talking to a bunch of teachers lately and pretty much everyone acknowledges that ChatGPT is a game changer for how schools will need to run. Plagiarism, tutors and parent help have been around for a while, but they still required most students to put in some effort. Now, any student can have chatGPT generate a book report for Lord of the Flies in less than 5 minutes. The common theme I hear from teachers is that the education model will need to flip, where students do more writing in class and more learning outside of class. It sounds simple, but it's actually a massive change to implement and things are gonna be very tricky over the next few years until the dust settles more.
[1] https://www.revisionhistory.com, which I originally thought would be most useful for cheating, but interestingly, many teachers are more interested in using it to help students work through revisions. I've gotten feedback from teachers that they plan to have students use ChatGPT to start their assignments and then task them with updating, modifying and annotating them, so I'm trying to figure out which features to build to support that.
I hope it will force the current system which favours a specific subset of learners to change. I hope it will evolve towards a collaborative system where learners are not tested as specimens in a vacuum. The current system is all about making it easier for the system to do its thing instead of the user(learner) getting something valuable out of it.
> I hope it will force the current system which favours a specific subset of learners to change. I hope it will evolve towards a collaborative system where learners are not tested as specimens in a vacuum.
That will also favor a specific subset of learners.
Current education system is broken. You can't learn anything. Learning need feedback loop, but this do not exist.
In school, you hand in your homework, get the feedback only after Christmas. Because human cannot evaluate 100 students' homework efficiently.
Majority of academic textbook are trash. Some author even say "we have 300 new exercises in this edition" proudly, while the solution is often either nonexistent, or at the end of the book. Why at the end, not next to that problem? Or next page? Why wasting my time to flip pages back and forth?
If the content of exercise is important, why not include that into the text?
If instructor is needed, why not making every concepts into Q&A as they had already been explained in office hour or email somewhere else? Why zero effort being put into knowledge accumulation?
Human failed at education. ChatGPT obviously is a more knowledgeable entity, personally I prefer that more than human instructor.
I'm sorry, but I've taught a lot of students at the university level. Some of them have exactly this attitude. You're simply wrong. And on the long term path to failure.
The exercises are invaluable. You don't get the answers because then you look at them. The whole point of the exercises is for you to do them, and then for you to figure out if you are right or wrong!
The content of the exercises isn't in the book because you need to actually do things to learn. You cannot learn math, physics, and many other topics passively.
You're ignoring the main lessons and features of the system.
And no. There are countless amazing books. The fact that you can't see that is because you don't understand how to learn.
Yeah reading and doing stuff is how you learn any topic, getting the right solution is not really the point (and arguably should not be in the book in the first place).
The main issue with education is grades and how obsessed people are with those as a sorting mechanism for students.
And why are they not doing the exercises? Are they lazy? The moment a significant percentage of learners are alienated by the system then there is something wrong with the system. Currently a specific subset of learners fit into this system. The rest either needs extraordinary grit or fail since it is not setup for them to succeed. I for one welcome AI to burn some of our old decrepit institutions to the ground.
The problem with linking success to attitude is the counterfactual; Would students with the right "attitude" been successful regardless?
That's why more formal academic studies are needed for topics such as these rather than anecdotes, because we'll find a whole range of contradictory opinions.
If you choose to fight with me, I fight with you. You dig your only grave. You just jump out and admitted that you are the problematic human teacher that causing social problems.
I do my homework when I feel like I need them. Accessibility to the solutions is a problem. Learning though examples is another prominent way that you don't see. People can choose to investigate into the problem right now or absorb knowledge then think about stuffs. I have 80 years in my life expectancy to do exercise and indeed solving problems everyday is that. What is the point of "figure it out yourself" about some useless math at the moment when people have their whole life to think about it? People know nothing because you never tell anything. You failed your job as a teacher. You are doing excellent job of not documenting stuff and being anti knowledge accumulation.
I'm sorry to read you've had such an awful educational experience. Where was this? It does not resonate with my experience at all. I had my fair share of awful teachers, but the handful of excellent teachers really made a difference. The course material was generally pretty decent as well.
My formal education ended some 20 years ago, and I'm sure things have changed, and from what I hear from younger friends and relatives, it doesn't seem to be for the better ...
I think I read more textbook then average person to a extend that can call myself textbook reviewer. People usually stuck with only one book that is being assigned to them at class, But as a self-learner, I read all, literally all, linear algebra, discrete math, stuff like that, all just terribly written.
I think it's broken, but for other reasons. In the UK at least, secondary school exams are overwhelmingly about memorisation and prediction of what questions there will be rather than applying logical or investigative thinking. There's very little practical work, you sit in a chair and listen to the teacher or things on the whiteboard most of the time. Also, there's a majority feminine influence in education. All of these factors lead to me believe that education simply does not cater to males, and soaring grades of girls confirms this. In the UK, white males have been the poorest performing demographic on exams for the past decade. But nobody wants to talk about it interestingly.
I was fortunate to get an apprenticeship at the age of 16, which enabled me to get into the real world of work and build experience years ahead of everyone else. It's a completed overlooked path - most students still think they need a degree to get a good job.
> secondary school exams are overwhelmingly about memorisation
That's depressing. It was starting to move away from that when I did my physics and chemistry A-levels in 1974, had been for some years in fact with Nuffield courses emphasizing exploration and comprehension rather than simply getting the 'right' answer. The Physics A-level had a substantial section that tested the student's ability to understand the results of experiments.
Paper is more focused on "feedback benefit learning" instead of how long should people wait for feedback.
Seems that you are quoting another paper instead. But I don't think it is definitive, as you see, n=27. Also the experiment setting is not about how long after initial test, it is about how short before the next test.
Problem with education is that it has a dual goal of both educating everyone and "filtering the cream of the crop". On the long term such goals begin to clash, but it's much easier to do the latter than the former.
It's easy to blame the individual, but I wonder what are the collective social costs of such stringent attitudes on the long term.
The AI I want in education generates a test and then based on my responses to questions figures out the exact content I need from a huge library. Imagine asking a student 15 math questions with adaptive difficulty in a dynamic test to figure out the next content they need to learn from complete videos of JK-Uni mathematics? Instead of teaching to an age, we give students exactly the content they need right now.
This is gross posturing and indicates future rent seeking behaviour. Imagine getting in front of people and telling them that some corporation trained the font of wisdom, and that you'd be happy to charge fuckin' admission! What absolute dogshit. All that future rent seeking for a glorified DAN prompt. This is the digital equivalent to making everyone do bullshit jobs because we never figured out how to deal with increased worker productivity from manufacturing.
I really see the potential here (if it is not already happening) for fine-tuned LLMs to start generating bespoke defence and economic policies. Think about the amount of data you could feed a system about orthodox management strategies, current supply chains, demographics etc.
Probably at least 50% of current global governments / bureaucracy could be wholesale replaced by a single decent LLM, and the economic and social outcomes would be far better.
I think teachers should probably use ChatGPT for a couple days before starting this conversation. It is the common ground. Not knowing what ChatGPT is able to do hinder valuable debate.
Readit News
Examples given are: in coding and math, identifying potential mistakes in partial solutions, and suggesting how the student might get unstuck; in literature, an AI impersonates characters in a novel so the student can ask them questions (e.g. "Mr. Gatsby, why do you keep staring at that green light?"); in writing, the AI writes with the student, rather than for students.
The flip side is teachers saving time on lesson planning, grading, etc, all the work that's adjacent to actually teaching.
A lot of this is a feature preview for Khan Academy, and a big part of it is just the ReACT pattern[1] (have people settled on this name?).
To me, even all of the above is just an efficiency increase, which means workloads will increase to fill the available time, and we might see student to teacher ratios in the order of tens of thousands to one. How much of a future teacher's career will be about maintaining their AI systems? How much of learning will be like that too?
[1] https://til.simonwillison.net/llms/python-react-pattern
truer words haven't been spoken. one big one off the top of my mind was the us governments decision to allow civillians to access gps at the same resolution as the arms. with that, all sorts of applications that require realtime location tracking. uber, lyft, rappi, gojeck off the top of my head.
I don't think they do. It sounds more like a world, where in person testing and certification is extremely important when a human skill needs to be evaluated.
Do students as we know them even exist in such a world ? I'm thinking if AI can effectively teach (AI asks questions to the student, gives feedback and leads the lesson, reacts to situational events, maintains discipline etc.) then AI can also do. Aside from the romantic take of learning for the pleasure of learning why would we assume that such AI teachers aren't also going to be AI employees. Why would humans even have the drive to learn stuff on a mass scale if the main or one of the main motivators for education is the market (or is it ?).
The tutors scale up, but so can the testing. Instead of standardized testing, every student can get interviewed as if the teacher had only them as a student and be given an in-depth assessment of their skills and knowledge, rather than being simplified to a letter grade.
Pedagogy is an area of study outside of my expertise, but I do know there are institutions that didn't give grades.
We are removing the last vestiges of empathy and interiority from our reading process? Good good, carry on then.
This is probably the number one thing that excites me about AI. Throughout all of history, when you look back at famous scientists, there's a clearly obvious pattern. Almost without exception they were children of enormous privilege who were given the opportunities to study and be tutored individually throughout their life. Unlocking that for every child on earth is going to accelerate human progress faster than anything else.
Even if you want it to, how could it make a very engaging environment when deep down inside you know it's fake?
Student: "Hey LLM, I'm having trouble learning this concept."
LLM: "Here's a real story that a teacher shared with us that will help explain this topic..."
(Later)
LLM: "Remember that story from before? Here's another story from that same teacher to further explain..."
So, like all good fiction from the Iliad and Odyssey to soap operas and Star Trek and wrestling kayfabe?
If anything I expect a problem where AI will become too capable at making things entertaining, such that reality no longer appeals.
Students have access to all the information they need on Calculus through Google and other resources, for instance, but actually sitting down with a living, breathing human being and having them help you through your troubles...to me, that cannot be replaced with an AI.
If access to knowledge was all we needed to acquire and maintain intelligence in a specific area, we would all be a lot smarter...
But there's really no question that a small class where teachers can know each individual student is better than a large class of hundreds of students or more, where the teachers can't realistically remember who is who.
>> No longer will private tutors be restricted to the rich only.
It's not as simple. I've met plenty of poor people who leave well-tutored rich folk to bite the dust, and the other way around too. That's neither here nor there. A kid with all the tutors but with parents or an environment that doesn't favor learning, gets nowhere. That I have seen too, more times than I can count. And, let's face it, we have had the Internet for a few decades now, bursting to the seams with free knowledge. What difference will it make an AI system of dubious trustworthiness?
this!
AI isn't as good as a great teacher but its better than a bad teacher and there aren't enough good teachers to go around.
[1] https://www.revisionhistory.com, which I originally thought would be most useful for cheating, but interestingly, many teachers are more interested in using it to help students work through revisions. I've gotten feedback from teachers that they plan to have students use ChatGPT to start their assignments and then task them with updating, modifying and annotating them, so I'm trying to figure out which features to build to support that.
Otherwise seems like snake oil thing.
That will also favor a specific subset of learners.
In school, you hand in your homework, get the feedback only after Christmas. Because human cannot evaluate 100 students' homework efficiently.
Majority of academic textbook are trash. Some author even say "we have 300 new exercises in this edition" proudly, while the solution is often either nonexistent, or at the end of the book. Why at the end, not next to that problem? Or next page? Why wasting my time to flip pages back and forth?
If the content of exercise is important, why not include that into the text?
If instructor is needed, why not making every concepts into Q&A as they had already been explained in office hour or email somewhere else? Why zero effort being put into knowledge accumulation?
Human failed at education. ChatGPT obviously is a more knowledgeable entity, personally I prefer that more than human instructor.
The exercises are invaluable. You don't get the answers because then you look at them. The whole point of the exercises is for you to do them, and then for you to figure out if you are right or wrong!
The content of the exercises isn't in the book because you need to actually do things to learn. You cannot learn math, physics, and many other topics passively.
You're ignoring the main lessons and features of the system.
And no. There are countless amazing books. The fact that you can't see that is because you don't understand how to learn.
The main issue with education is grades and how obsessed people are with those as a sorting mechanism for students.
That's why more formal academic studies are needed for topics such as these rather than anecdotes, because we'll find a whole range of contradictory opinions.
I do my homework when I feel like I need them. Accessibility to the solutions is a problem. Learning though examples is another prominent way that you don't see. People can choose to investigate into the problem right now or absorb knowledge then think about stuffs. I have 80 years in my life expectancy to do exercise and indeed solving problems everyday is that. What is the point of "figure it out yourself" about some useless math at the moment when people have their whole life to think about it? People know nothing because you never tell anything. You failed your job as a teacher. You are doing excellent job of not documenting stuff and being anti knowledge accumulation.
My formal education ended some 20 years ago, and I'm sure things have changed, and from what I hear from younger friends and relatives, it doesn't seem to be for the better ...
I was fortunate to get an apprenticeship at the age of 16, which enabled me to get into the real world of work and build experience years ahead of everyone else. It's a completed overlooked path - most students still think they need a degree to get a good job.
That's depressing. It was starting to move away from that when I did my physics and chemistry A-levels in 1974, had been for some years in fact with Nuffield courses emphasizing exploration and comprehension rather than simply getting the 'right' answer. The Physics A-level had a substantial section that tested the student's ability to understand the results of experiments.
Is this supposed to be a problem? Research says that delays in getting corrections are not all that important, you still end up learning even if there's a fairly large gap. https://pcl.sitehost.iu.edu/rgoldsto/courses/dunloskyimprovi...
Seems that you are quoting another paper instead. But I don't think it is definitive, as you see, n=27. Also the experiment setting is not about how long after initial test, it is about how short before the next test.
https://link.springer.com/article/10.3758/MC.37.8.1077
> The participants were 27 students enrolled in Grade 6 at The School at Columbia University in New York City.
It's easy to blame the individual, but I wonder what are the collective social costs of such stringent attitudes on the long term.
Probably at least 50% of current global governments / bureaucracy could be wholesale replaced by a single decent LLM, and the economic and social outcomes would be far better.