I was glad to see this paragraph included in the article:
> By the time they take the eighth-grade tests in the spring of the year, they already know which high school they will be attending, and their scores on the test have no consequences. “The eighth-graders don’t care; they rush through the exam, and they don’t check their work,” Abbott said. “The test has no effect on them. I can’t make an argument that it counts for kids.
Anytime anyone makes an argument that teachers should be evaluated based on their kids' test scores I have to explain about the problems of misaligned incentives. If the kids are not motivated to do their best, it's not going to be a fair evaluation of the teacher even if you manage to solve ALL the many other problems with high-stakes testing. And a lot of students aren't motivated to do their best on these kinds of tests, for a variety of reasons.
I teach 6-8th grade music at a local charter school, and while my performance isn't tested through standardized exams, I can attest to my colleagues' enormous frustration with this very issue.
For example: during our standardized testing this year, the school put in huge amounts of effort to align student incentives, most of which were silly and clearly ineffective. (My personal favorite? Disallowing students to bring their own books to read after they finished the test to "encourage" them to check their work instead of hurrying through so they could read the next chapter of The Hunger Games. Talk about easy ways to incite student frustration and contempt!)
I can attest to students finishing a test in less than 10 minutes despite being given more than an hour, or refusing to do any of the test at all. I've heard horror stories of students using the multiple-choice answer documents to make pictures by filling in the bubbles, or simply choosing to color in bubbles so lightly that they can't possibly be read. I know of examples in prior years wherein students chose to write essays about how much they hate testing on the essay portion instead of addressing the infantile tripe they were assigned.
Now, these are obviously exceptions to the rule (I'm a bit biased, but I think our students are almost uniformly excellent), but it is amazing how much damage one student can do by choosing not to care. (Or, in some cases, using their answers to vent their frustration).
And why should they? As far as I know, rewarding students for higher performance is disallowed, leaving teachers with very few tools with which to convince students that these tests matter. Sure, you can go on and on about how their scores will be "important" in their future (which, to my knowledge, isn't always true), but that only goes so far. Sometime during the 2 weeks of intensive testing, the kids just don't care anymore.
I'm all for assessing the performance of teachers, and I'm not sure how to fix the problem (or even if it can be fixed), but one thing seems clear to me: they're doing it wrong.
Sounds similar to the situation of a programmer with an app in the Android store. Your users will give 1 star reviews for minor problems or as feature request blackmail, without caring they're undermining your livelihood.
The incentive for the person grading is just disconnected from the supposed meaning of the rating itself.
Want an incentive to have students perform better?
Schools need to implement a standard performance-based path for skipping grade levels so that students can get out sooner.
I was reading Hawking, Nietzsche, and college-level humanities books in my free time in 8th grade because I was bored out of my gourd with my classes. In 12th grade, I was forced to take classes that taught, for example, how to balance your check book.
I'm pretty sure that when he's older, I'm going to encourage my son to get a GED when he's 16 yo or so, just so that he doesn't have to put up with all of the bullcrap classes.
And, I don't care about your socialization arguments. My son is 2 yo and already has great table manners, is polite, shares, and is quite empathetic. Most of the "normal" young people I meet these days behave either like wild Indians (feather not dot) or slovenly Barbarians.
Yeah, you're incentivized to punish your students for disobedience, and not respecting The Test. That's not a bug, but a feature. (At least from the perspective of those who formulate policy.)
Would you want your employees to seriously disrespect tasks they morally disagree with, like sabotaging your advertising system because they think advertising is harmful to society, or changing your website to admit how you're raiding people's pensions? Telling off your big corporate/military customers? (If you did, you probably wouldn't last long in your position.)
I don't know many bosses who want their subordinates to rush through their tasks, in order to read some book of personal interest, or to build a competing company with better service.
This is almost exactly what I was going to write. Until you get to ACT or SAT's, students really don't have much incentive to do well on standardized tests. Generally they have no effect whatsoever so many kids don't take them seriously. The tests mean so much more to the teachers and the school districts than the do to the kids. This makes them a terrible proxy for how the teachers and districts are actually doing.
I probably spent over 100 hours preparing for the SAT/ACT exams as a 16 year-old. Given that I received over $50K (early '90's) in merit-based college scholarship money based mostly on these scores, I effectively "earned" $500/hour studying. I have yet to earn this much as an adult.
I did relatively badly in my last year of school compared to my previous years as I had been given an unconditional offer for university, so spent most of my time that year reading books and dicking around.
I'm surprised to see so many HNers come out against standardized testing. Sure, it's imperfect and this teacher got screwed, but how are administrators supposed to make informed, data-driven decisions on what the right course of action is? Many of us extol the virtues of A/B testing landing page designs and gathering deep metrics and analytics but turn around and slam educators for making decisions based on standardized test scores.
Give me a better metric for success that you can measure over the course of a year.
I'm surprised to see so many HNers come out against counting lines of code. Sure, it's imperfect and this developer got screwed, but how are managers supposed to make informed, data-driven decisions on what the right course of action is? Many of us extol the virtues of A/B testing landing page designs and gathering deep metrics and analytics but turn around and slam managers for making decisions based on line code counts.
Your post is a logical fallacy. The fact that a bad metric exists does not mean all metrics are bad.
To address your specific fallacy, the output of coding is heterogeneous while the output of teaching is homogeneous. If I code today, it's a realtime optimization system. Tomorrow it might be a search product. When I taught, the output was always the same: students who understand calculus.
Unlike website visitors, the teachers in a school are not acting completely independent of each other, but part of a social group. Even if you could adequately measure their 'teaching quality', you might miss factors that contribute to the success of the school as a whole.
And measuring 'teaching quality' is not a simple as counting ad click-throughs. Education is at least as much about helping people learn about their role in the world, as it is about learning facts, concepts, and tools. Just measuring the increase of students' scores on standardized tests misses a lot of the impact teachers have on their students' minds.
Nobody in his right mind would judge a developer solely on the amount of features he has implemented during the previous year, but also on how robust and maintainable the code is.
I you want to take these factors into account, you have to actually read the code, understand it, and ideally talk to the developer about it.
Unless you do this, you can not judge the quality of the code.
The same holds true for evaluating teachers, imho. Administrators need to talk to students, parents and teachers and form a comprehensive image of a teacher's ability. I really think it is that simple. Certainly not perfect. But not less perfect than standardized testing.
As a German, maybe I am misunderstanding some properties of the US educational system.
This is spot on. Does anyone know of a single instance of standardized teacher testing that recognizes important teacher qualities beyond their skill in imparting knowledge to students?
Doesn't that just imply that tests are failing to measure what we want teachers to teach and we should fix the tests ?
The whole reason that test results are being used as opposed to manual assesment, is that test results have proven to be a better predictor of long term outcomes. Test results aren't perfect but they're better than what was previously used.
Remember the kerfuffle about the bad performance of Germany in the Pisa study? We also sometimes standardized testing, but not to compare individual teachers, but to argue endlessly the slightly different curricula/school policies of the different German states.
Addendum:
My understanding is, that in the United States anyone can relatively easy train to be a school teacher. In Germany they are tenured civil servants with at least 3-4 years of study.
Education is at least as much about helping people learn about their role in the world, as it is about learning facts, concepts, and tools.
Could you explain what this means more precisely? Specifically, how can I differentiate a student who knows "about their role in the world" from one who doesn't?
>Give me a better metric for success that you can measure over the course of a year. //
Define success.
If one gets the best score but has no social skills, no friends, no fun and no creativity then is that success.
This (only considering a test score) is like doing A/B testing but only looking at impressions and not at conversions, or only looking at conversion rates and not at conversion value. In web terms you can succeed by reducing both your impressions and conversion rates (by taking more money of less people and only attracting motivated customers).
Sorry that's more destructive than constructive WRT what the right course of action is.
Are there [large scale] education systems that consider happiness of pupils, reduction in bullying, positive social interaction and such to be metrics to assess and improve?
I don't think you should be optimizing for test scores though, you should be optimizing for "education", which is ill-defined and hard to measure. In the analogy to A/B testing, you wish to maximise your profits, not the number of email addresses you collect.
I had this discussion with a guy who is now the state superintent of public education. Our discussion led to classroom visits of about 10 minutes per instructor, multiple times each semester. Evaluation criteria were: 1) Are learning objectives for the lesson clearly visible or otherwise available? 2) What percent of students were on task? 3) ... I forget the others, but there were five. (It's been about five years, but I think they included the students knowing how they were to be evaluated, and the teaching style used, e.g., lecture, group work, etc., and whether the instructor was using data to inform the approach. It came down to the standard description of leadership: vision, expectations, support, feedback)
He was pretty firm that this method would be better—he'd spent years thinking about it—but we both agreed that it required a level of intervention by the administrator that although it could reasonably be expected was unlikely. Using just the test data is the lazy way, which means it's the method most will use.
Still, I believe _some_ standardized testing is important. ... but there are two types of tests: norm testing and standards testing, and both types have their uses.
Give me a better metric for success that you can measure over the course of a year.
This is assuming that the net effect of the tests is positive, thus we should keep them until we find something better. But a lot of comments are pointing at the harm caused by standardized testing. For people who think the net effect is negative, the rational thing to do is get rid of them until we find something better.
So if you have 10 good teachers, you just rank them and kick out the "worst" 2?
If your rank the teachers in a strict order, you have to make someone the worst and someone else the best teacher. But is the administration going to check how much worse the worst teacher really is? I doubt it.
No. You set performance goals for teachers specific to the teacher and district, evaluate the performance objective and make educated decisions based on that data. It doesn't need to be ranked, but it does need to be empirical.
As a parent of four children (but living in a different state), I had to think for a minute about what the real problem is here. If I knew that a particular teacher was teaching well in advance of the meager United States curriculum, perhaps even coming close to east Asian standards, I would (as a parent) take that into account when looking at teacher ratings based on a slower curriculum pace. My question might be, "How do I sign up?" rather than "What is the matter with this teacher?" (By the way, the school's answer about how to sign up is found on its website:
The school accepts new applicants for sixth grade or seventh grade, but not for eighth grade.)
But, really, the problem with the system is not mostly about how teachers are rated, but how little power most parents have to shop. If a teacher is doing a good job for a particular learner, most parents will be glad to seek that teacher's instruction. And if the parents are shopping, the hard-to-measure things in the aggregate that are crucially important at the individual level, for example whether or not a teacher encourages a child to do his or her personal best, will be given proper weight among all the trade-offs involved in choosing one teacher over another. But right now the great majority of pupils, in New York City and elsewhere, are mostly assigned to teachers without parental power to shop, and teacher advancement in the profession is based mostly on seniority and degrees attained
rather than on the basis of the teachers meeting learner needs better than other teachers. The best way to promote system reform is to let the parents have more power to shop. They will ask for the information they need and weigh it in appropriate ways.
The 'let's give parents the ability to shop' ethos is what directly led to the current clusterfuck that is the US public education system, through NCLB and its state equivalents.
I'm not opposed in principle to the idea of statistically ranking teachers and giving the power of choice to parents, but you guys have royally fucked it up in practice.
Signed,
A victim of the evil socialist Canadian public school system.
I see that this thread has been busy overnight. I'll quote here first another kind reply I received, and then respond both to it and to your reply.
If you give the parents the ability to "shop", what happens when everyone thinks that teacher X is the best English teacher for 7th grade and they ALL want their kids in her class?
For me, this is not a theoretical question, because I live in a state of the United States where there is actual "power to shop." In the entire state of Minnesota in the United States, there is public school open enrollment,
and the school district for the neighborhood in which I live includes open-enrolled students from the territories of more than forty other Minnesota school districts, with funding following the students on a per-capita basis.
but parents with power to shop and funding that follows students allows school districts with the better programs, overall, to thrive and produce innovative new programs (e.g., language immersion programs, specialized fine arts programs, and school-within-a-school programs for highly gifted learners) and the schools that are forced to lay off teachers (alas, by seniority rather than by effectiveness) are the schools with laggard overall programs.
the evil socialist Canadian public school system
This is from your comment. I'm not aware what province you grew up in, but it's my understanding that there have been elements of school choice in some Canadian provinces in our liftime,
In principle, as already observed in reality, it is perfectly possible for there to be a general public subsidy for some service that is deemed to have a positive externality, while still allowing user choice of the provider of the service.
There are other international examples of school choice. I particularly like the example of the Netherlands with its very wide array of choices for parents, all at equal publicly subsidized expense,
By the OECD testing program called PISA, the Netherlands does as well as or outperforms other countries both as to helping students from disadvantaged families
I'm not opposed in principle to the idea of statistically ranking teachers and giving the power of choice to parents, but you guys have royally fucked it up in practice.
I'll heartily agree with you and with other comments here that the current state testing systems, which vary state by state, but which are mostly poorly designed, are a weak basis for publishing ratings of schoolteachers, but as you correctly point out, that's not to prove that a better system of assessing students and their academic progress has NO value in helping parents shop for schools. What I don't want to go back to is the day in which no one had any idea how well any teacher was teaching, because no one was looking, and no one was looking because parents couldn't shop for schools anyhow. If parents can make global evaluations of what is good for each of their children, better incentives exist to improve teaching, improve school administration, and improve all other aspects of the school experience.
Returning to the other comment's thoughtful question,
If you give the parents the ability to "shop", what happens when everyone thinks that teacher X is the best English teacher for 7th grade and they ALL want their kids in her class?
that is precisely the kind of situation that builds curiosity among other teachers about "What is that teacher doing that I'm not doing?" and among administrators about, "What value do families perceive in that class that they don't perceive in our classes?"
As other comments have already pointed out, parents in all countries of the world shop for schools at least by how they choose their residence addresses. But decoupling school choices from residence choices allows schools and families to respond more efficiently to their own mix of trade-offs. When many shoppers prefer one grocery store to another, and take their business to the better grocery store, what usually happens is that the worse grocery store changes the way it does business and improves its overall customer value proposition for shoppers. As I noted above, one HUGE problem with schools all over the United States, even schools in states that do not formally have "union shops" with mandatory schoolteacher union membership, is that administrators have little flexibility in reassigning teachers to the work that they do the best. So today is a somewhat slow process at the margins for schools to improve (by realigning staff assignments) as families shop for the best classrooms. But I've seen what Minnesota school districts have been able to do even within the limitations of the current system, and I'm confident that adding incentive for school improvement by giving learners more power to shop makes as much sense (as a matter of basic public policy) as it does for providing most other services.
AFTER EDIT: Commentary on the link submitted here by a blogger based in New York City,
which I learned about from a Facebook friend who lives in New York State and has been following the controversy on school testing in New York State closely.
While I agree that it's introduced some problems in states where I've lived (Utah, for example), it's also allowed parents to move their children around a bit more easily, and there are some benefits to that.
On the flip side, I'm now in Georgia. We live on the boundary between a poor school district (98% "minority" district-wide) and a rich one (10-50% "minority" depending on the school). I can only "shop" for schools in my poorer district, which are typically around the bottom quintile of schools in the state. The kicker is, because of the geographic boundaries, the bus stop for the "richer" district stops on either side of our subdivision, and our kids pass withing 1 mile of five different elementary schools in the "rich" district on their way to their assigned "poor" one. Shop for schools? I'd love to. I'd even like for my kids to go to a school close to their home. But a long history of bussing (which in our area has reinforced segregation rather than combat it) and civil rights policies perversely means it's more difficult to move to the better schools.
If I think I need a new suit and the only information available to me is the colors of various suits and their thread counts, I might be forced to make a selection solely based on these attributes. But, because I can touch the suits, try them on, and examine their quality in whatever ways I deem fit, I don't have to rely upon price as a measure of quality. Without exchanging experiences with other kids' parents, aren't these "shoppers" forced to make do with the available data? Perhaps a parents' "shopping night" where each teacher gives a short talk on his/her education philosophy and then sticks around for Q/A would allow use of better selection criteria?
> The 'let's give parents the ability to shop' ethos
There's no way to avoid it, even in principle. What if you move?
In the US, home prices are elevated in certain areas because people who live at that address get to go to a good school. So there's still school shopping, when it comes to deciding where to live (it's often a key factor in that decision).
I think that's how it is most places, actually. At least where I grew up in California, I was assigned to a certain high school based on where I live. We could have applied for a transfer, but those aren't often granted. This is all for the public school system, of course. There are private schools as well, which operate differently, but I think for the most part people go to public schools.
If you give the parents the ability to "shop", what happens when everyone thinks that teacher X is the best English teacher for 7th grade and they ALL want their kids in her class? Or what about the kids whose parents aren't as involved (maybe they both work full time or it's a single-parent household) and therefore don't get a chance to hear about which teachers are best... do those kids just get the less-capable teachers?
The LA Times published a database of these value-added results two years ago, for teachers in the LA USD. The LA USD contains a mix of good teachers and time-servers, with the time-servers well-protected by the teachers' union's last-hired/first-fired policy. Teacher performance is not allowed to be a factor in layoffs.
These value-added statistics, which on a per-teacher basis must be incredibly noisy (they measure a difference, the "value added", of two already-noisy performance numbers) resulted in the suicide of a low-ranked teacher:
The evaluations of the same teachers teaching the same subjects in two different classes are almost completely uncorrelated. The distribution of the scores on the scatterplots look almost random.
It's a travesty that perfectly fine teachers are being publicly shamed based on these unreliable metrics.
> “They’re not accepting answers that are mathematically correct,” Abbott notes, “and accepting answers that aren’t mathematically correct.”
I don't believe that 'the professionals' setting up, promoting and administering a system like this can possibly be ignorant of this fact and/or well meaning.
Of course Abbott had to be eliminated as a teacher. She was interfering with the program of deliberate dumbing down.
We just sent in our six year-old's Montessori application for first grade next year. Our suburban public school system has an amazing reputation, and the passing rates for standardized tests are some of the highest in the state. Sadly, our little Aspie with ADHD issues has already expressed his dread of first grade. He doesn't want to do "seatwork", "paperwork", or any of the other drudgery that our local district tells him is "his job". [Quotes indicate words used by teachers with the best of intentions.] I can't figure out the value of this pseudo-rigor. Maybe I'm unusual in my autodidactism, but I can't think of anything I know which I (1) find valuable and (2) learned by force.
I bought a K'NEX roller coaster kit and helped him put it together. At first, he didn't grok the pseudo 3D instructions or the way the pieces fit together. However, after an hour or so, he was rotating our partially assembled segments in his head to compare them to the illustrations. But, this doesn't count as rigorous seat work. Oh well.
Our school actually does an excellent job of avoiding the binary incentives of No-Child-Left-Behind and attempts to provide somewhat individualized instruction to students. Ohio has yet to adopt a value-added measurement scheme, and I wonder how our district would fare, especially if achievement was normalized for socio-economic status (the average household income is over $100K, a high number for Ohio).
We hope that the Montessori school's motivated teachers can use their freedom from mandated curriculum requirements, standardized testing, etc. to facilitate our son's exploration of what interests him. I don't care if his education is seen as incomplete, unbalanced, or otherwise non-standard so long as he grows to become a curious, thoughtful, and creative person. When has someone with these qualities ever failed because he had some perceived weakness in his childhood education?
>When has someone with these qualities ever failed because he had some perceived weakness in his childhood education? //
Define "failed". Failed to get a job, too right. Failed to fit in to society, check. Failed to be happy, seems to go with the territory(!?).
Will you excuse a short aside:I'm interested in the diagnosis of a 6yo as having ADHD - what procedures and tests were performed to come to that decision. In my, albeit limited, experience of primary aged (4-11yo) boys if they're not running around half the time like some sort of sugar rushing lunatic then they're the exception to the rule. Can he sit and watch a TV show for 20 minutes without leaving to do something else?
Without ADHD meds, our son often can't concentrate long enough to finish a sentence. He starts one, shifts to another, and repeats. Often, in the mornings before his meds kick-in, he lacks sufficient impulse control to avoid hitting his mother (also my wife -- the English language seems to lack a single word which denotes both relationships) or threatening her with physical violence. The differences in behavior are striking in their contrast. While I don't like the idea of "medicating away" a problem which could be solved via better parenting, I don't see how we can possibly be effective parents if we had to spend most of our time with him (1) protecting his little brother from his aggression, (2) protecting physical possessions, or (3) attempting to correct behaviors which himself seems to wish he did not have but can not stop due to lack of impulse control. On meds, we can do what normal parents get to do.
If you think that ADHD boys are running around half the time like some sort of sugar rushing lunatic then you do not know what ADHD actually is.
It is not generally a problem of too much energy and lack of focus. Rather it is a problem of inappropriate focus, frequently very intense. For instance my nephew would get so interested in whatever he was doing that he would fail to realize that he needed to pee, and then after he peed his pants he would get very upset that people were making him change his pants when he wanted to do something else.
It becomes a problem in school when children are unable to follow repeated directions because they are unaware that directions have been given, because their interest has been caught by something - anything - else.
Disclaimer: I am the parent of a 7 year old who has been identified as likely having ADHD, though I have not yet done the official screening.
> Can he sit and watch a TV show for 20 minutes without leaving to do something else?
ADHD causes __attention_regulation__ difficulties. Many mistakenly see it as a simple lack-of-attention, but fail to realize that hyper-focus is also a common symptom.
I suggest reading the first few chapters of ``More Attention, Less Deficit'' to gain a basic understanding of ADHD.
This is what happens when education law is created by politicians. Politicians who get their information from lobbyists lobbying on behalf of the test makers. They believe that more testing is good, mostly because it means more money for them. In NY, the ELA is mandatory for students in grades 1-10 iirc. And these are the scores we are judging teachers on. There are many teachers in schools who are given classrooms of students who need more attention and won't perform as well on exams.
We need to seriously rethink education and how we are testing students and teachers.
Readit News
> By the time they take the eighth-grade tests in the spring of the year, they already know which high school they will be attending, and their scores on the test have no consequences. “The eighth-graders don’t care; they rush through the exam, and they don’t check their work,” Abbott said. “The test has no effect on them. I can’t make an argument that it counts for kids.
Anytime anyone makes an argument that teachers should be evaluated based on their kids' test scores I have to explain about the problems of misaligned incentives. If the kids are not motivated to do their best, it's not going to be a fair evaluation of the teacher even if you manage to solve ALL the many other problems with high-stakes testing. And a lot of students aren't motivated to do their best on these kinds of tests, for a variety of reasons.
For example: during our standardized testing this year, the school put in huge amounts of effort to align student incentives, most of which were silly and clearly ineffective. (My personal favorite? Disallowing students to bring their own books to read after they finished the test to "encourage" them to check their work instead of hurrying through so they could read the next chapter of The Hunger Games. Talk about easy ways to incite student frustration and contempt!)
I can attest to students finishing a test in less than 10 minutes despite being given more than an hour, or refusing to do any of the test at all. I've heard horror stories of students using the multiple-choice answer documents to make pictures by filling in the bubbles, or simply choosing to color in bubbles so lightly that they can't possibly be read. I know of examples in prior years wherein students chose to write essays about how much they hate testing on the essay portion instead of addressing the infantile tripe they were assigned.
Now, these are obviously exceptions to the rule (I'm a bit biased, but I think our students are almost uniformly excellent), but it is amazing how much damage one student can do by choosing not to care. (Or, in some cases, using their answers to vent their frustration).
And why should they? As far as I know, rewarding students for higher performance is disallowed, leaving teachers with very few tools with which to convince students that these tests matter. Sure, you can go on and on about how their scores will be "important" in their future (which, to my knowledge, isn't always true), but that only goes so far. Sometime during the 2 weeks of intensive testing, the kids just don't care anymore.
I'm all for assessing the performance of teachers, and I'm not sure how to fix the problem (or even if it can be fixed), but one thing seems clear to me: they're doing it wrong.
[Updated to fix punctuation errors]
The incentive for the person grading is just disconnected from the supposed meaning of the rating itself.
Schools need to implement a standard performance-based path for skipping grade levels so that students can get out sooner.
I was reading Hawking, Nietzsche, and college-level humanities books in my free time in 8th grade because I was bored out of my gourd with my classes. In 12th grade, I was forced to take classes that taught, for example, how to balance your check book.
I'm pretty sure that when he's older, I'm going to encourage my son to get a GED when he's 16 yo or so, just so that he doesn't have to put up with all of the bullcrap classes.
And, I don't care about your socialization arguments. My son is 2 yo and already has great table manners, is polite, shares, and is quite empathetic. Most of the "normal" young people I meet these days behave either like wild Indians (feather not dot) or slovenly Barbarians.
Deleted Comment
Would you want your employees to seriously disrespect tasks they morally disagree with, like sabotaging your advertising system because they think advertising is harmful to society, or changing your website to admit how you're raiding people's pensions? Telling off your big corporate/military customers? (If you did, you probably wouldn't last long in your position.)
I don't know many bosses who want their subordinates to rush through their tasks, in order to read some book of personal interest, or to build a competing company with better service.
> “They’re not accepting answers that are mathematically correct,” Abbott notes, “and accepting answers that aren’t mathematically correct.”
The test just seems to be a retarded thing designed by a clueless bureaucrat.
Give me a better metric for success that you can measure over the course of a year.
To address your specific fallacy, the output of coding is heterogeneous while the output of teaching is homogeneous. If I code today, it's a realtime optimization system. Tomorrow it might be a search product. When I taught, the output was always the same: students who understand calculus.
And measuring 'teaching quality' is not a simple as counting ad click-throughs. Education is at least as much about helping people learn about their role in the world, as it is about learning facts, concepts, and tools. Just measuring the increase of students' scores on standardized tests misses a lot of the impact teachers have on their students' minds.
Nobody in his right mind would judge a developer solely on the amount of features he has implemented during the previous year, but also on how robust and maintainable the code is. I you want to take these factors into account, you have to actually read the code, understand it, and ideally talk to the developer about it. Unless you do this, you can not judge the quality of the code.
The same holds true for evaluating teachers, imho. Administrators need to talk to students, parents and teachers and form a comprehensive image of a teacher's ability. I really think it is that simple. Certainly not perfect. But not less perfect than standardized testing.
As a German, maybe I am misunderstanding some properties of the US educational system.
The whole reason that test results are being used as opposed to manual assesment, is that test results have proven to be a better predictor of long term outcomes. Test results aren't perfect but they're better than what was previously used.
Addendum:
My understanding is, that in the United States anyone can relatively easy train to be a school teacher. In Germany they are tenured civil servants with at least 3-4 years of study.
Could you explain what this means more precisely? Specifically, how can I differentiate a student who knows "about their role in the world" from one who doesn't?
You're actually better off with no information than with bad information.
Define success.
If one gets the best score but has no social skills, no friends, no fun and no creativity then is that success.
This (only considering a test score) is like doing A/B testing but only looking at impressions and not at conversions, or only looking at conversion rates and not at conversion value. In web terms you can succeed by reducing both your impressions and conversion rates (by taking more money of less people and only attracting motivated customers).
Sorry that's more destructive than constructive WRT what the right course of action is.
Are there [large scale] education systems that consider happiness of pupils, reduction in bullying, positive social interaction and such to be metrics to assess and improve?
Nobody is arguing against testing. They're arguing about doing tests where you know the results will be deeply flawed.
He was pretty firm that this method would be better—he'd spent years thinking about it—but we both agreed that it required a level of intervention by the administrator that although it could reasonably be expected was unlikely. Using just the test data is the lazy way, which means it's the method most will use.
Still, I believe _some_ standardized testing is important. ... but there are two types of tests: norm testing and standards testing, and both types have their uses.
This is assuming that the net effect of the tests is positive, thus we should keep them until we find something better. But a lot of comments are pointing at the harm caused by standardized testing. For people who think the net effect is negative, the rational thing to do is get rid of them until we find something better.
If your rank the teachers in a strict order, you have to make someone the worst and someone else the best teacher. But is the administration going to check how much worse the worst teacher really is? I doubt it.
http://www.ps334school.org/admissions/anderson-school-7th-gr...
The school accepts new applicants for sixth grade or seventh grade, but not for eighth grade.)
But, really, the problem with the system is not mostly about how teachers are rated, but how little power most parents have to shop. If a teacher is doing a good job for a particular learner, most parents will be glad to seek that teacher's instruction. And if the parents are shopping, the hard-to-measure things in the aggregate that are crucially important at the individual level, for example whether or not a teacher encourages a child to do his or her personal best, will be given proper weight among all the trade-offs involved in choosing one teacher over another. But right now the great majority of pupils, in New York City and elsewhere, are mostly assigned to teachers without parental power to shop, and teacher advancement in the profession is based mostly on seniority and degrees attained
http://www.nytimes.com/2010/11/19/us/19gates.html
rather than on the basis of the teachers meeting learner needs better than other teachers. The best way to promote system reform is to let the parents have more power to shop. They will ask for the information they need and weigh it in appropriate ways.
I'm not opposed in principle to the idea of statistically ranking teachers and giving the power of choice to parents, but you guys have royally fucked it up in practice.
Signed,
A victim of the evil socialist Canadian public school system.
If you give the parents the ability to "shop", what happens when everyone thinks that teacher X is the best English teacher for 7th grade and they ALL want their kids in her class?
For me, this is not a theoretical question, because I live in a state of the United States where there is actual "power to shop." In the entire state of Minnesota in the United States, there is public school open enrollment,
http://education.state.mn.us/MDE/StuSuc/EnrollChoice/index.h...
and the school district for the neighborhood in which I live includes open-enrolled students from the territories of more than forty other Minnesota school districts, with funding following the students on a per-capita basis.
http://www.house.leg.state.mn.us/hrd/pubs/mnschfin.pdf
Aspects of the system in Minnesota that are rigid and suboptimal as they are in many other states include
lock-step union seniority pay and promotion systems for nearly all public school teachers,
http://minnesota.cbslocal.com/2012/05/03/seniority-still-rul...
but parents with power to shop and funding that follows students allows school districts with the better programs, overall, to thrive and produce innovative new programs (e.g., language immersion programs, specialized fine arts programs, and school-within-a-school programs for highly gifted learners) and the schools that are forced to lay off teachers (alas, by seniority rather than by effectiveness) are the schools with laggard overall programs.
the evil socialist Canadian public school system
This is from your comment. I'm not aware what province you grew up in, but it's my understanding that there have been elements of school choice in some Canadian provinces in our liftime,
http://www.heritage.org/research/reports/2010/11/school-choi...
http://www.societyforqualityeducation.org/parents/canada-cho...
although just what is going on in Canada in comparison to other countries receives different degrees of emphasis even in Canadian sources
http://educhatter.wordpress.com/2012/02/19/school-choice-and...
and perhaps right now when Canada is in the world news
http://www.startribune.com/world/151854375.html
we might conclude that there are some difficulties with the system in some Canadian provinces.
Anyway, Canada provides an example, through its system of provincially administered health insurance programs,
http://www.hc-sc.gc.ca/hcs-sss/medi-assur/index-eng.php
of a general public subsidy program that still allows much user choice. Patients get to choose their doctors in Canada.
http://www.cwhn.ca/node/40789
In principle, as already observed in reality, it is perfectly possible for there to be a general public subsidy for some service that is deemed to have a positive externality, while still allowing user choice of the provider of the service.
There are other international examples of school choice. I particularly like the example of the Netherlands with its very wide array of choices for parents, all at equal publicly subsidized expense,
http://www.denhaag.nl/en/residents/to/Want-to-send-your-chil...
which has been studied for years as part of broader studies of parental choice in schooling.
http://www.eric.ed.gov/ERICWebPortal/search/detailmini.jsp?_...
http://books.google.com/books/about/Choice_of_schools_in_six...
By the OECD testing program called PISA, the Netherlands does as well as or outperforms other countries both as to helping students from disadvantaged families
http://www.oecd.org/dataoecd/5/5/49603617.pdf
and as to getting educational achievement results per unit of money spent by the school system.
http://www.pisa.oecd.org/dataoecd/50/9/49685503.pdf
I'm not opposed in principle to the idea of statistically ranking teachers and giving the power of choice to parents, but you guys have royally fucked it up in practice.
I'll heartily agree with you and with other comments here that the current state testing systems, which vary state by state, but which are mostly poorly designed, are a weak basis for publishing ratings of schoolteachers, but as you correctly point out, that's not to prove that a better system of assessing students and their academic progress has NO value in helping parents shop for schools. What I don't want to go back to is the day in which no one had any idea how well any teacher was teaching, because no one was looking, and no one was looking because parents couldn't shop for schools anyhow. If parents can make global evaluations of what is good for each of their children, better incentives exist to improve teaching, improve school administration, and improve all other aspects of the school experience.
Returning to the other comment's thoughtful question,
If you give the parents the ability to "shop", what happens when everyone thinks that teacher X is the best English teacher for 7th grade and they ALL want their kids in her class?
that is precisely the kind of situation that builds curiosity among other teachers about "What is that teacher doing that I'm not doing?" and among administrators about, "What value do families perceive in that class that they don't perceive in our classes?"
As other comments have already pointed out, parents in all countries of the world shop for schools at least by how they choose their residence addresses. But decoupling school choices from residence choices allows schools and families to respond more efficiently to their own mix of trade-offs. When many shoppers prefer one grocery store to another, and take their business to the better grocery store, what usually happens is that the worse grocery store changes the way it does business and improves its overall customer value proposition for shoppers. As I noted above, one HUGE problem with schools all over the United States, even schools in states that do not formally have "union shops" with mandatory schoolteacher union membership, is that administrators have little flexibility in reassigning teachers to the work that they do the best. So today is a somewhat slow process at the margins for schools to improve (by realigning staff assignments) as families shop for the best classrooms. But I've seen what Minnesota school districts have been able to do even within the limitations of the current system, and I'm confident that adding incentive for school improvement by giving learners more power to shop makes as much sense (as a matter of basic public policy) as it does for providing most other services.
AFTER EDIT: Commentary on the link submitted here by a blogger based in New York City,
http://nycrubberroomreporter.blogspot.com/2012/05/carolyn-ab...
which I learned about from a Facebook friend who lives in New York State and has been following the controversy on school testing in New York State closely.
While I agree that it's introduced some problems in states where I've lived (Utah, for example), it's also allowed parents to move their children around a bit more easily, and there are some benefits to that.
On the flip side, I'm now in Georgia. We live on the boundary between a poor school district (98% "minority" district-wide) and a rich one (10-50% "minority" depending on the school). I can only "shop" for schools in my poorer district, which are typically around the bottom quintile of schools in the state. The kicker is, because of the geographic boundaries, the bus stop for the "richer" district stops on either side of our subdivision, and our kids pass withing 1 mile of five different elementary schools in the "rich" district on their way to their assigned "poor" one. Shop for schools? I'd love to. I'd even like for my kids to go to a school close to their home. But a long history of bussing (which in our area has reinforced segregation rather than combat it) and civil rights policies perversely means it's more difficult to move to the better schools.
There's no way to avoid it, even in principle. What if you move?
In the US, home prices are elevated in certain areas because people who live at that address get to go to a good school. So there's still school shopping, when it comes to deciding where to live (it's often a key factor in that decision).
I think that's how it is most places, actually. At least where I grew up in California, I was assigned to a certain high school based on where I live. We could have applied for a transfer, but those aren't often granted. This is all for the public school system, of course. There are private schools as well, which operate differently, but I think for the most part people go to public schools.
These value-added statistics, which on a per-teacher basis must be incredibly noisy (they measure a difference, the "value added", of two already-noisy performance numbers) resulted in the suicide of a low-ranked teacher:
http://www.msnbc.msn.com/id/39404037/ns/us_news-life/t/la-te...
I've been surprised to hear the credence placed in these statistics when I listen to presentations by principals and superintendents.
The evaluations of the same teachers teaching the same subjects in two different classes are almost completely uncorrelated. The distribution of the scores on the scatterplots look almost random.
It's a travesty that perfectly fine teachers are being publicly shamed based on these unreliable metrics.
See my previous post on the topic:
http://news.ycombinator.com/item?id=3749530
I don't believe that 'the professionals' setting up, promoting and administering a system like this can possibly be ignorant of this fact and/or well meaning.
Of course Abbott had to be eliminated as a teacher. She was interfering with the program of deliberate dumbing down.
I bought a K'NEX roller coaster kit and helped him put it together. At first, he didn't grok the pseudo 3D instructions or the way the pieces fit together. However, after an hour or so, he was rotating our partially assembled segments in his head to compare them to the illustrations. But, this doesn't count as rigorous seat work. Oh well.
Our school actually does an excellent job of avoiding the binary incentives of No-Child-Left-Behind and attempts to provide somewhat individualized instruction to students. Ohio has yet to adopt a value-added measurement scheme, and I wonder how our district would fare, especially if achievement was normalized for socio-economic status (the average household income is over $100K, a high number for Ohio).
We hope that the Montessori school's motivated teachers can use their freedom from mandated curriculum requirements, standardized testing, etc. to facilitate our son's exploration of what interests him. I don't care if his education is seen as incomplete, unbalanced, or otherwise non-standard so long as he grows to become a curious, thoughtful, and creative person. When has someone with these qualities ever failed because he had some perceived weakness in his childhood education?
Define "failed". Failed to get a job, too right. Failed to fit in to society, check. Failed to be happy, seems to go with the territory(!?).
Will you excuse a short aside:I'm interested in the diagnosis of a 6yo as having ADHD - what procedures and tests were performed to come to that decision. In my, albeit limited, experience of primary aged (4-11yo) boys if they're not running around half the time like some sort of sugar rushing lunatic then they're the exception to the rule. Can he sit and watch a TV show for 20 minutes without leaving to do something else?
It is not generally a problem of too much energy and lack of focus. Rather it is a problem of inappropriate focus, frequently very intense. For instance my nephew would get so interested in whatever he was doing that he would fail to realize that he needed to pee, and then after he peed his pants he would get very upset that people were making him change his pants when he wanted to do something else.
It becomes a problem in school when children are unable to follow repeated directions because they are unaware that directions have been given, because their interest has been caught by something - anything - else.
Disclaimer: I am the parent of a 7 year old who has been identified as likely having ADHD, though I have not yet done the official screening.
ADHD causes __attention_regulation__ difficulties. Many mistakenly see it as a simple lack-of-attention, but fail to realize that hyper-focus is also a common symptom.
I suggest reading the first few chapters of ``More Attention, Less Deficit'' to gain a basic understanding of ADHD.
We need to seriously rethink education and how we are testing students and teachers.
Yes, the politicians are clearly in the pockets of test makers. After all, the test makers are the #5 and #10 biggest politicals donors ever.
http://www.opensecrets.org/orgs/list.php?order=A
Oops, my mistake - it's actually teachers (generally opposed to accountability) who are the big political donors, not the test makers.
There is no way around this fact.