Last week I promised I’d write about something worthwhile this time, so as promised: one of the most worthwhile things I’ve ever read. This piece is not typically seen as “math education research,” but I think it’s of vital importance for us.
You don’t have to read the whole original book by Oskar Pfungst; there are plenty of good summaries of the research online (for instance here and here). This amazing story gets talked about in comparative psychology but I think it has special significance – often missed – for math educators.
Take-home lesson: never underestimate your ability to fool yourself into believing your students understand something when really what they are doing is watching you. To force them to engage the material it is often necessary to restrict their access to you or systematically confound the signals they get from you.
I think this is a central issue for modern math teachers. We need to explicitly develop ways of question-posing and interacting with our classes and individual students that hide or disguise our intentions for how they are supposed to respond. This needs to be part of the core training of math teachers, much more than it already is.
Clever Hans was a horse owned by Wilhelm von Osten, a high school teacher in Germany around the turn of the twentieth century. von Osten attempted to teach several animals basic arithmetic and German. The other animals failed but Hans, a Russian stallion, seemed to get it. He indicated answers by tapping his front right hoof. He was able to produce correct answers to a shocking array of questions including the four basic operations, some square roots, calculating days of the month, spelling words, etc. He was regularly displayed by von Osten in public. His feats caused an international stir. Nobody who hadn’t seen it could believe it (von Osten must be cueing the horse somehow, right?), but skeptics were converted when they saw and interacted with the horse themselves. A commission convened by the German board of education found in 1904 that no trickery was involved. von Osten didn’t even have to be present.
Psychologist Oskar Pfungst solved the puzzle: no trickery was involved, to be sure. But if the questioner didn’t know the answer to the question, then Hans couldn’t answer it. (For example, if Pfungst whispered one number to it, and von Osten another, and then asked for their sum, Hans would answer incorrectly.) What was going on was that whoever asked the question was cueing the horse totally subconsciously by tiny imperceptible body movements when the answer was reached. The horse took these movements as an indication to stop tapping that hoof.
Once Pfungst became aware of these movements, and trained himself to control them in himself, he was able to cause the horse to answer a question with any answer he desired. For example, he could say “tap 14” but then cause the horse to tap 8 by using the body language he had found. Later, in laboratory experiments, he assumed the role of the horse, and “read the mind” of his subjects. He would instruct them to think of a number, and then begin tapping his hand. If he saw the characteristic body language indicating the “right answer” had been arrived at, he would stop tapping. The subjects were amazed by his ability to guess their number and had no idea they were sending tiny signals.
In comparative psychology they’ve taken the lesson of this story, and they’re diligent about preventing the “Clever Hans effect” from interfering with their experiments. In math education we still need to. If a horse can have all of Germany and beyond believing that it can extract square roots, when what it’s really doing is taking subconscious cues from its trainer, think of what human children can do.
This is not idle speculation, but a very real dynamic that shows up at least occasionally, in one form or another, in almost every math class I have ever watched or taught. It can happen at two levels:
1) Some students are committed Clever Hanses.
2) The whole class becomes Clever Hans.
To some students, getting cues from the teacher rather than thinking about the math has become such an ingrained habit that it is their entire modus operandi when doing math with an expert around (teacher, tutor, parent, peer, etc.). They usually have lost faith in their ability to have math actually make sense to them, and this cueing has become their sole survival strategy. They regularly fail tests and have come to accept this, but they would rather fake you out than admit ignorance – understandable, since they often carry a feeling of stupidity arising from their lost faith, and admitting ignorance would mean exposing this supposed stupidity. So they watch you and produce the right answers more often than not. Not with any sense of trickery or getting over on anyone – often, they’re barely conscious of the game. But it’s how they survive.
Most of us have had students like this in our class whether or not we recognized it. I’ve worked with several such students as a private tutor and this context puts their habit into very stark light. The first day I meet such a student, the very first problem I have them work on, they’ll make a guess quickly and watch my reaction to see if they’re on the right track. I’ve trained myself to respond noncommittally to everything:
“Is it 1/3?”
“Why do you say so?” (Even if 1/3 is correct.)
“Oh, no no, 1/4?”
“Why do you say so?”
I also often hide my face when posing a problem, or (even better) move out of my student’s visual field. These are not the only moves in this situation, but something similarly radical is needed: such a student will never engage the math itself unless all recourse to his or her standby survival method has been methodically denied. I believe that a necessary part of the repertoire of every math teacher is a set of moves designed to hide or confound the cues we send. The above are just a few suggestions.
In Pfungst’s research, when Hans was fitted with blinders and posed a question by a questioner outside his visual field, he made strenuous efforts to see the questioner. Similarly, when we begin to work on not telegraphing the answer we are hoping for – developing a good poker face, for example, or responding with “what makes you say so?” whether the proffered answer is right or wrong – students who are committed Clever Hanses will make every effort to get the answer out of us anyway. They will ask questions, make guesses, keep the game going till we let something slip. One of the lessons of the story is that it’s virtually impossible not to let something slip. Pfungst found that people communicated answers with minute physical signals they weren’t aware of. Even people who were made aware had trouble not sending them. So when working with a committed Clever Hans you can’t just trust yourself not to give up the goods. It’s often necessary to physically vacate the space between the student and the problem. When working with an individual kid this can even mean leaving the room.
Now even kids who are not in the situation described above, who have not come to rely solely on observation of the expert, can be drawn into a Clever Hans game if the teacher is telegraphing the answers too intensely. In most classrooms, for example, the question “Is this enough information to solve the problem?” does not receive a straight answer. Since this question is not usually asked when there is enough information, it is generally safe to conclude that the answer is “no.” Furthermore, this conclusion is a lot less work than actually thinking about the problem. The teacher obviously wants to hear “no” so let’s say it and be done with it.
Similarly, any yes-or-no question posed by a teacher to a class is a setup for a Clever Hans game. It is a lot less work to guess one answer blindly (or wait for someone else to do so) than to actually think about the question. The teacher’s response to that one blind guess gives away the game and the class moves on with the question “answered,” but the very real possibility that no one in the room actually thought about it.
With this in mind I believe it’s a very productive exercise to scrutinize our lessons (working with the whole class, a group, or an individual) with the question “how far was it possible to get by just following my lead?” It’s especially powerful to videotape class and then watch the videotape with this question in mind – you see a lot more that way. It’s also worthwhile to observe other teachers with this same question in mind, because the dynamic is much easier to recognize from the outside than in the heat of it.
I have one other suggestion related to all this: it helps to be legitimately open to students’ thought processes whether or not they initially sound like what we had in mind. This is something I’ve had to work on. I have throughout my career been repeatedly surprised by the discovery that nearly every time a student offers an idea authentically (i.e. not as just a random guess), it makes some sort of sense. Maybe not complete sense, and maybe it’s not at all where I was headed. But if I can curb my initial reaction of “this kid is totally confused” long enough to actually take in the train of thought, there is almost uniformly some worthwhile reasoning inside it. Then even if I need to say “we’re going to stick to the topic,” I can do so after acknowledging the reasoning. The connection to Clever Hans is that if we want them actually thinking, we have to make sure our questions are legit. This gets communicated by acknowledging people for treating them as legit. If the only answers we acknowledge are ones that fit our preexisting image for what the answer is supposed to be, this communicates that the question wasn’t authentic, and it’s probably easier to try to guess what the teacher is up to than to engage it authentically.
To summarize: the lesson of Clever Hans is of central importance for our profession. We want our students thinking about math, not watching us for cues. But it is natural to subconsciously cue them as to what we want to hear. So a necessary part of becoming a math teacher is developing techniques to deny access to or confound these cues. Vacating the students’ visual field while they work is one important method. Another is responding the same way (e.g. “why do you say so?”) whether we believe the student is on the right track or not. But all of us need to be thinking about this.
36 thoughts on “Required Reading for Math Teachers I”
I was at my office yesterday, and heard an exchange between a math teacher and the student he was helping. “What’s 0 to the 0?” “Not defined? … 1? … 0?” I missed a moment but then I heard the student saying that the teacher had looked dubious so she knew she was wrong. Because, as we all know, the look on the teacher’s face is more important than any knowledge in the student’s head!
Wow, this has really opened my mind to the siginificance of this effect. I was aware that I often telegraph the answer, and I sometimes do it deliberately (especially in “is this enough information” or “is this the only answer” type questions). But I hadn’t thought about the idea that students might use this in place of actually understanding. I’ll have to be much more careful from now on.
I had completely forgotten the “Clever Hans” story, but it makes so much sense. I have worked hard at my ‘poker face’ in class, and design the work my students do so that they are forced to think on a daily basis. Thanks for this great article!
I wonder what effect such teacher’s cues have on the good students as well as on the poor ones. I suspect the cuing phenomenon tends to amplify the good students’ competence. That would be unfortunate.
Though it’s difficult to remember classroom experiences fifty years later, I do recall noticing that some SAT questions contained hidden cues suggesting which answer-choice was correct. It seemed to me that sometimes the question-writers were unconsciously (surreptitiously?) giving away the answers. This mainly happened in the language-related questions, but occasionally in the math questions as well. Sometimes I had the disturbing feeling that the question-writers were trying to slip the answer to a certain “in-group” of students who were able to perceived those cues.
My idea that question-writers were giving away some answers sounds ridiculous, I know, but that was my clear impression at the time.
I think it depends on what kind of “good student.” I believe the most long-term successful math students are precisely the ones who approach math and math class with the question “what are these mathematical objects? How do they work? How do they interact with each other and with reality?” (Rather than, “how do I get the answer?” or “what is the answer?” or “do I look smart right now?” or anything besides “what is this and how does it work?”) A student with this mind-set is the one you least have to worry about regarding the Clever Hans issue because his or her attention is already turned toward what you want it turned toward – the math – and away from your body language and tone of voice. However, if a student is very concerned with looking smart, then even if she or he is “good at math” there is something to worry about, because for such a student the fear of being wrong is a major motivator. Then taking cues from you (rather than or in addition to thinking) becomes a practical way to avoid being wrong. I think it really helps to regularly praise thoughts that are not correct but nonetheless move the conversation forward, and to praise risk-taking, because it makes being wrong less scary for such students.
When I first started teaching it was to college students as I was an undergraduate TA. I was always on guard for students trying to get me to answer questions I’d posed to them through whatever means … mainly because I would do that to my professors.
So I developed the point of replying, “Possibly,” to all questions that are some version of, “Am I right?” I also learned to always intone it in a positive way or else the students would be able to tell if it really meant yes or no.
Just last year, one of my students told me that my eyebrows would shoot up if I heard a correct answer, so I started shooting up my eyebrows for absolutely no reason.
My advice is to ask the students what you are doing. When a student states and then retracts an answer, ask them, “What clue did I give you that this was the wrong answer?” Stating the question this way keeps them from being able to fudge and claim that they knew their original answer was false. It forces them to tell you your tells.
Once you are aware of your tells, then you regain your power as an educator.
This is a great suggestion! This reminds me that it’s also a good move with students who are committed Clever Hanses to (lovingly) call them on it, especially in one-on-one situations. They give it up faster if they are tuned into the dynamic and the impact it’s having on their learning.
I drive my students crazy by controlling my tone and body language. It was a goal of mine from day 1. I agree that it is very important in class to make sure kids know what they are supposed to. I find it key to ask “Are you sure?” with a very skeptical tone even when they are right to make sure they are confident. Another favorite of mine is “So, then we’re done the problem” when there are more steps.
Nice story to illustrate this!
Thank you for this article. You are right. It is so important not to cue them one way or another. I often ask the question “Are you sure?” even when the answer is right to get them to think critically. I also catch myself talking too much and need to take a step back and let the students work problems out independently or in groups. You are right about the questions being key. Many of the resources we use do not help students develop a deeper understanding of the topic. I have been using a few resources lately that have been effective. One is “Good Questions for Math Teaching: Why Ask Them and What to Ask, 5-8”. This book really helps you to ask thought provoking questions. It is written by Anderson and Schuster and is published by Math Solutions. We have had some wonderful class discussions about the “why” in math as opposed to just “how” to solve it. Thanks again!
Thanks for the book recommendation!
One of my favorite questions (and my students hate it) is “are you sure?” – because in my class it comes up about twice as often when they are right than when they are wrong.
I hate Hans.
In the last two weeks I argued in one class that odd numbers have an odd number of factors (9 has three: 1, 3 and 9. 15 has four, but I argued that 1 shouldn’t be counted…) In another I insisted that there are 16 ways to triangulate a hexagon (there’s only 14), and they must be missing some.
Trust the math, not the teacher.
And I do hate Hans. But when they argue back, and I am losing, I sometimes smirk or giggle, and they know they are right.
Love it! I’ve often threatened to lie to classes, but I’ve never really followed through. I feign ignorance a lot, hide myself a lot, and do a lot of “are you sure?” typed things, but I’ve never had the guts to just straight up assert something false. But your example may inspire me to do it. Perhaps this week…
As a new educator (finishing my certification in may) I’ve been using the “are you sure?” “tell me why?” “what makes you think that?” responses… but how far do you go with this… how do you accept a correct answer? And what do you say/do when a student is SURE of a wrong answer?
Welcome to the profession! This is a wonderful and important question. I’ll give you my two cents but I’m hoping that others who’ve described strategically dealing with the Hans issue will weigh in as well (Justin Lanier? jd2718? J? Mr. Sweeney? Tim?)
I think the most powerful way to accept a right answer is to create a classroom structure or norm that allows the whole class to judge an idea correct. Following the amazing Jason Cushner (an article about him here), I use the word “consensus” a lot. I’ll write something on the board, pose a question to the class, and say “figure this out and let me know when you’ve reached consensus.” Then I’ll vacate the space between the students and the board; move to a back corner of the room and watch. If students turn to me to say something, I direct them to each other. Especially early in the year, I need to do a lot of work to make sure the students stay engaged with each other in this situation. If they do, magic happens. If they don’t, then either there can be a lot of staring at each other waiting for something to happen, or there can be a few kids getting involved in the problem and not being that concerned if others are engaged with them, and everybody else tuned out. The best results I’ve had with this method have involved:
a) Telling them up front what I want to see. (When you have something to say, you have to be invested in other people understanding it; ask each other individual in the class “do you understand?” Then ask, “do you agree?” When you are listening to someone else, you have to be invested in understanding what they say. If you are confused, say “I don’t get that, could you explain it again?” If you disagree, say “I disagree, because…”)
b) Being absolutely consistent in insisting that they stay engaged with each other and not move on or try to engage me till consensus is reached.
Anyway, if all this is accomplished, then I don’t need to do anything to acknowledge the right answer. The class will already have reached consensus and then the only thing I accept is their consensus. (“We think it’s this.” “You have consensus on that?” “Yes.” “Okay.”) Almost always, their consensus will be correct. To quote the aforementioned Jason Cushner – “I’ve never seen a class that’s actually engaged with the problem come to an incorrect consensus. I have seen an incorrect ‘consensus’ form when people are not really engaged with the problem and just want to move on.” At least 3 or 4 kids with attention on a problem is usually enough to catch any mistake if the problem is at an appropriate level of challenge.
Now what if the class is totally engaged with the problem, and comes to a wrong conclusion as a whole anyway? Okay well honestly usually I’ll say, “you’re sure?” “Yes.” “Well I should tell you I disagree. See if you can figure out why.” I think this is an acceptable but less-than-ideal move. I use the language “I disagree” to highlight that they have as much of a right to reason about the problem as I do; but I’m still communicating that they’re wrong, and if I do this too many times, there’s a danger they’ll get lazy and not press themselves to really satisfy themselves about the problem, since they know I’ll be there at the back end to bail them out if they blow it.
It takes a lot of guts, and a certain amount of flexibility with the curriculum pacing, but a really powerful thing to do if the class is honestly engaged with a problem, and comes to a wrong conclusion, and reaches consensus on the wrong conclusion, is to accept the consensus! And then design the next problem they receive to highlight the flaw in their previous thinking, and see if anyone notices. Here’s an example of how this could work. Sort of an abstruse one, sorry about that, but it’s on my mind because it’s based on a class I taught recently.
The class was for middle and high school teachers. I was working with them on proving the “fundamental theorem of arithmetic” that a natural number factors into primes in only one way. I had introduced them to the ring generated by the integers and the square root of -5 as an example of an arithmetic in which prime factorizations are not unique. (For example, 6 factors as 2*3 or as (1 + root -5)*(1 – root -5) and both factorizations are into numbers that don’t factor any further.) Then I let them stew for a long time trying to figure out how we could establish that they are unique when we’re in the normal world of natural numbers. After a lot of casting about and coming up short, one teacher described a deterministic algorithm for finding the prime factorization of a number (start with 2, determine if it’s divisible by 2; if it is, divide by 2 and begin again; if not, move on to 3…) and argued that because this algorithm will always produce the same factorization, there can only be one. This argument is incorrect. The algorithm only picks up one factorization, but this doesn’t mean there aren’t others. For example, a similar algorithm can be designed for the ring generated by the integers and root -5, and it will always produce the factorization 2*3 for 6, but (as above) this is not the only factorization of 6 in this ring.
I don’t think my response was very good. I argued the point myself. Now this particular participant was very confident, so he was not at all inclined to take my word for it; the opposite of Clever Hans. And he got plenty out of the exchange. But it wasn’t particularly good for everyone else. Retrospectively, I should have turned the question to the rest of the group to judge. Now it’s possible someone would have seen the flaw in his reasoning, and that would have been a powerful and productive conversation, much more so than it was coming from me. But actually I think it’s pretty likely he would have had all the rest of the participants convinced that his algorithm was a proof of unique factorization of natural numbers. They were worn out on the problem and desperate to make progress, so not inclined to be critical of a promising idea. I could have said “okay.” Then, next time, I would say “last time, you guys agreed that unique factorization holds because of this algorithm.” Then, I would describe the analogous algorithm for the ring in which unique factorization does not hold, and present them with the problem of sorting out why the algorithm leads to unique factorization in one case and not the other. With some disappointment they’d eventually be forced to conclude that the algorithm didn’t really lead to unique factorization after all; it must be happening for another reason.
Now if I’d taken this path, it would have pushed back the goal of actually proving unique factorization. It would also have disappointed everybody since they’d discover they hadn’t really solved a problem they thought they’d solved. This is probably why I’ve rarely had the guts to actually do something like this, although as I talk about it now I’m feeling more and more sure I should do it almost every time it comes up.
Because a big (arguably the big) goal of math education is cultivating people’s reasoning, right? And what more profound experience for cultivating a sense of rigor than to uncover a flaw in an argument subtle enough to escape an entire class for an entire period? I hope to write about this in a post soon, but I’ve recently (thanks to Paul Lockhart) become convinced that this kind of experience, where you’re totally convinced of something and then you become convinced it has to be wrong, is at the heart of learning how to prove things. Because without the combination of self-trust and self-doubt forced on you by this kind of experience, what’s the point of rigor?
All of the above is just one way of addressing your question. Another, also awesome way is described by Justin Lanier in a comment here. I’ve occasionally used something like this too. A kid says something. Once you have them explain/justify their point, you say “raise your hand if you agree with so-and-so.” Then, “raise your hand if you disagree.” Then, “raise your hand if you’re not sure.” Then have kids representing each position say why they think so. Then you can decide what to do next, including taking another poll, having them debate the point, introducing some thoughts yourself, or tabling the question and returning to it later. I think a nice thing about taking a second poll and then tabling the question for later is that it creates a natural punctuation of the end of the conversation, allowing you to move on, without committing you to a position on the point. When you come back to it, the answer might be obvious to everybody.
Sometimes you say “very good” or “that’s correct” – perfectly reasonable. Not every question needs to lead to a discussion – use your judgment. Ask the probing question when you want them to dig. But the class needs to move forward at some point – your call.
Rhythm takes work.
“Think pair share” is another useful strategy to deal with this issue. Students have some think time then discuss their ideas with a partner. Lots of these discussions are going on at once so the students can’t get signals from the teacher about the validity of the ideas being expressed.
Think-pair-share is a wonderful classroom structure. It’s a very powerful thing to put students in the position of having to evaluate their classmates’ thinking in addition to their own. All math students should have lots of opportunities to judge for themselves whether the thinking or work of their peers makes sense.
It doesn’t let you off the hook on the Hans question, though, if you do any interacting with any of the pairs. If anyone in one of the pairs asks for your help, all the same issues come into play if you respond.
Well, I enjoyed your post, where I arrived from Dan Meyer’s blog.
I’m a K-12 (I think that you will say it this way) Spanish Math teacher and I’m going to use your reflections in my blog too.
I want to add a new reflection:
It’s not only when we pose questions, it’s also the whole structure of some classes and curricula: all our students do in exams is exactly what we are expecting. Some examples:
Many students don’t understand what an integral is, but they know what the answer to Int (x^n) is.
Sorry for my bad English. I’ve tried my best 😉
Yes – I guess we need to have the courage as teachers to keep our mouths shut and let youngsters discuss a question for themselves. When it comes to how you address the final consensus, you have described approaches to that very clearly above 🙂
Reblogged this on Math Education Concepts and commented:
Reblogged from “Research in Practice”