This is a followup to last week’s post on “The Advantage of Abstract Examples in Learning Math” by Kaminski et. al. I promise that next week I’ll write about some research that actually has something worthwhile to say. But I just wanted a complete dismemberment of this article to be available to all as soon as possible.
I gave you the back story last week. Here’s my quick summary:
The study authors claim to find that introducing math concepts through abstract representations did a better job than introducing them in a concrete way in causing students to be able to generalize the concepts to a new situation. However:
* The study authors confuse the ability to produce correct calculations with the understanding of the concepts taught; and more importantly,
* The study authors used a lesson that was poorly adapted to concrete situations.
I conclude that their results don’t have the scope they claim. At most, they show that throwing in distracting information can make math harder to learn, but we already knew that. A better conclusion is that if you’re going to use concrete examples to teach something, be real about it. Don’t present a concrete setting and then dive straight into formal properties; use the setting to bring out the properties by having students think naturally about the setting.
The details:
First of all, the study authors declare in the online supporting material that “in the present research, study participants learned the concept of a commutative mathematical group of order three.” If that doesn’t mean anything to you, don’t worry, I’ll explain it very clearly in a moment. But this sentence already suggests an important flaw in the study design that I actually think is a flaw in a lot of curriculum design, so I’ll highlight it:
“The commutative group of order 3” is a math object you learn about in a first course on abstract algebra. The “training” the authors used to introduce the idea to the students basically describe the group’s formal definition. (I’ll show you exactly what they do below.) But the students’ learning was assessed by a test that measured the ability to do computations inside the group. The thing taught and the thing tested were not the same at all. I say this is a flaw in a lot of curriculum design: we want students to have a deep understanding of mathematical objects, like fractions. But they’re often just tested on their ability to produce correct computational answers. Knowing how to add 1/2 and 1/3 is not the same as knowing what 1/2 and 1/3 are. Lots of kids never really learn the meaning of the objects (or even register that this is the important part about learning math) because they know they are only going to be accountable for the computational technique. The irony is that in the end, this makes the computational technique much harder to learn. But we set them up for this as long as the only things we ever test them on are the computations.
Now the bigger problem with the study is how the “training” used was inappropriate to concrete situations. But before I can show you this you have to understand something about the concepts and skills supposedly being taught. So, that explanation I promised (skip the next section if you already know this stuff). I am actually going to use one of the “concrete” situations from the study! But I am writing the “training” a little differently.
You know how tennis balls come 3 to a container? Suppose you work in a tennis ball factory packing balls. Every time you get 3, you can pack them away into a container and send it down the assembly line.
So, suppose in front of you is 1 tennis ball, and then along the conveyer belt come 2 more. You pack them away, send them along, and then you have – none!
How about if you have 0 balls, and 1 comes along? Then of course you can’t do anything because you don’t have enough for a container. So you’re stuck with 1.
Or if you have 1 ball, and 1 comes along? Again, you don’t have enough for a container, so you’ve now got 2. But if 1 more comes along, it and the 2 make 3, you can pack them up and send them along, and so now you’ve got 0 again.
Or – this is “the hard one” – you have 2 balls, and 2 come along. You pack up a container, send it along, and you’re left with 1 ball.
I think you’ve probably got the gist by now. This is an alternative arithmetic in which the numbers never go above 2 because every time you get 3 balls you just pack them up and send them along. So, 1+1=2 like normal, but 1+2 = 0 and 2+2 = 1.
Okay, that’s it. You can now do arithmetic in the “commutative group of order 3.” You can probably out-perform all of the study subjects in Kaminski et al.’s study on the tests they took at the end.
Compare this to the training set in the exact same “concrete situation” from the study (this is found in the online supplemental materials):
“A tennis ball manufacturing company is having trouble with their ball-making machine. Instead of producing batches of three balls to fill a container, it is producing batches of zero, one or two balls represented as {no balls}, {1 ball}, and {2 balls}. Consequently two or more batches need to be produced to fill a container. In doing so, the number of extra balls produced needs to be determined.
“Rules for finding the number of extra tennis balls:
“Rule 1. The order of the batches doesn’t matter. The number of extra balls will be the same. For example, if this batch {no balls} is made first and then this {1 ball}, then this much {1 ball} is extra. The same thing happens if this batch {1 ball} is made first and then this {no balls}. We will have this much {1 ball} extra.
“Rule 2. If this batch {no balls} is made with any other single batch, the other amount is always extra. Here are a couple of examples: If this {no balls} and this {1 ball} are made, then this {1 ball} is extra. If this {no balls} and this {no balls} are made, then {no balls} is extra.
“Rule 3. If {1 ball} and {2 balls} are produced, then one container can be filled and {no balls} is extra.
“Rule 4. If {1 ball} and {1 ball} are produced, then we cannot fill a container. So, {2 balls} is extra.
“Rule 5. If {2 balls} and {2 balls} are made, then one container can be filled and {1 ball} is extra.
“Rule 6. If more than two batches are produced, the order in which they are made doesn’t matter. The extra will be the same. For example, if {1 ball} and {no balls} and then {2 balls} are made, then {no balls} is extra. The same amount is extra if {no balls} and {2 balls} and then {1 ball}.”
Now if you actually have an abstract algebra background, you see what they’re going for here. Rule 1 is saying the group operation is commutative, rule 2 is saying that {no balls} is an identity, rule 6 is sort of a bastardization of the associative property, rule 3 declares that {1 ball} and {2 balls} are inverses, and the other two rules are examples of arithmetic in the group.
But my point is what they’re not going for. They’re not making use of the concrete situation to help the reader understand what’s going on. In fact, the very first “rule” leaves the reader with the sense that we might as well stop imagining tennis balls right now. If we were really talking about tennis balls, I wouldn’t have to tell you that the order in which they come doesn’t affect how many you have. Everybody knows this. The excessive formalism divorces the story from any concrete reality.
Rule 2 makes the divorce deeper. If we were really talking about tennis balls, I wouldn’t call {no balls} a “batch.” So by the time you come to rule 3 (when finally, if barely, the concrete situation is referred to in a sensible way – “one container can be filled…”) you have already completely given up on the idea that what they are talking about has anything to do with reality as you know it.
So here’s the lesson: if you’re thinking about using a concrete example to introduce a math idea, don’t worry. The results of the study don’t really mean “it’s better to leave the apples etc. in the real world” as the NYT writeup asserted. The safe conclusion is this: don’t be phony about it. If you are going to use taxi meters to develop linear functions, use the kids’ knowledge of how that situation actually works to bring out the math ideas. If you’re using temperature to explain negative numbers, use it with examples that actually make sense when you think about temperature. (For example, you can use temperature to explain why 5 – 7 is -2 but not why 5 – -7 = 12.) Being excessively formal about something that’s supposedly concrete reality makes the kids stop listening to their own logical reasoning and common sense and start just trying to guess what you’re up to.
Research in Practice 
Thanks, Ben, for a lovely dissection of their research. I discussed this research with my colleagues shortly after it came out, and noticed some of the same things you did, though I didn’t write it up so elegantly. I’ll post what I wrote at my blog.
If you look up those names, you’ll see that they’ve been doing minor variations on this same research over and over. Seems like a waste of money to me. Here’s one link from 2005: http://researchnews.osu.edu/archive/absconc.htm
Yay, Ben blogging!
A pedagogical move that like whenever a kid is answering a question (mine or someone else’s) aloud in a classroom setting is to ask, “Raise you hand if you agree with Susie. Raise your hand if you disagree with her. Raise your hand if you’re not sure.” Then I call on people and we hear arguments from both sides. Some variation on that procedure happens regularly in my classes, regardless of whether the original answer is right or wrong. I like it because I’m not giving away anything, not claiming authority, setting up consensus as the answer key, fostering a sense that it’s okay to disagree, to make mistakes, and to not be sure, and finally that I care about making compelling arguments–that it’s their thinking that matters. It’s also helpful in assessing where the class is–how confident are they in their answers? is the class divided? confused? did the original answerer just make a careless error that their classmates can easily set straight? do kids vociferously argue against what is actually the correct answer?
It’s a technique that I like that I’m still learning how best to use. Anyone else do something like this?
This is a great move. Actually, I kind of think it’s the move. Or something like it. They need to be on the hook for their reasoning all the time, and conversely to know that they have the right to their own reasoning.
I do something similar; sometimes I precede the “raise your hand if you agree” with “raise your hand if you feel like you understand the point so-and-so is trying to make.” The idea is to distinguish understanding from agreement, since “do you understand?” is so often used in math class to mean “do you buy it?”
The drawback of “raise your hand if you feel like you understand” is that to some kids it signals that I the teacher think the point being made is correct. I try to avoid this by using the question whether I understand the point myself or not. I think I sometimes tip my hand with tone of voice though.
I think that what the study shows is that it’s possible to come up with really bad “concrete” examples, and that bad examples are worse than good examples. Possibly, bad examples are worse than none at all. I’m surprised they didn’t use the clock-arithmetic way of looking at Z3 for their concrete example, since that’s the most common of the concrete examples of the group, but maybe they didn’t want to use that in the study because it might actually be helpful.