This is a followup to last week’s post on “The Advantage of Abstract Examples in Learning Math” by Kaminski et. al. I promise that next week I’ll write about some research that actually has something worthwhile to say. But I just wanted a complete dismemberment of this article to be available to all as soon as possible.

I gave you the back story last week. Here’s my quick summary:

The study authors claim to find that introducing math concepts through abstract representations did a better job than introducing them in a concrete way in causing students to be able to generalize the concepts to a new situation. However:

* The study authors confuse the ability to produce correct calculations with the understanding of the concepts taught; and more importantly,
* The study authors used a lesson that was poorly adapted to concrete situations.

I conclude that their results don’t have the scope they claim. At most, they show that throwing in distracting information can make math harder to learn, but we already knew that. A better conclusion is that if you’re going to use concrete examples to teach something, be real about it. Don’t present a concrete setting and then dive straight into formal properties; use the setting to bring out the properties by having students think naturally about the setting.

The details:

First of all, the study authors declare in the online supporting material that “in the present research, study participants learned the concept of a commutative mathematical group of order three.” If that doesn’t mean anything to you, don’t worry, I’ll explain it very clearly in a moment. But this sentence already suggests an important flaw in the study design that I actually think is a flaw in a lot of curriculum design, so I’ll highlight it:

“The commutative group of order 3” is a math object you learn about in a first course on abstract algebra. The “training” the authors used to introduce the idea to the students basically describe the group’s formal definition. (I’ll show you exactly what they do below.) But the students’ learning was assessed by a test that measured the ability to do computations inside the group. The thing taught and the thing tested were not the same at all. I say this is a flaw in a lot of curriculum design: we want students to have a deep understanding of mathematical objects, like fractions. But they’re often just tested on their ability to produce correct computational answers. Knowing how to add 1/2 and 1/3 is not the same as knowing what 1/2 and 1/3 are. Lots of kids never really learn the meaning of the objects (or even register that this is the important part about learning math) because they know they are only going to be accountable for the computational technique. The irony is that in the end, this makes the computational technique much harder to learn. But we set them up for this as long as the only things we ever test them on are the computations.

Now the bigger problem with the study is how the “training” used was inappropriate to concrete situations. But before I can show you this you have to understand something about the concepts and skills supposedly being taught. So, that explanation I promised (skip the next section if you already know this stuff). I am actually going to use one of the “concrete” situations from the study! But I am writing the “training” a little differently.

You know how tennis balls come 3 to a container? Suppose you work in a tennis ball factory packing balls. Every time you get 3, you can pack them away into a container and send it down the assembly line.

So, suppose in front of you is 1 tennis ball, and then along the conveyer belt come 2 more. You pack them away, send them along, and then you have – none!

How about if you have 0 balls, and 1 comes along? Then of course you can’t do anything because you don’t have enough for a container. So you’re stuck with 1.

Or if you have 1 ball, and 1 comes along? Again, you don’t have enough for a container, so you’ve now got 2. But if 1 more comes along, it and the 2 make 3, you can pack them up and send them along, and so now you’ve got 0 again.

Or – this is “the hard one” – you have 2 balls, and 2 come along. You pack up a container, send it along, and you’re left with 1 ball.

I think you’ve probably got the gist by now. This is an alternative arithmetic in which the numbers never go above 2 because every time you get 3 balls you just pack them up and send them along. So, 1+1=2 like normal, but 1+2 = 0 and 2+2 = 1.

Okay, that’s it. You can now do arithmetic in the “commutative group of order 3.” You can probably out-perform all of the study subjects in Kaminski et al.’s study on the tests they took at the end.

Compare this to the training set in the exact same “concrete situation” from the study (this is found in the online supplemental materials):

“A tennis ball manufacturing company is having trouble with their ball-making machine. Instead of producing batches of three balls to fill a container, it is producing batches of zero, one or two balls represented as {no balls}, {1 ball}, and {2 balls}. Consequently two or more batches need to be produced to fill a container. In doing so, the number of extra balls produced needs to be determined.

“Rules for finding the number of extra tennis balls:

“Rule 1. The order of the batches doesn’t matter. The number of extra balls will be the same. For example, if this batch {no balls} is made first and then this {1 ball}, then this much {1 ball} is extra. The same thing happens if this batch {1 ball} is made first and then this {no balls}. We will have this much {1 ball} extra.

“Rule 2. If this batch {no balls} is made with any other single batch, the other amount is always extra. Here are a couple of examples: If this {no balls} and this {1 ball} are made, then this {1 ball} is extra. If this {no balls} and this {no balls} are made, then {no balls} is extra.

“Rule 3. If {1 ball} and {2 balls} are produced, then one container can be filled and {no balls} is extra.

“Rule 4. If {1 ball} and {1 ball} are produced, then we cannot fill a container. So, {2 balls} is extra.

“Rule 5. If {2 balls} and {2 balls} are made, then one container can be filled and {1 ball} is extra.

“Rule 6. If more than two batches are produced, the order in which they are made doesn’t matter. The extra will be the same. For example, if {1 ball} and {no balls} and then {2 balls} are made, then {no balls} is extra. The same amount is extra if {no balls} and {2 balls} and then {1 ball}.”

Now if you actually have an abstract algebra background, you see what they’re going for here. Rule 1 is saying the group operation is commutative, rule 2 is saying that {no balls} is an identity, rule 6 is sort of a bastardization of the associative property, rule 3 declares that {1 ball} and {2 balls} are inverses, and the other two rules are examples of arithmetic in the group.

But my point is what they’re not going for. They’re not making use of the concrete situation to help the reader understand what’s going on. In fact, the very first “rule” leaves the reader with the sense that we might as well stop imagining tennis balls right now. If we were really talking about tennis balls, I wouldn’t have to tell you that the order in which they come doesn’t affect how many you have. Everybody knows this. The excessive formalism divorces the story from any concrete reality.

Rule 2 makes the divorce deeper. If we were really talking about tennis balls, I wouldn’t call {no balls} a “batch.” So by the time you come to rule 3 (when finally, if barely, the concrete situation is referred to in a sensible way – “one container can be filled…”) you have already completely given up on the idea that what they are talking about has anything to do with reality as you know it.

So here’s the lesson: if you’re thinking about using a concrete example to introduce a math idea, don’t worry. The results of the study don’t really mean “it’s better to leave the apples etc. in the real world” as the NYT writeup asserted. The safe conclusion is this: don’t be phony about it. If you are going to use taxi meters to develop linear functions, use the kids’ knowledge of how that situation actually works to bring out the math ideas. If you’re using temperature to explain negative numbers, use it with examples that actually make sense when you think about temperature. (For example, you can use temperature to explain why 5 – 7 is -2 but not why 5 – -7 = 12.) Being excessively formal about something that’s supposedly concrete reality makes the kids stop listening to their own logical reasoning and common sense and start just trying to guess what you’re up to.