I recently finished reading this article that describes a research report commissioned by the National Association for College Admission Counseling. The report discusses the effect of test preparation and tutoring services on SAT scores.  It suggests that gains from test prep services are minimal, only around 30 points, but that even 30 points can make a difference.  These results are thought to validate the College Board's longstanding position that the SAT isn't really coachable.  The report suggests that there be more research into this question, as all the current research has been done by the College Board.

As I read about this report, everything in it seemed to fit together logically, but there was an element that was missing.  I'd like to make an important distinction, one that might sound obvious but that wasn't mentioned anywhere: there is a difference between good tutoring and bad tutoring.

The difference between good tutoring and bad tutoring is like night and day.  Bad tutoring is pernicious because it makes you think you're learning something about the test.  In reality, you're just being forced to memorize a bunch of abstractions about the test, and then when you take the test you realize that nothing has really changed from the first time you took it, except that you expected to do better the second time.  In most debates, this falls into the categories of "teaching to the test" or "getting comfortable with the format."  These approaches don't really provide any benefit and are rightly downplayed.

Good tutoring works in an entirely different way.  Good tutoring discourages quick fixes and works instead at increasing depth of understanding.  If a student is having difficulty with a particular problem, good tutoring first seeks to understand why before it suggests a solution.  Students often struggle even with good tutoring because they are working to overcome longstanding biases about their own knowledge and problem solving ability.  Once they overcome those biases, however, they arrive at a new plateau where the test in question is fundamentally easier for them to solve.

The tricky thing is that it's tough in advance to determine which tutoring is good, and which tutoring isn't.  Both make reference to "special techniques," "expert faculty," and "lots of experience," but the unfortunate truth is that there's no way to tell whether it's going to work for you, unless you've had some experience in the field.  Even though this distinction is hard to spot, the distinction does exist, as is obvious to anyone who has gone through the process.

I'm making a big deal about this because the conclusion of the article is that test prep categorically doesn't really help that much, that the test is relatively uncoachable.  For bad test prep, I have no problem with that conclusion.  On the other hand, to pretend that good test prep can't improve someone's score significantly, even hundreds of points, is pretty darned silly.

Think about it this way:  do you, the reader, know of any activity where you aren't able to get better than you were the first time you tried it?  I'm pretty sure that I would be the world's worst ballet dancer, but I know that if I laced up some shoes and took a bunch of lessons, then I would probably be able to at least do some of the very basic moves.  Think about it for a second:  does it really make sense that there is this thing called a standardized test that is totally immune to any sort of preparation or improvement?  That humans are able to invent a test that they themselves can't master?  I hope this sounds as wacky to you as it does to me.

The fact of the matter is that good tutoring can have a huge impact on your scores, and the result is not due to some magical formula.  It comes from understanding students, addressing their concerns, challenging them on their bad habits, and trying at all times to get them to see the bigger picture.

I have eight years of experience in thinking about these things and helping students improve their sccores.  As long as the there are reports out there that contend the SAT is uncoachable, I can keep doing what I'm doing, because the makers of the test can pretend I don't exist.  Unfortunately, these types of reports also preclude an honest discussion about education and equity in the United States.  If test makers refuse to concede that their tests can be mastered, then students with resources will continually outperform equivalent students without resources.  Thus, the tests will be working at cross purposes with the problem they were originally intended to solve: to level the playing field so that everyone has equal access to education.

Comments? Thoughts? Please share.

Charles Williamson has helped hundreds of students prepare for standardized tests. He blogs for the Ivey Files about test prep, the intersection of education and technology, education policy, and whatever other topics strike his fancy.