Wednesday, April 28, 2010

A tale of two issues

One thing is undeniable: grades on Calc II exams have gone down since the last time I taught this course.

By quite a bit.

I don't think the students are any less bright, and I don't think my teaching has plummeted in quality since Spring 2008. The classroom format is nearly identical, many of the activities the same.

What's changed, and changed substantially, is my exam grading policy: instead of allowing a single round of revisions in which students can earn back up to 1/3 of the points they initially missed, I now allow unlimited revisions, with an opportunity to make up every single point missed.

My aim in allowing such revisions is to mimic a not uncommon practice in writing-intensive courses, in which students are allowed a chance to continually revise their written work until it meets whatever standard they hold for themselves: if a student is happy with the C she earns in the first place, she can stop there. But if she's so motivated, she can continue to clean up her work, polish her argument, practice her diction, until she's made the paper the best damned paper it could ever be.

Why shouldn't this translate well into mathematics?

I'm finding, as I mentioned above, that it's having a deleterious effect on the initial grades. On yesterday's exam (the third for the semester, regarding the infamous sequences and series), students averaged roughly 61% between my two sections. Though the topic is an inherently difficult one for beginning students, the exam was not overly long and fairly straightforward. It had no "curveball questions" (for instance, I felt that all four "test for convergence" problems were solvable by obvious choices of convergence tests) and two or three of the problems were nearly identical to those worked out during a review session attended by roughly 30 students the night before. I thought the exam was extremely fair. (Students, if you're reading this, feel free to chime in in the comments section. I'd be curious to know your thoughts on the matter.)

In the past my pre-revision grades have been in the neighborhood of 70% or 72%. What's caused this drop?

My hypothesis: students feel less pressure to do well on the initial go, knowing that they'll have the chance to make up whatever points they miss in the aftermath. Simply put, the stakes are lower, so students don't prepare as well. Granted, you're always going to have those "gotta have a 100" grinds (and I use this term appreciatively!) who'll by god be sure to nail it on the first go-around, but I think it's likely that most students, knowing that the stakes are low, will ease up a bit on the throttle. They'll forgo that preliminary all-nighter, they'll pass on the practice exam, and they'll sleep in an extra hour on the morning of the exam instead of getting up early to pore over integration formulas while they chomp on their Lucky Charms.

For one reason or another, the grades are lower. To me, though, the issue isn't so much the grades as it is the students' learning. I'll gladly accept lower grades on the initial run of an in-class exam as long as I know that the students are still effectively mastering the concepts we discuss in class.

But how to know that they're doing this? I would argue that, the way things are now, I simply can't know this, as the exams are now saying very little about how much students have learned. (It could be very cogently argued that few traditional exams have ever said anything about how much students have learned, but that's a post for a different day.) I suspect that my exams have ceased providing summative feedback of any kind and offer almost exclusively formative feedback. In theory, they may be providing this service very effectively: by bombing this or that question on a particular exam, a student can know exactly what she needs to brush up on before the semester's over, allowing her to very carefully target her studies for the next iteration of the exam revisions, and the next, and the next, until perfection is achieved.

If the exams really are performing this function, offering meaningful contribution to students' understanding, then the revision policy is definitely worth keeping. As my students well know, I don't give a rat's tuckus about grades so long as students are learning.

But are the students really learning? Are the exams, cut free of any indexical mooring they might once have had, performing any meaningful assessment function? How can I tease apart the two matters, grades and learning?

Students, if you're reading this, let me know by commenting on this post: do you feel that the exams (and, more to the point, the revision policy on the exams) are serving a meaningful purpose? Are you learning effectively by performing the revisions? And, honestly, do you feel as though you've slackened your efforts at achieving a high initial score in light of the revision policy? (Topology students, feel free to chime in too, regarding the similar revision policy on our homework.)

Colleagues, if you're reading this, let me know your thoughts as well: do you have revision policies? Do they work? Have you had to relearn how to discern students' mastery by means of exams?

A postscript: I realize I've been incredibly absent from this forum this semester; it's not for lack of things to say. In fact, I've found myself overwhelmed by matters pedagogical. It's a cruel thing that the semesters in which I've had the most food for thought are precisely those in which I've had the least time to reflect on those thoughts in writing. I hope this will change!


Derek said...

Here's my best theory: If your students weren't taking other courses, then your full-credit revision policy would likely have its intended effect. However, since your students are indeed taking other courses, they have to make very practical choices about how they spend their time. If they have a test in your course and a test in another course the same week, there's an efficiency in spending more time studying for the test in the other course since they only have one shot at doing well on that one. They know they can delay studying in your course to a later time when they don't have other competing demands without having their grades suffer.

I once had a student who failed to turn in the final project in one of my courses. He emailed me a few days after it was due and explained that he had final projects in four of his courses, didn't have time to do them all, and mine counted for the least. So he didn't do it. I couldn't argue with his logic, but I didn't give him an extension, either!

What do you think of my theory?

Michael F. said...

A problem that I have with exams (in general) is doubting that I know the material well enough, over-thinking problems, and then seizing up on test/quiz day. I usually end up kicking myself for a few days afterwards because of the simplicity of a problem or two that completely eluded me during the taking of the exam or quiz. However, with the revision policy you've had in place this semester in Calc II, the stress, nervousness, and seize-ups are pretty much gone. I still meet with a few classmates the day before and the day of the exam to do some extra studying to make sure we've got the concepts down, but going into the test I feel much more relaxed and okay with doing as best as I can. The relaxed atmosphere has made a drastic difference in my testing abilities, as I've done consistently well on tests on the first go and I nailed the 3rd test a week or so ago.

I can understand the logic behind students prioritizing the test due to being able to regain any lost points at a later date, but I've found that if you have a full slate of 5 classes (and work and extra-curricular activities, in my case =P ) it's really hard to find time to do revisions, and as such it pays off to do well the first time. And when I end up missing a question or two, the concept feels like it sinks in more solidly when I can resubmit it for official grading and feedback rather than taking the problem by the math lab and just smacking my forehead over it and being done.

Just my two cents.

- Michael F.

DocTurtle said...

To Derek: as always, your comments are thoughtful and insightful ones. I'm so glad we've managed to stay such good colleagues over the years!

You're right in that I'm probably idealizing student time and focus, in some sense; in order to re-establish a meaningfully high priority on my students' schedules, I need to build in some sort of deadline, even if it is a relative one. (After all, even in "real life," whatever that is, most every task we'll ever undertake must be done, and done completely, by a certain date.)

I'm thinking of building some sort of artificially diminishing returns: the first round of revisions might offer half credit back, the second round one-third, and so forth...or maybe a geometric, rather than a harmonic, diminishing? Effectively, after one or two rounds of revisions, students wouldn't get any credit back, no matter how well-revised their work.

Or, in order to offer an incentive to get it right the first time, I might allow unlimited revisions but ONLY for those students who pass the exam the first time around (where "pass" might mean obtaining at least 60%, or even 50%). The issue there would be that the students most assisted by performing revisions would be those who would be ineligible to perform the revisions in the first place. I can see my grades for such a course already: a bimodal chasm, with a cluster of Ds and Fs on one side and a cluster of As on the other. (However, I should mention that the grade distributions in all three of my courses this semester, in all three of which some sort of unlimited revisions are in effect, have hovered surprisingly close to the grade distributions in my classes historically.)

More to think about, for sure!

To Michael: I really appreciate hearing a student's take on the matter. It sounds like the revision policy is helping you, and in particular it's helping you in the way I would like it to, by alleviating stress and making the testing environment on which is more conducive to learning rather than assessment. As I've always said, I hope that my exams can offer as much an opportunity to learn as they can an opportunity to demonstrate skill (i.e., that they be formative as well as summative). It sounds like for you at least the policy I've adopted this term is having that intended effect.

Thank you, both, for sharing your thoughts!