On Usability Testing and Not Finding a Silver Bullet in Haswell

Two years ago at Mayinar, Joyce made an offhand comment to our usability class to the effect that someone ought to do (or maybe had done—I can't remember; it was Mayinar after all) usability testing on instructor feedback to student writing. Apparently the results have been in for a while, says Haswell, and the news isn't good. Of course I've known that for a while, and his comments that students tend to follow only surface, directive, negative comments and ignore global non-directive and positive comments were exactly in line with my experience. So when I saw the title of Haswell's piece, I was ready for The Answer; I wanted to hear the definitively perfect way to comment on essays that would prompts deep thought, stunning insight, and a startling increase in work ethic. But, while I didn't get that, I did get an article that forced me to consider my own "teacher culture" as manifest in my comments. I think here the four instructional types could be linked back to Haswell's discussion of Du Gay's circuit of culture, and in particular to regulation, what Haswell calls "The Epicenter of Response." (Perhaps I'm messing up my Du Gay here, as the four types also echo Haswell's discussion of the 12 types of representation). In any case, my takeaway here is that I need to comment in accordance with the type of instruction I want to do (discipline specific feedback), which seems pretty obvious but I think often doesn't happen. In particular, if I teach process and rhetoric, but I spend my time marking up grammar, then I'm really a traditional/formalist. So how will this change my procedures? Well, tentatively, I think I need to separate feedback. I do still believe that final drafts should be largely error free, but I think it's far more important that they be effective arguments representing real insight. So maybe I just need to have multiple drafts with comments on each geared toward the level of completion (early drafts ignoring surface stuff entirely, etc). Of course this would mean at least doubling my grading load, which would mean abandoning my family and dropping out of TTU (and going entirely batty), so I need a way to do this that doesn't involve stacks of papers and hours at Starbucks.

My solutions here are as yet unformed. Way back when I began I used to have 30 minute conferences with three students where they'd read their papers out loud and we'd have a quick discussion. I'd really love to get peer reviews working well, not only as a time saver but because (and much more importantly) teaching is the best way to learn, and that's a possibility. But the final idea came from Haswell mentioning the teacher doing a think-aloud protocol. In fact, I remember Joyce talking about a conversation she'd had with a rejected applicant to the online program. Apparently she read through his application and simply talked through the mental process that led her to reject him, and he was both a bit stunned and quite grateful. What I love about that is the way it demystifies what is normally such a secret process. We don't want our grading process to be a secret (just the opposite, in fact, right? Isn't that the whole point of teaching?), and we try to write enough to clarify it, but really we can never get it all down on paper, and anyway the act of writing it down is itself an act of invention and revision so we're already removed from the original. So perhaps to get the peer review process going, and to demystify the reality of me as the audience for my students' writing, what I really need to do is talk through the grading process and reveal my thoughts, warts and all.