August 02, 2008
Student evaluations: reflective, not reflexive
Posted by Gordon Smith

August is not the time of year when I normally think about student evaluations, but I have been re-designing my Business Associations course this summer, and in doing some background reading on teaching, I stumbled across an interesting article by Deborah J. Merritt called Bias, The Brain, and Student Evaluations, 82 St. John's L. Rev. 235 (2008). After describing several interesting studies exploring the importance of nonverbal behavior on student evaluations, Merritt offers this indictment of the present system:

The research on student evaluations is troubling. It confirms not some connection between a professor's style and student evaluations, but an overwhelming link between those two factors. Nonverbal behaviors appear to matter much more than anything else in student ratings. Enthusiastic gestures and vocal tones can mask gobbledygook, smiles count more than sample exam questions, and impressions formed in thirty seconds accurately foretell end-of-semester evaluations. The strong connection between mere nonverbal behaviors and student evaluations creates a very narrow definition of good teaching. By relying on the current student evaluation system, law schools implicitly endorse an inflexible, largely stylistic, and homogeneous description of good teaching. Rather than encouraging faculty to use nonverbal behaviors to complement excellent classroom content, organization, and explanations, the present evaluation system largely eliminates the "dog" of substance, leaving only the "tail" of style to designate good teaching. Neither law students nor faculty benefit from such a narrow definition of good teaching.

The psychology literature, moreover, identifies three further difficulties with the disproportionate role that nonverbal behaviors play in student evaluations. First, the behaviors that most influence these evaluations are rooted in physiology, culture, personality, and habit. Those behaviors are difficult for any faculty member to alter and they often reflect characteristics like race, gender, nationality, or socioeconomic class. Second, the current evaluation process allows social stereotypes to filter students' perceptions of instructor behaviors. Students see the nonverbal behaviors of some faculty differently than they view identical behaviors in other professors, potentially placing women and minority faculty at a greater disadvantage. Finally, the ratings that students award through the present evaluation system bear little relationship to objective measures of learning. The current system of student evaluations, in other words, rewards and penalizes faculty according to relatively trivial indicia, rather than what they accomplish in the classroom.

None of this seems new, exactly, but that's a nice synthesis of the perceived problems with the present system. Given my present interest in obtaining a better understanding of teaching and learning, I am most intrigued by the last complaint: that evaluations do not correlate with learning.

Can evaluations help us to become better teachers? Many law professors, particularly young law professors, use evaluations as an aid to improvement. After my first semester of teaching, for example, I went through the student evaluations and found five or six suggestions for improvement. In the next semester, I worked on those things. I did this every semester until the evaluations became too predictable to be useful. Now, I still read them, but usually rather quickly, unless I am looking for feedback on a specific part of the class (e.g., last semester I used teams for the first time, and I was curious to see how the students reacted in the evaluations).

But what if I wanted to know whether my teaching resulted in meaningful learning? Or whether certain changes in my teaching improved learning? Could evaluations guide me? The research summarized by Merritt suggests not: "The cumulative research suggests that there is little, if any, positive association between the ratings students give faculty and the amount they learn. The most recent study, in fact, suggests a negative correlation between evaluations and learning."

Hmm. Not so good. So, should we abandon our hope of improving our teaching -- rather than just improving our scores -- from
student evaluations? Merritt says not so fast:

Students have essential feedback to offer faculty on teaching. They can tell professors what they learned from a course and how that compared to what they expected to learn. They can describe the educational techniques that worked for them and those that did not. They can provide suggestions for how a faculty member might teach differently. Law students can assess the quality of their educational experience in myriad ways.

The key to unlocking this information is in the evaluation technique. We need a technique that allows students to be reflective, not reflexive. Merritt suggests something along the lines of Gregory Munro's Small-Group Instructional Diagnosis. The idea is to have small groups of students provide feedback to a facilitator: "The students discuss their perspectives as a group, expanding the information available to each student, checking individual biases, establishing accountability, and implicitly noting the seriousness of the process and need for accuracy. These group discussions reduce cognitive overload by focusing attention and providing adequate time for thoughtful assessment."

In this part of her paper, Merritt cites to Eric Orts' short essay on the use of quality circles in the classroom. Eric W. Orts, Quality Circles in Law Teaching, 47 J. Legal Educ. 425 (1997). By using a quality circle, you can create your own system of reflective feedback. No need to wait for the law school administration to implement a whole new system of evaluations. I used a quality circle once, shortly after Eric's article was published, and it seemed to work well. I am not sure why I haven't gone back to it, but reading Merritt's article has inspired me to try it again.

Teaching | Bookmark

TrackBacks (0)

TrackBack URL for this entry:
https://www.typepad.com/services/trackback/6a00d8345157d569e200e553cacb198833

Links to weblogs that reference Student evaluations: reflective, not reflexive:

Bloggers
Papers
Posts
Recent Comments
Popular Threads
Search The Glom
The Glom on Twitter
Archives by Topic
Archives by Date
January 2019
Sun Mon Tue Wed Thu Fri Sat
    1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31    
Miscellaneous Links