April 04, 2005
Blind Grading
Posted by Christine Hurt

A conversation on blind grading began over the weekend as comments to an unrelated post.  I am moving the conversation to its own post so that this valuable discussion does not get lost.

The comments raised two different but related issues:  (1) Is blind grading a myth? and (2) Why do professors grade participation?  I'm going to address these questions in reverse order because the second question has the shorter answer.

Why do I grade participation? Grading participation is an administrative hassle, but this year I have chose to include participation in the final grade as a "carrot." The one thing that professors want is an active class. I want students to read the material, come to class, and talk about the material. I want volunteers and I want students to answer my questions when I call on them. I don't even care what your answer is. I am not in love with the sound of my own voice (for many reasons!) and I abhor that Ferris Bueller silence ("Anyone, Anyone?") I also hate it when I call on students and they read out of the book. So, I can try to achieve my classroom goals in many ways. First, I could have an attendance policy and reduce your grade if you don't show up. That's a stick. Instead, I use a carrot. I will raise your grade if you suit up and show up. In practice, it may work like a stick. If the whole class except one comes to class and participates, then effectively the no-show's grade is reduced because everyone else's grade benefits.

Is Blind Grading a Myth? No.  I have been teaching since 1998, and I have never seen any evidence that any professor graded exams on any other basis than blind.  Yet, this perception still continues among students.  I'm not sure if there is anything that I can add to dispel that misperception, but I'll try.  First, I think this perception is based on an assumption that the professor has either an intense like or dislike for another student that would trump that professor's own good judgment.  To give a biased grade, a professor would have to be willing to put his or her own career at stake to either benefit or abuse a student to the tune of a few credit hours of a total of 90-100 credit hours.  I can't imagine that the enormous cost would justify the small benefit.  Absent an inappropriate romantic relationship, a professor has too many students over the course of a career to become so passionate about one student's situation to get involved in that manner.

Second, I also think that the elusive quality of grades, especially first-year grades, leads students to latch on to bias as a factor in grades.  Unlike in undergraduate courses, grades in law school do not equate neatly to effort.  As you sit in your first-year Torts class, you guess who will do well based on who seems to study the most.  You gauge your own studying accordingly.  However, the resulting grades surprise everyone.  How did the professor grade these exams?  As students struggle to make sense in a world that seems more arbitrary every minute, law students seem to subscribe to either a chaos theory of throwing the exams down the steps or a bias theory of nonanonymous grading.  (We were convinced our Crim Law teacher randomly assigned grades.) 

Because it is too unwieldy for professors to hold after-exam rap sessions where the professor explains why each student received each grade, there is a gap in the information.  Neither students nor professors are very good at guessing who will get the top scores in their classes because most exams are written to grade application, not information-gathering, and the ability to apply knowledge is not readily recognizable until the exam.  I know that last semester, I was surprised by the top-scorers in my Torts class.

Finally, once students are willing to consider that the professor was biased in the grading, students may point out that some high-scorers were students that the professor called on in class a lot or were talking to the professor after class.  The cause and effect part of the scenario is not: Student A talks to Professor B after class a lot/Professor B gave Student A a high grade/Professor B plays favorites.  The cause and effect part is probably:  Student A really likes Torts and understands it/Student A wants to ask questions that are beyond the class discussion, so Student A talks to Professor B after class/Student A gets a high grade in Professor B's class/Student A has an aptitude for analyzing the rules and policy of Torts law.

Professors are human.  We like students who like us.  We call on students who raise their hands.  We go back to students who give good answers and do the reading.  We like to chat after class with students who are interested in the material.  However, we do not take actions to ensure that those students will get good grades.

Law Schools/Lawyering | Bookmark

TrackBacks (7)

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d8345157d569e200d83422cbeb53ef

Links to weblogs that reference Blind Grading:

» Blind Grading of Law School Exams: from The Volokh Conspiracy ...
" Christine Hurt gives a lawprof's perspective over at [more] (Tracked on April 4, 2005 @ 8:02)
» Blind Grading of Law School Exams: from The Volokh Conspiracy ...
" Christine Hurt gives a lawprof's perspective over at [more] (Tracked on April 4, 2005 @ 8:03)
» Interesting Conversation from Letters of Marque ...
"There's an interesting conversation over on The Conglomerate. It started out about blind grading; it ..." [more] (Tracked on April 4, 2005 @ 19:53)
» Speak up! from Ditzy Genius ...
"Today I was reading the conversation over at The Conglomerate about blind grading. A lot of the stud ..." [more] (Tracked on April 5, 2005 @ 13:06)
» The right reason to bitch about law school exams from Glorfindel of Gondolin ...
"Anytime you talk about grades in law school, you inevitably set off a law school bitch fest. I'm (us ..." [more] (Tracked on April 5, 2005 @ 14:26)
» The right reason to bitch about law school exams from Glorfindel of Gondolin ...
"Anytime you talk about grades in law school, you inevitably set off a law school bitch fest. I'm (us ..." [more] (Tracked on April 6, 2005 @ 11:57)
» Is Grading Really Blind? from Blawg Wisdom ...
"As spring finals approach, there’s no better time for a huge discussion law school exam gradin ..." [more] (Tracked on April 10, 2005 @ 21:23)
Comments (71)

1. Posted by jh on April 4, 2005 @ 8:31 | Permalink

Do you keep a record of class participation? If not, I think participation grades are wrong--just a way for profs to reward people who brown nose. If, on the other hand, the prof writes a check mark per comment or something, I think it is good.


2. Posted by phred on April 4, 2005 @ 8:32 | Permalink

I'm unclear on what you mean by 'blind' here. You argue that blind grading is a myth based on the costs of giving a biased grade. Grading can be biased with or without being blind (e.g., biased against bad handwriting), and it can be unbiased with or without being blind (e.g., where the professor doesn't even know the students' names). Is your argument that law professors don't circumvent exam anonymity because the only reason for doing so would be to bias a grade, and biasing a grade isn't worth it? Or is it just that by 'blind' grading, students just mean 'unbiased' grading.


3. Posted by Christine on April 4, 2005 @ 8:45 | Permalink

1. I do keep a running list on participation, and I think that is the norm.

2. I am using "blind" grading to mean "anonymous" grading, so when I then refer to "biased" grading, I refer to grading that would be biased in favor of a student's known identity. I think all good professors struggle with other types of conscious bias in grading law school exams -- bias against bad handwriting, bad spelling, incoherent sentences, and stream of consciousness organization. I tell my students ahead of time that I will try to stay with them and wade through their exam looking for substance, but I'm not sure how many profs are willing or able to do that. I had 43 Torts students last semester. The next time I teach it, I will have twice that. I'm not sure if I will have the patience to hunt for substance if the form becomes too much of an obstacle. (I only had one exam that I would describe that way.)
And at some point, form and substance merge. You could say that I have a "bias" against superficial analysis, and I would admit that I have a "bias" for well-reasoned answers.


4. Posted by D. Sgruntilt Wonelle on April 4, 2005 @ 8:53 | Permalink

So while you're yakking it up after class, day after day, sharing subtleties of Torts beyond the class discussion with the non-favorite favorites, we students in percentile 1-50 stand angrily by trying to get some remedial help, and feeling further lost by the substance of your musings with your non-favorite who winds up with the A. Your conscience should be clear, since the non-favorites pay so much more for the privileged of sitting in your class than the rest of us, and you have no obligation to make sure we all get a minimum level of instruction. Nice ivory tower there, pal.


5. Posted by Joshua on April 4, 2005 @ 9:04 | Permalink

I'm just a passer-by, neither law prof nor student, but I would think that the "minimum level of instruction" that the professor is obliged to provide would be covered by the class, and perhaps office hours. Maybe the disgruntled should try asking questions during the actual class, rather than pinning their entire hopes on the few minutes of informal yakking afterwards?


6. Posted by Bob Dobalina on April 4, 2005 @ 9:10 | Permalink

Don't fellow students hate "gunners?"


7. Posted by Kate on April 4, 2005 @ 9:10 | Permalink

Sometimes there are students who hijack the class and WILL NOT shut up when all you had was one or two short questions. These students think they know everything there is to know about, say, Torts, and will ramble on and on about things that are a chapter or two ahead. So you wait till after class to ask the simple question and there is another one, going on and on about something he read in a magazine article and compares it to something amazing he read in a hornbook. Classes are not that long.


8. Posted by Scott Moss on April 4, 2005 @ 9:10 | Permalink

Dear D. Sgruntilt Wonelle:

Perhaps the reason you're in percentile 1-50 is not some professorial conpsiracy against you, but the simple fact that you can neither write a coherent parahgraph nor respond in a civil manner to a reasoned discussion.


9. Posted by DBL on April 4, 2005 @ 9:12 | Permalink

One way to help students understand how exams are graded is to make available, after the exam, the best student exam paper in the class for all to read. Your average students will marvel at all the issues they missed, at the subtleties they glided by, and will begin to perceive that grading is not purely random.


10. Posted by Scott Moss on April 4, 2005 @ 9:32 | Permalink

In the prior thread that turned into a discussion of blind grading, I at one point posted that as I applied Marquette's blind brading system, “we grade blind, turn in the grades for each exam, and then get a sheet back with those grades attached to names. We then can change the students' grades based on participation, attendance, or other relevant factors (in my view, 'factors I mentioned in the syllabus')."

And then an anonymous poster responded, “So, you people admit that exams are not really graded blindly. Grading things up or down with a 'specific justification' means they are not graded blindly.. . . Yes. I think professors act in bad faith. I am not saying that they are sitting around evilly cackling about how to harm or help students, but they are usually quite up front about 'the type' of person they like to see succeed. Students often try to pretend to be that type of person and usually it works, especially if that student has had years of practice.”

This is an interesting response that bears response.

(1) This clearly IS blind grading of exams, which eliminates the real possibility that professors will be biased in assessing how well you answered the exam questions -- biased based on which students professors expect to do well, biased based on race or gender, whatever.

(2) I can't see how it's "bad faith" when "professors ... usually quite up front about 'the type' of person they like to see succeed." I think the ideal grading system is (a) blind grading of exams plus (b) full, transparent disclosure of what else affects your grade. That's why I'll go up or down from the blind exam grade only for factors I mention in my syllabus.

(3) I think it's 100% educationally legitimate for professors to have a "type" of student who will get a boost for participation. Obviously a bad or vain or stupid professor can choose an inappropriate "type" to grade up, like a preference for suck-ups or a gender bias or whatever. But I don't think I'd be grading students' academic performance accurately if I said, "OK, two students got a BC on my exam, and one of them consistently gave cogent analyses in class, whereas the other never spoke a word -- but I have to give the same BC grade to both." I think a higher level of educational performance -- legal reasoning, ability to form and respond to arguments, etc. -- was shown by the student who had the BC exam but all semester showed a strong ability to comprehend and apply case law. That student earned a B rather than a BC in my book.

(4) In law, there's always a tension between hard-and-fast "rules" and murkier "standards" (as my students know). I respect professors who go by blind grading with no participation adjustments; that "rule" has the advantage of clarity/transparency -- but it's a mistake to think that automatically means it's the a superior system. The "standard" I apply, i.e., modifying blind grading (for only a small number of students, incidentally) based on fully disclosed considerations, is slightly less transparent but has the advantage of, in my view, more accurately assessing eductaional performance. The trick with "standards" is to make them as transparent and predictable as possible. I go up or down from the exam grade for only a small number of students; I disclose the limited number of reasons for an up-or-down adjustment in my syllabus; I never go up more than one grade notch; I won't adjust downward without very good reason, and I won't go down more than one notch without very very very good reason.

I think that most of my students would guess right if I asked (though of course I won't really ask) whether they thought they'd get a plus factor, which is to say I think my system maximizes accuracy of assessment without sacrificing all that much transparency.


11. Posted by been there, done that on April 4, 2005 @ 9:38 | Permalink

Blind grading is a myth.

If a student is outspoken in class, or just plain spoken, but with a point of view that the professor finds politically incorrect, it isn't hard to figure out who wrote which exam. There's no "risk" to the professor to figure out which paper sounds like who, and grade accordingly.

I was graded based on ideology by my communist professors as a 1L but I quickly caught on. Second and third year, I made an effort to sound like Ralph Nader, and I got a near perfect GPA. In one class, my professor just about announced that I and another right-wing kid were going to fail; he looked at us and apologized for being unable to teach everyone. I got an A, and surprised him with how wonderfully progressive I truly was. My friend wrote an equally insightful, coherent, but honest exam answer, and got a C.


12. Posted by Kate Litvak on April 4, 2005 @ 9:38 | Permalink

Many of my students are actually quite happy to get an adjustment for class participation because they want to diversify the factors that go into their final grade. Adjustment for class participation not only encourages work during the semester, but also reduces the element of randomness and luck in grading. If you came to all classes prepared, asked thoughtful questions (and yes, we can tell which questions are thoughtful and which are nonsense, give us some credit here), and otherwise demonstrated steady diligent work throughout the semester, you will not get hit with a very low grade just because you had a bad case of flu on the day of the exam or got nervous and misread the hypothetical.

I find it ironic that the same students who complain that their entire grade hinges on the performance during a single 3.5-hour exam also complain about us using anything other than final exam in grading. And no, I cannot use blindly-graded midterm to diversify grading (that would be my first-best outcome): our school does not allow me to hire a TA to grade student exams. I have no physical capacity to grade 120 midterm exams and give meaningful feedback in the middle of the semester. So, class participation is it: it's not ideal, but better than alternatives.


13. Posted by Christine Hurt on April 4, 2005 @ 9:39 | Permalink

I think this thread hit on several tensions in law teaching. First, I recognize that classes can get hijacked, as well as post-class podium time. I think that some law professors are adept at not letting a few students monopolize class. However, most law professors I know would prefer that anyone talk in class than no one. Monopolizers generally are born because a vacuum was there to be filled. Please don't sit in silence. Raise your hand, and then the professor has a good reason to cut the monopolizer short.
Second, to some extent law school learning is about self-help. If you have a question, ask it. If you have a post-class question, stick around after class or come to office hours. I never leave the classroom until I've heard every question. If your patience is shorter than mine, then come to office hours. I sit every week in my office hours, and nobody comes. (Until of course, the week that I have a dr. appt. or something.)
Also, every student has the opportunity to look at their exam. I don't post the best exam paper, but I create a model outline that you can look at it if you come to my office. Out of 43 students, I think I had 3 students come and look at their grades (high, low, and middle). I have not wanted to post a model answer because of thoughts that I may want to use similar questions in the future. I may weigh that against the benefit in the future, though.


14. Posted by Scott Moss on April 4, 2005 @ 9:48 | Permalink

"I had a wacky, psychotic professor who graded me unfairly and therefore all blind grading is a myth." That's the gist? Any professor can be unfair -- as noted, even with blind grading, an unfair professor can grade you on your ideology. A well-defined, transparent system of adjusting blind grades (of the sort I mentioned above) would control these aberrational situations. We don't need to ban all discretion in grading, as if it were possible to do so (given, as noted, the discretion in grading even blind exams).

Law students: there's so much in American legal education worthy of reform and discussion (e.g., depending on the institution: teaching quality; practical skills education; ideological bias; and all the studies showing the high rate of stress and mental distress that law school causes) that "blind grading isn't REALLY 100% blind" should be about 897th on your list of 893 topics on which to complain. I think some of what's really going on here is mistargeted complaining by justifiably disgruntled law students. Channel your disgruntlement (?) toward the real problems, not this sideshow.


15. Posted by Christine Hurt on April 4, 2005 @ 9:48 | Permalink

This comment is in response "Been there." Let's call the kind of bias that you are talking about "ideological bias." Your hypothesis is that professors have ideological biases that affect grading. These biases could be conscious or subconscious. I'm sure others have stories of instances where you clashed with the professor on an ideological basis and you recieved an unpleasant grade. I have no way of proving or disproving your hypothesis. I can think of examples of people I knew in law school who clashed with professors but still did well, but that doesn't disprove your hypothesis either. I don't teach that way or write exams that way. My exams generally require you to analyze both sides of a cause of action, so you would have to know different kinds of arguments.

I will say that when I'm grading, a thought will come into my head like, "I bet this is Mr. X's paper," but I'm always wrong.


16. Posted by Not on the Waaambulance on April 4, 2005 @ 10:01 | Permalink

I don’t have time to form a write a point-by-point breakdown, but I should state that in my experience, ideological bias can be found for all sort of political and religious views, but it pretty rare in law school I am not going to get on the “I am a Conservative and so oppressed” waaaamulance, and I don’t think anyone else should either.


17. Posted by been there, done that on April 4, 2005 @ 10:06 | Permalink

OF COURSE you have to know both sides of the argument, identify all the issues, and carefully consider everything, etc. etc.

But if, in a torts class, after exhaustive analysis, the question asks, "how would you decide this case? who wins?" the answer is always PLAINTIFF. It also doesn't hurt to kick in a comment about the joys of wealth redistribution.

I don't know you and I'm sure that YOU aren't biased.... but I went from near the bottom of the class first year to the very top in years 2 and 3, and it wasn't because of steroids or any notable improvement in my handwriting. Sorry, but most law school profs are ideological hacks. (If you're wondering, let's just say I went to a top 10-15 law school, so this isn't hinterlands behavior.)


18. Posted by John Steele on April 4, 2005 @ 10:25 | Permalink

I've taught at a few schools. At Boalt, you grade exams anonymously, and then get the ordinal rank with names, at which point you can move students up or down based on participation. I announce at the beginning of the semester that class participation will be especially important in moving up high Passes to Honors grades, and I provide an estimate, based on prior years experience, of how many students will move up to Honors grades and how many low Honors grades will move down to Pass. I tell them that because it's only fair to inform them, and I adjust the grades because class participation makes the class much better for all students.

At Santa Clara, you must submit a list of plus points and minus points for class participation at the same time you submit the blindly graded exams. So I don't have quite the same fine control as I do at Boalt.

As I recall, Stanford was more similar to Boalt than to Santa Clara on that point.


19. Posted by Kate Litvak on April 4, 2005 @ 10:27 | Permalink

Since this thread has degenerated from blind grading into ideological-bias discussion, here is the question. Is it ethical to steer your students away from other profs who you know won't be a good ideological match for them? A fair number of my first-year students asked me for recommendations on upper-level courses. So far, I’ve been recommending them to “take people, not subjects” – go for the most interesting profs no matter what they teach. By “most interesting” I always mean “most academically accomplished” – smart, engaging, provocative, deep. Should I also start mentioning their politics, for full disclosure? I just told a conservative future prosecutor to take Evidence with a very liberal prof, whom I respect greatly for the sheer brain power. Good idea?


20. Posted by TJVM on April 4, 2005 @ 10:45 | Permalink

Most of the analysis so far seems to focus on whether grading on participation is fair or not. That’s not surprising. However, I wish that professors would consider the fact that this practice seems to have a significant, and bothersome, side effect – namely, the encouragement of “participation” designed to impress the professor.

I think it’s safe to say that a law school student body includes a significant number of people in two categories: (a) highly-motivated students driven to get high grades; and (b) opinionated people who like to hear themselves talk (there’s no doubt some overlap between those groups). Offering the possibility of higher grades in exchange for class participation triggers endless blathering that is clearly designed to win brownie points (either by brown-nosing, or expressing sympathy for the professor’s point of view, or just trying demonstrate that the student did a super-thorough job on his homework).

Whether or not this works, it consumes a substantial portion of instruction time. As best I can recall, there were only a handful of instances where I heard genuinely helpful “analysis” from fellow students. However, I clearly recall spending hours and hours in law school waiting for some idiot(s) to shut up so that we could get on with actual learning. I suspect that most law students would have a similar impression of “class discussion” time.


21. Posted by John Steele on April 4, 2005 @ 10:46 | Permalink

Kate:

I would lay it all out to the students and let them decide. It's particularly important for students to know which profs grade ideologically. Law school is very expensive, and grades can translate into big money differences. The best of all worlds is that profs don't let ideology affect grades, but the second best world is one where students can decide whether to avoid those profs, feed back ideologically correct answers, or say what they really feel and take the grade hit.

One difficulty with making full disclosure is that many profs, even ones with strong ideological views, don't grade ideologically, and it can be hard to figure out which ones do. So, simply stating the prof's politics might interject politics where it's not relevant at all.


22. Posted by D. Sgruntilt Wonelle on April 4, 2005 @ 11:16 | Permalink

Dear Scottie:

Did you ever notice that the universally beloved professors, the ones who win awards from their students for the quality of the education, are never the ones with the mile-long CV of publications and speaking gigs?

The point is that we dumb kids catch on that there are professors who favor (whether on purpose or otherwise) gunner students. Great professors manage to reach everyone more equally.

I.e., I guess all my professors would know if they get a plus factor from gunners.


23. Posted by fredbinkle on April 4, 2005 @ 11:21 | Permalink

YOU WROTE: "Because it is too unwieldy for professors to hold after-exam rap sessions where the professor explains why each student received each grade, there is a gap in the information. Neither students nor professors are very good at guessing who will get the top scores in their classes because most exams are written to grade application, not information-gathering, and the ability to apply knowledge is not readily recognizable until the exam."

MY COMMENT: This is the great flaw in legal teaching. It does not teach the student how to apply the knowledge you are so glibly doling out. It is as if you are teaching people to play the piano without giving the student a piano to practice on.

And none of my law instructors -- not one -- was willing or able to explain how to write the exam for his course. They attempted to paw off this very important responsbility to the legal writing instructor, who, at my school, could not teach the subject in anything other than generalities. And by the time the writing course rolled around, it was already second year.

In other words, the current method of teaching gravitates to those who somehow are naturally able to write the exam the way you wish it to be written. All others sink or swim. You -- the legal educator -- have abdicated the responsibility of teaching the fundamental skills to succeed in your course. BarBri Essay Review taught me a hell of a lot more about writing well for the law than Law School.


24. Posted by Kate Litvak on April 4, 2005 @ 11:37 | Permalink

>>Did you ever notice that the universally beloved professors, the ones who win awards from their students for the quality of the education, are never the ones with the mile-long CV of publications and speaking gigs?

No, didn't notice that. Brian Leiter, Pam Karlan, Sam Issacharoff, John Manning, Eugene Volokh, Ernie Young... much-beloved teachers, excellent scholars. The list can go on.


25. Posted by asd on April 4, 2005 @ 11:47 | Permalink

It's entirely possible for teachers to unknowingly bias grades in small classes. E.g. thinking that a student glossed over something because (s)he is so bright that it seemed obvious. I've experienced this firsthand in a one philosophy class where I think my grade was enhanced by a (Platonic) relationship with the prof. It's possible to come off as brighter than you actually are by carefully restricting when and how you participate.

Of course there's a lot of truth to the claims that biasing takes too much effort, especially in large classes, and the strategy of *seeming* very bright can backfire since a prof can also downgrade since this student could have done so much better. In technical disciplines, a lot of students believe that this sort of biasing happens frequently. "My friend gave pretty much the same answer that I did, but I got half off." In my experience as a student, this is usually caused by missing subtle but important concepts and fallacies (e.g. missing a base case in induction).

Most claims of bias are, I think, caused by misunderstanding the nature of studying. When one student studies for 10 hours and gets a C and another doesn't study at all and gets an A, bias seems like the only reasonable explanation. The truth is that some students don't understand the material so no amount of studying (usually interpreted as memorization) will get a great grade. Others have a good understanding to begin with, so little studying is required.

(An amusing anecdote from highschool: in my precalculus class, we studied logarithms. For some reason, many students had a lot of trouble with the basic concepts, even though everything can be derived from the definition of log in about two steps. My test consisted of half a page of dense calculations for which I received a perfect grade. Another student saw my test, and complained to the teacher that it wasn't fair that she wrote over five pages (!) and was rewarded with a D. The teacher's response was priceless: "Grades are not directly proportional to length in math. Maybe in English, but not math.")

Post a comment

If you have a TypeKey or TypePad account, please Sign In

Bloggers
Papers
Posts
Recent Comments
Random Walk
Search The Glom
The Glom on Twitter
Archives by Topic
Archives by Date
July 2009
Sun Mon Tue Wed Thu Fri Sat
      1 2 3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30 31  
Syndicate The Glom
Subscribe

The Glom's Blog Network on Facebook:

Miscellaneous Links