I have publicly proclaimed my affection for SSRN on this blog, but I object to recent efforts by Bernie Black and Paul Caron to promote SSRN as a measure of scholarly performance. I am very fond of Bernie and Paul, but I am highly skeptical of their claim that "there is reason to be optimistic that gaming will not seriously undermine the reliability of the law school and law author download measures."
If SSRN threatened to become a serious force in the world of scholarly rankings (Bill Henderson contends that my "if" is misplaced), ambitious schools would commit themselves to gaming the system. Could they do it? Absolutely.
Last night, I speculated that I "could improve my own ranking by at least a couple hundreds spots quite easily." Bill encouraged me to try, but I have decided not to provoke my friends at SSRN, who take this download stuff seriously. Instead, I will describe what I would have done to improve my ranking substantially overnight.
At present, I have 12 papers posted. My total downloads are 3,184, and my SSRN Author Rank is 966. To move up to #766, I need to achieve more than 3,687 total downloads. That means I would need at least 504 more downloads. How would I get that many new downloads in one day? Answer: many hands make light work.
First, I could use self-help, and download each of my papers from home and from the office. I might even stop at the public library and download some from there. At some point, if I download the same paper multiple times from the same IP address, SSRN stops me, but I could get at least 12 downloads in each location ... and perhaps a lot more. I am pretty sure I could get at least 100 downloads on my own.
Then I would email my students (almost 100 this semester), colleagues, friends, and family. I would ask all of them to visit my SSRN page and download my papers. I am not sure how many would actually do it or how many each person would download, but I am confident that I could achieve many more than the remaining 404 downloads in this way. (I have very nice students, colleagues, friends, and family.)
Obviously, this sort of blitz is not a sustainable strategy for gaming the system, but it illustrates the basic problem: SSRN measures an activity that does not reflect the scholarly merit of the underlying work. A law school that wanted to create a more sustainable strategy for downloading would enlist faculty, students, alumni, and friends of the law school in a patriotic effort: help your school by investing a few minutes online! Among other things, the law school could subscribe all of its faculty and staff and as many students and alumni as they could muster to the law school's research paper series. Then it would get the word out, that if you want to help the law school, you will start downloading. Better yet, how about a "Download-a-Day" email list? Just send that day's link to the listserv and ask recipients to click. It's better than writing a check!
If that doesn't produce enough downloads, we could hire mercenary downloaders. Surely a few students would accept $15 an hour to download papers from their homes. Just have a small army of students taking one spin through the entire list of Wisconsin papers every day for a month and voila! They have earned their law degrees from a Top 10 faculty! Heck, they might do that for free!
Wouldn't it be great if we all started to play this game and diverted even more resources to self-promotion? Goodbye, law porn! Hello, downloads!
TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d8345157d569e200d8345a522769e2
Links to weblogs that reference Gaming SSRN:
11. Posted by Alan Meese on November 11, 2005 @ 21:24 | Permalink
Kate says:
"As a general rule, if _you_ wouldn’t admit to a particular behavior, the prospect of someone actually doing it shouldn’t worry us too much." Really? Don't lots of people --- even academics --- do things that others would not admit doing? Don't some academics shirk in various ways, even though others would not admit doing so. Indeed, don't Law Schools themselves do pretty shameless things when it comes to, for instance, gaming the US NEWs rankings?
Note also that, for people with fewer downloads than Gordon (like me), a 100 or so downloads a month could mean quite a bit. I could, for instance, assign a few of my papers to my antitrust class, distributed via SSRN, and pick up 150 downloads that way alone. I could also assign my corporate law paper to my Corporations class and suggest that my colleagues do the same. Don't some people assign their own texts, even when another text might be better? Yes, they should be ashamed, but "good luck."
Note that it might take a few bad apples to do things like this to undermine confidence in the system. And, some of these strategies at least are pretty secret, with zero chance of detection.
Of course, I share Kate's outrage at people who would do such things. We should just write our stuff, post it, distribute it, and "let the market work." But, everyone is not as virtuous as Kate and Gordon.
12. Posted by Laura Heymann on November 14, 2005 @ 12:37 | Permalink
What leads us to think that the SSRN rankings will be viewed, ultimately, as any different from, say, the New York Times bestseller list? Both reflect the number of times the work has been acquired, although not necessarily read; both send signals about certain gaming attempts (the New York Times by indicating when a ranking is dependent on bulk sales; SSRN by monitoring repeated requests from certain IP addresses); and neither is necessarily a signal as to the quality of the work. I'm quite sure that the Pulitzer committee does not take a book's ranking on the NYT list as guiding its decision in any way; perhaps we could work toward a norm that treats SSRN rankings in the same way, instead of assuming the value of their signal and worrying about who might tinker with the inputs?
| Sun | Mon | Tue | Wed | Thu | Fri | Sat |
|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | |||
| 5 | 6 | 7 | 8 | 9 | 10 | 11 |
| 12 | 13 | 14 | 15 | 16 | 17 | 18 |
| 19 | 20 | 21 | 22 | 23 | 24 | 25 |
| 26 | 27 | 28 | 29 |
