I very much appreciate SSRN. However, the gaming game can get irksome with its competition with other irksome gaming systems. To me, it is a matter of the content of the articles I read on interesting legal subjects that are available more promptly after they are written than law review articles that can be stale by the time of their publications. I would like to furnish comments to the SSRN articles authors from time to time. Some include their Email addresses and my comments can be provided privately. Perhaps for those who do not provide such contact, an SSRN comments section might be appropriate.

As for the gaming, do those interested in reading the articles benefit, or is the gaming for purposes of enhancing careers and reputations of individuals and their institutions? As a reader, I benefit from the content of SSRN articles and applaud their authors for increasing my knowledge. "Let the gaming begin?" Who really cares?

Gordon:

Yes, you can spend a lot of time thinking of ways to create fraudulent downloads – and then argue that downloads should not count towards any measures of scholarly quality because they can be manipulated.

Likewise, you can cook data for your empirical projects – and then argue that empirical studies should not count towards any measures of scholarly quality because it’s so easy to cook data.

You can also threaten a young conference organizer with a bad tenure review to get your submission accepted – and then argue that competitive conferences should not count towards any measures of scholarly quality because it’s easy to force you way into them.

You can create phony survey records. You can lie in the description of your experiments. You can hire some aspiring academic to ghost-write articles for you in exchange for your promise to support his candidacy on the job market. You can organize a “citations exchange list,” where members would pledge to cite each other at least 5 times in every forthcoming article. You can “tailor” your research results to make a career as an expert witness. The possibilities are endless.

Most of these schemes involve a fairly low chance of being caught. But most of us don’t do that sort of thing because we think it’s _shameful_ to defraud our colleagues. We also understand that if we are ever caught plagiarizing, cooking data, or extorting phony co-authorships, we’ll never be taken seriously again. Even if none of these activities is criminal. Even if you can organize your fraud in a way that doesn’t technically violate your school’s rules. Even if you claim that everyone else could do the same thing, and in fact, you’ve heard that a cousin of a friend’s neighbor personally knows someone who’s done it. No matter. Fraud would end your academic career.

The same norms should be (and, I think, increasingly are) applied to electronic means of fraud, like gaming download counts. I think most of us understand that participation in a “download-a-day” email list would be shameful. Hiring someone to download your papers is shameful. Ditto for the rest of the schemes you described. Yes, you can stop at every library to download your own papers. But you better keep this little hobby private or you risk finding yourself without any dinner party invitations – much like you would if you boasted your clever ways of falsifying the Stata output.

People may disagree on how to classify marginally fraudulent download-gaming strategies, much like people disagree on how to classify marginally fraudulent attribution or data-processing practices. But disagreement over marginal behavior hasn’t stopped us from shunning plagiarists and data-cooks; it shouldn’t stop us from shunning download manipulators either.

Sure, it’s possible that someone is defrauding us as we speak. But is it a reason to discard quality signals that could be fraudulently manipulated? I don’t think so.

Kate: "you can spend a lot of time thinking of ways to create fraudulent downloads."

Actually, it didn't take much time at all. About 30 seconds and I had a whole list. One problem is that fraud is so easy here.

Perhaps a bigger problem is that distinguishing between fraud and respectable conduct is hard. Would it be fraud to encourage alumni to keep up with happenings at their alma mater by downloading papers from their old professors? Would it be fraud to assign my papers to my students and ask them to obtain the papers via downloads? Is it fraud to write on a topic of general interest, post it on SSRN, then blog about how everyone should read my paper? Of course not. But then to count all of those downloads and pretend that they say something about scholarly achievement is a joke.

Gordon: It’s interesting that arguments against download-based rankings often start with wild fantasies about bizarre and quite public frauds that some unspecified “others” could conceivably commit. Your original post is a good example of such thought experiments. As a general rule, if _you_ wouldn’t admit to a particular behavior, the prospect of someone actually doing it shouldn’t worry us too much. A “download-a-month” club? Paying your students to download your papers? Sending your RA to download your papers from every public computer in town? You must be kidding.

When pushed, the people who worry about wild frauds usually abandon their doomsday scenarios and just switch to complaining about “grey areas.” But grey areas, again, are not at all unique to download-count measures. They are pervasive in every ranking and evaluation system, and I am not sure that download-count measure is more prone to “grey area” problems than most other measures.

Is placing your name on every paper written by your grad students ok? Who knows... Does this uncertainty mean that counting co-authored publications towards any measure of scholarly quality is, as you put it, “a joke”? No. Is broad circulation of unpublished work, which increases a chance that “blind” peer review won’t really be blind, ok? Who knows... Does this uncertainty mean the use of peer-reviewed publications as a measure of scholarly quality “a joke”? No. Slight tweaking of data… subtle pressure on a referee… soft reminders to cite your papers… I can point to dozens of such grey areas. They don’t turn other measures of quality into “jokes”; nor should they kill download counts as one of many measures.

Generally, it seems that “permissible” behaviors are not too likely to affect download rankings significantly, while impermissible behaviors are not too likely to occur in the first place. E.g., general advertising of your papers on a blog encourages mostly abstract viewing, not downloads – and, frankly, if your abstract persuades someone to download and read your paper, that isn’t so bad, even if affects rankings on the margin. In contrast, a blog entry telling readers “Support your blogger! Download my papers! No need to read!” may generate lots of downloads, but you aren’t likely to ask. Likewise, encouraging alums to keep up with school news by reading papers of their old profs is not likely to generate many downloads – busy practicing lawyers aren’t exactly consumers of our indiosyncratic papers. In contrast, telling alums to “download, no need to read” for the sake of increasing the school’s ranking may generate lots of downloads, but you aren’t likely to ask. Similarly, writing lots of op-ed style papers or general-interest papers may increase your download count, but would you really want to become known as “Gordon who writes a lot of crap and thinks it’s scholarship?”

There seems to be enough resilience in the system to discourage outright fraud and make skirting on the margins not terribly profitable. And the system is getting more sophisticated every day.

Kate, I didn't "abandon [my] doomsday scenarios," (indeed, I wrote, "One problem is that fraud is so easy here"), because I think it is highly likely that people will engage in these or similar strategies if SSRN downloads became an important measure of faculty quality. Most of the strategies could be accomplished beyond public view, and those that are visible can be shrouded in high ideals like "disseminating the work of the institution." The common link is that all of the strategies are about getting people to click through, without regard to whether that click is a meaningful measure of quality.

You belittle law review editors for their lack of rigor in selecting articles, but you defend SSRN downloads as a plausible measure of quality. I am having a hard time reconciling these positions. They are both quality measures of a sort, but you would prefer downloads by who-knows-who to publication choices made by the brightest law students at our best law schools. Admittedly, the law review process has lots of shortcomings, but they pale in comparison to the shortcomings of SSRN downloads.

Gordon, your original list of hypothetical frauds was improbable but funny; now, you are just too vague to understand. Which exactly “beyond-public-view strategies” of defrauding our colleagues do you seriously expect to arise? Who exactly would consciously risk his reputation to gain downloads?

How do you persuade a meaningful number of people to “just download, no need to read” your papers without admitting your fraudulent intentions and risking an embarrassing public exposure? What kind of high-minded language would seriously fool anyone? And if you want to keep your name clean: how do you persuade a meaningful number of people to download (not just to see an abstract, but to download) your papers without first persuading them to read those papers?

I like Gordon’s idea. It would seem to me that much is gained by letting SSRN be exclusively a system to share knowledge. To burden the system with the additional purpose of ranking scholars may diminish this primary goal of the system, which it has already started to achieve so wonderfully. I could see it going to a situation where papers are not allowed to be downloaded unless the downloader show her academic bona fide’s, and to me that would put the purposes that SSRN can serve in exactly the wrong order.

Right, Michael, re thinking it'd be unfortunate to limit SSRN, or SSRN download counts, to academic downloads.

About two dozen of the downloads of my latest paper on SSRN were from practicing employment lawyers. I induced those downloads purposely: I posted about my paper to two listservs of practicing employment lawyers.

But I didn't do this to "game" SSRN; I did it because I'm excited about the possibility that some of my theory-headed writing might actually make a difference in how some folks practice law. I think those are worthy uses of SSRN, and they should "count" in any effort to quantify the impact of my article.

I will respond to the point about reputation because Larry Ribstein has jumped on that bandwagon. I have strong views on the subject of reputational constraints, but I will keep this short. We like to point to reputation as a constraint when we don't have any structural constraints, but in many instances -- and I assert that this is one -- reliance on reputational constraints is little more than wishful thinking.

Two problems here. First, the chances of detection are really low. It is worth remembering that only a few hundred downloads can make a big difference in rankings. Post a paper, get the downloads over a period of a couple of months, and no one is the wiser. Easy.

That's about the "bad guys," but what about the rest of us? The second point is that reputational constraints are weakest where norms are unclear, and Vic's post illustrates the lack of clarity in the norms about download promotion. If you don't believe that counting SSRN downloads will change norms, I encourage to think about how the US News rankings have changed norms. People will push the edge of the envelope on downloads, and what is acceptable self-promotion will change.

With so much riding on perceptions of law school quality, any system that purports to measure quality will be subject to attempts at gaming. I think SSRN has been relatively free from such attempts so far because it has not achieved widespread recognition as a quality measure, but if it becomes that and the only defense against gaming is a reputational constraint, well, let the games begin!

One thing that differentiates gaming SSRN from some of the other types of gaming described above is that with SSRN it is possible for third parties to get in the game, there is an incentive for them to do so without any prodding from profs or schools, and reputational constraints on them are not nearly as strong. Here’s what I mean: it is in the best interests of students and alums to improve the stature of their school. They, therefore, may come to the conclusion on their own that downloading all the papers of all the profs at their school is a cheap, quick and easy way to help improve stature, especially if the importance of SSRN downloads continues to grow and becomes widely known.