I am at the SEALS annual conference and getting a chance to see some interesting panels as well as workshops for new law professors. One panel I attended focused on new developments in empirical legal research. Although the work in which people were engaged sounded interesting, each panelist asked the question, should young scholars engage in such research? The answer appeared to be no, with some qualifications. There were essentially four reasons why people responded no to the query.
First, the research takes a long time, too long for people on a tenure clock.
Second, data that is not public is extremely hard to get, feeding into the first problem and potentially undermining the saliency of the study.
Third, the finished written product is generally not that long and possibly too scientific for traditional scholarship, making it difficult to place the article in a traditional law review, and hence potentially undercutting the weight given to the article during tenure review.
Fourth, for purposes of external reviews associated with tenure and promotion, it is difficult to find outside people who can evaluate the work. And apparently if you find someone with a social science background who understands how to conduct empirical research, there is the possibility that she will be overly critical if the law professor fails to appropriately defend her methodology for the study.
Alas, there were some who encouraged young scholars to pursue empirical research saying that it was rewarding, that young scholars could develop a theory piece first so that they were not relying solely on their empirical article for tenure purposes, and that young scholars can team up with people from other disciplines to help with the research and writing. Of course this last point raises the concern that during the tenure process some may discount the work because they could not discern which portion of it should be attributed to the young scholars.
Overall, there was a strong and I think familiar undercurrent against taking on this kind of research as a young scholar.
TrackBack URL for this entry:
Links to weblogs that reference Should Young Scholars Engage in Empirical Legal Research?:
1. Posted by Gordon Smith on July 18, 2006 @ 10:55 | Permalink
This is an important issue, Lisa, and I am glad that you raised it. If young professors are trained in empirical methods, there is no reason they should refrain from using them. It is, of course, the norm for untenured professors in other departments to gather and analyze data. The problem lies in getting the training while the tenure clock is ticking. Not everyone can pull that off.
2. Posted by Michael Heise on July 18, 2006 @ 12:41 | Permalink
I agree with Gordon, especially about the issue's importance for younger (read: untenured) law faculty. I would only add that emerging law faculty hiring and legal scholarship trends suggests that, at some point, this issue's salience might fade over time.
3. Posted by William Henderson on July 18, 2006 @ 13:00 | Permalink
As an untenured person doing (pretty much exclusively now) empirical work, I would like to offer a little push back.
In the fall of 2001, as I job-talked my first empirical article, I remember hearing the caveat on pre-tenure empirical work from the chair an appointments committee. But two years later, an IU colleague told me, "In this business, we spend a lot of time doing scholarship. You've got to follow your star."
So which advice is better? In my case, I could not follow both. So I picked empiricism because I loved the work.
Granted, I have spent a huge amount of time (and money) learning about statistical methodology, but always in the process of moving an empirical project forward. Further, I would caution against generalities on data collection and the amount of time it takes to write an empirical article. I have often been dumbfounded by the quality of data that can be collected by a little research and 60 to 120 hour of RA time. I have enough original data to write for years, and more and better data in the pipeline. Moreover, there are lots of datasets that are public and underutilized, with lots of still-unexplored research questions. People wholly committed to empirical research don't see nearly as many obstacles as non-empiricists.
Sure, two years from now I may not get tenure (I remain optimistic!), but intellectually, this has been the best five years of my life. Why give in to fear? Follow that damn star.
4. Posted by anon on July 18, 2006 @ 13:33 | Permalink
Some comments from someone who participates in both disciplines. First, you seem to distinguish "empirical" pieces and "theory" pieces. This strikes me as a horrible dichotomy. Quantitative work in the social sciences is not atheoretical (I know you didn't say that); theory motivates any good piece. What social science pieces tend to lack in comparison to law review pieces are (a) fluff and (b) the relative absence of passing off ideas as "original" when in fact they're clear derivations or one step removed copying from another piece or line of work.
On to your points:
"First, the research takes a long time, too long for people on a tenure clock."
This is an odd claim. Social science folks are on a tenure clock too, and many produce quite a lot of sound empirical work. Sure, it's relatively easy to fire off a good law review piece; if you mean that doing good social science work takes more effort than doing what passes for good (not all, certainly) law review type work, then you're probably right.
"Second, data that is not public is extremely hard to get, feeding into the first problem and potentially undermining the saliency of the study."
Data is not that hard to collect, generally, if you have focused research questions. And sure there will be tough to collect data, but if the opportunity cost is too high, move on to another project -- there's plenty of them.
"Third, the finished written product is generally not that long and possibly too scientific for traditional scholarship, making it difficult to place the article in a traditional law review, and hence potentially undercutting the weight given to the article during tenure review."
This may be true, though, of course, the "not too long" is generally a substitute for the fluff that gets thrown in most law review articles. And with the move to shorter articles at the top reviews and competition from outlets like JLE, JLS, JELS, I suspect this either (a) no longer holds much, or (b) is on its way out as a norm.
"Fourth, for purposes of external reviews associated with tenure and promotion, it is difficult to find outside people who can evaluate the work. And apparently if you find someone with a social science background who understands how to conduct empirical research, there is the possibility that she will be overly critical if the law professor fails to appropriately defend her methodology for the study."
Law professors aren't often punished for shoddy work. If by "overly critical" you mean you will get called out on serious design flaws, model misspecification, or other serious errors, they you're probably right. But avoiding empirical work for this reason is hardly justified -- do good work, think about the methodological issues, and be prepared to justify specifications, assumptions, and the like.
5. Posted by Mike Guttentag on July 18, 2006 @ 14:13 | Permalink
This is such an interesting debate. It seems to me the claims on both sides are correct. If you have a passion for empirical work, you should do it. The challenges are surmountable, the rewards are substantial, and, in fact, almost any good legal research should have an empirical component to it. That said, Lisa’s advice is probably correct as well. Doing empirical work pre-tenure is probably a risky strategy, particularly if you are not already fairly adept at using the machinery of empirical research. Conclusion: the problem is ourselves and our tenure processes.
6. Posted by Jeremy A. Blumenthal on July 18, 2006 @ 14:50 | Permalink
I would echo Bill H.'s and "Anon's" comments, especially disagreeing with at least the first 3 of the reasons against empirical research by junior people. The thinking may reflect a general - false - perception in the legal academy that "empirical" means "econometric" and "political science" research. A great deal of relevant empirical research is in other social science fields and is experimental (mock jury research as just one example). Taking that approach can give interesting results and, IMHO, belies these 3 reasons.
1. "takes too long" - I'm impressed by the activity in many psychology departments and by those doing "psych & law" research, running and submitting more experimental work in a year or two than many law profs publish before tenure. This ties into
2. "availability of data" - as "anon" suggests, collecting original data is common in law-related contexts. A number of examples at the recent Law & Society Association conference; a vast number at the annual American Psychology/Law Society conference and in primary empirical "law-and-" journals. Certainly the traditional undergraduate subject population is easily available, but many social scientists doing experimental work are using samples from the community, i.e., jury-eligible people for mock jury studies.
3. Article length might be an issue, I suppose, despite the trend nowadays for journals to profess that they want shorter articles. But I know Cardozo, and some time ago the Penn Journal of Constitutional Law, have both stated an interest in empirical pieces, and of course more and more is appearing in law reviews. And I think distilling empirical work - "scientific" work - into comprehensible, legally relevant writing is a great opportunity for clear writing.
4. The external review issue I think is more salient, though increasingly, there are more people available and knowledgeable enough to review empirical work.
So I'm in favor! (Full disclosure, I do this kind of stuff.)
7. Posted by Frank Cross on July 18, 2006 @ 14:56 | Permalink
To amplify on the comments a little
The tenure standards at law schools are clearly less than in other more empirical fields, but those other areas provide more support, i.e. students, for the research. I would think the extent of law school support, such as through RAs, would be relevant to this decision.
There are many datasets publicly available for research on various topics, so the data collection problem can be exaggerated.
There's no intrinsic reason why it should take more time. The problem I see is the learning curve for law professors who are not yet empirically trained. Even in this csae, though, descriptive statistics are pretty easy and can represent a valuable empirical contribution.
I don't think there is any shortage of outside people to evaluate this research any more.
The "shorter article" issue is in part an artifact of the requirements of particular publication outlets. Empirical articles in law reviews are not generally so short. See Heise, Morriss & Sisk in NYU LR or Brudney & Ditslear in Vanderbilt LR as examples. Moreover, this problem should be dispelled by the recent law review preferences for shorter articles.
8. Posted by Paul Gowder on July 18, 2006 @ 15:29 | Permalink
Good heavens, that's such an indictment of law schools! While I have my skepticism about "Emprical Legal Studies" as a movement, the notion that law faculties (and of course the students who edit the law reviews, godloveem) are unable to find anyone to review the work who understands it except Mean Evil Social Science Researchers who ask the poor beleagured law professor to -- gasp -- justify the methodology they use and thus ensure they will never get tenure is just horrrifying. If it really is that bad, the legal academy needs to undergo some serious reform.
9. Posted by Jason Czarnezki on July 18, 2006 @ 15:40 | Permalink
I agree with most of the "push back" offered by Henderson, anon, Blumenthal, and Cross. I would like to add that perhaps "Should Young Scholars Engage in Empirical Legal Research?" is not the correct question to be asking. I often hear from fellow junior faculty that they "want" to engage in empirical work, but are concerned about the perceived potential costs (time, tenure, etc.). While I agree with Frank that start up costs to becoming empirically trained are potentially high, I also think that if a junior scholar wants to engage in empirical work or is already trained, the proper question might be: "If we think empirical scholarship adds value, what can institutions do to encourage young scholars to engage in empirical legal research?" The potential answers to this question (e.g., available RAs, on-site statistician/methodologist, funding for ISCPSR, ecourage co-authored work) would encourage empirical work from both junior and senior faculty.
10. Posted by Jason Czarnezki on July 18, 2006 @ 15:56 | Permalink
Since I think it might be a more fruitful endeavor to discuss how junior scholars might reduce the roadblocks to empirical work, I thought I'd note that the Empirical Legal Studies Blog (www.elsblog.org) blog has offered suggestions before: