This is a special US News rankings guest post. Many thanks to Gordon for the opportunity.
With release of this year's U.S. News & World Report rankings, law professors, administrators, students and alumni are once reflecting upon how our respective law schools can improve our position in this zero sum game. Last year, Andy Morriss and I wrote an article for the Next Generation of Law School Rankings Symposium that examined the market for high LSAT students. (If you want all the Moneyball lessons from that analysis, click here.) The dynamics of academic and lawyer-judge reputation is something we hope to examine in a forthcoming paper. In the meantime, here are two scatterplots that tell an interesting story (and corroborate findings by Dan Filler at Concurring Opinions): [Click on scatterplots to enlarge!]
Here are three observations. I welcome more in the comments.
First, USNWR academic reputation does not change much overtime. Where a school was in 1992, which is the first year that U.S. News published it full rankings, explains approximately 93 percent of its current U.S. News academic reputation. Especially for the top 50 schools, there is funnel effect with progressively smaller variations over time.
The San Diego and George Mason "miracles" can be seen in the top graph (blue and green circles at ~ 65 for 1992 and ~ 50 for 2006), which equals approximately +15 spots (ranking from 1 to 180). Objectively, in terms of faculty productivity, perhaps these schools deserved a larger bump, especially over a 14-year time horizon. (Brian Leiter says as much here.) Yet, the GMU/San Diego miracle probably reflects the outer limit of how well a law school can do (at least in the USNWR rankings) by hiring and retaining talented scholars. Moreover, would the miracle have occurred in less attractive metro areas, in terms of amenities and spousal professional opportunities, than DC or San Diego? I doubt it.
Second, lawyer-judge reputation has a lot more variation but the same familiar funnel effect. This explains why Dan Filler's post summarizing up/down lawyer-judge movements is dominated by lower ranked schools--the errors are not randomly distributed (it is also why the fit line seems off center). However, one large winner is George Mason (+17 between 1992 and 2006 in overall reputational ranking), which is high for a school that started in the Second Quartile in 1992. Is that bump attributable to scholarship, or the Law & Economics conferences that Henry Manne (former dean of GMU) sponsored for judges during the early to mid-1990s? Perhaps the Lawyer-Judge reputation is just a noisy variable. On the other hand, because few law schools have strategic plans that attempt to reach judges and lawyers (beyond alumni), there might be quite a bit of opportunity for movement.
A third point relates to the "positional competition" that Andy Morriss and I discussed in our LSAT paper. Why does every law school strategic plan, formed in the crucible of USNWR rankings angst, emphasize a plan of more and better scholarship when, empirically, such a strategy is unlikely to produce substantial improvements relative to peer schools? Scholarship is a highly portable asset. Highly productive scholars tend to have tremendous upward mobility. When all schools compete on the basis of scholarship, it drives up costs with no clear social benefit to students, institutions, or society.
Moreover, as Andy and I discuss, this system is not sustainable over the long-run. Outside the first tier, our regression results clearly show that students with marginally higher LSAT scores routinely trade down in USNWR ranking for (a) lower tuition/debt or (b) proximity to a vibrant corporate legal market. In other words, the choices of students (i.e., the market) are remarkably rational--why go $10K more into debt to move up ten spots in the second tier, especially if the higher ranked school is in a weak or dying legal market? In this environment, a scholarship-first strategy can really backfire for many law schools that are not already highly ranked.
TrackBack URL for this entry:
Links to weblogs that reference Variation in US News Reputation Over Time:
1. Posted by Gordon Smith on April 4, 2006 @ 20:09 | Permalink
Thanks, Bill. This is very interesting.
This is a naive question, but why are the lawyer-judge reputation numbers noisier than academic reputation numbers? Dan had some ideas about this, but I wonder if you have thoughts on that.
2. Posted by Cynic on April 4, 2006 @ 20:42 | Permalink
Maybe scholarship is cheaper? Lowering prices enough to matter to the LSAT numbers is much more expensive -- maybe beyond the means of most law schools?
3. Posted by William Henderson on April 4, 2006 @ 21:17 | Permalink
The question is far from naive.
One theory is academics care more about rankings that judge and practicing lawyers. The prestige of our law schools (or the law school that might make us a lateral offer) is a non-pecuniary form of compensation. Hence, we actually pay attention to USNWR rankings and know our relative "neighborhood"; indeed, USNWR affects how many law professors expedite their law review articles. Thus, the overall score for 2006 will affects the 2007 academic reputation score. My colleague, Jeff Stake, did a nice job of empirically demonstrating this "echo effect" in his ranking article from last year's symposium. Another paper by Sauder and Lancaster (both sociologists) in Law & Society Review demonstrates a similar reflectivity dynamic among students. (Abstract here.)
In short, a typical law professor probably has a MUCH better knowledge of the law school hierarchy (i.e., he or she has heard of the school) than the typical judge or practitioner.
By that same token, GMU's appearance in the Final Four may produce a bump in next year's Lawyer-Judge ranking. And schools with big football programs probably do slightly better, all else equal--seriously!
Re cynic's comment that pursuing more and better scholarship may be cheaper than lower tuition, law schools' largest expense is faculty payroll. And the clear trend is toward lighter teaching loads. Some schools at the lower end of the hierarchy may fill teaching slots with heavier teaching loads and minimal publishing expectations. In that environment, salaries won't be bid up by lateral offers. While I think that the lack of scholarly engagement would be bad for some students, I am not sure if the marginal benefits of the current system outweigh the marginal costs, especially for students at nonprestigious schools (where that line gets drawn is another matter). The universe of great teachers is probably much larger than the universe of great scholars, and the two don't always overlap.
4. Posted by Jeff Lipshaw on April 5, 2006 @ 4:09 | Permalink
I think Bill is right about the fine tuning of rankings meaning more to academics. Two anecdotal observations. First, when I was a law firm partner, partner compensation was completely public as among the partners. We all made respectable incomes, but the philosophy of the compensation committee was to make very small distinctions that were far more important psychologically and emotionally than economically. That was really the only way we had of comparing ourselves (now you can ask why did we care so much about comparing ourselves...)
Second, I am right now on the classic "nine schools in nine days" tour of the Northeast with my son who is a junior in high school. Yesterday our tour guide at one of the schools told us she was going to law school, had gotten into Chicago, Georgetown, Boalt, Penn, Columbia and NYU, and had gotten deferred at Harvard and Stanford. She wanted my advice on which one. Of course, the first thing I did was to congratulate her on being in a situation in which it would be difficult to make a bad choice. But what really is important to a student? I suspect at that level it is not the fine tuning of the rankings, as opposed to the perception of tiers.
5. Posted by Scott Moss on April 5, 2006 @ 7:59 | Permalink
I can think of two explanations (in addition to Bill's, which make perfect sense too) of why there is more variabilty in lawyer/judge than academic rankings.
First, it's possible that the lawyer/judge rankings are just unreliable, featuring too much variance to mean much of anything. Most lawyers, I would guess, know zero about the vast majority of law schools; back when I was in practice, I hadn't heard of the vast majority of schools -- even though I was thinking of going on the law teaching market someday. If my ignorance is typical, and if academics tend to know more about schools (at least their US News rank!), then the lawyer/judge ranks will feature a lot of randomness, i.e., did U.S. News happen to sample a few of the 10% of non-Wisconsin lawyers who'd heard of (for example) Marquette?
Second, at least for schools outside the top 10-20, the vast majority of graduates traditionally stay in the school's geographic region. My sense is that in the past decade or so, many non-elite law schools (including Marqutte) have drawn a substantially more "national" applicant pool, and relatedly have produced graduates who move to different parts of the country. When 99% of Marquette's graduates practiced law in Wisconsin, our "reputation" in other states was probably almost nonexistent. Now, there may be some such reputation. Further, non-elite law schools may succeed in this "nationalization" to a different degress -- which would explain the variance we've seen in lawyer/judge scores.
If my second explanation is valid, then the variation in lawyer/judge scores in the past 10+ years may reflect only a one-time shift caused by the (at least partial) "nationalization" of non-elite law schools. That nationalization won't occur again, so it's possible that lawyer/judge rankings will be more stable from this point forward.
6. Posted by Jeff Yates on April 5, 2006 @ 8:17 | Permalink
Let's hope that William Henderson's assertion (below) regarding a link between sports ranking and academic reputation (at least among judges and lawyers) doesn't get much press. Evidence to support students' frequent arguments that football "pays my salary" is the last thing that I need. ;-)
"By that same token, GMU's appearance in the Final Four may produce a bump in next year's Lawyer-Judge ranking. And schools with big football programs probably do slightly better, all else equal--seriously!"
Jeff Yates - J.D., Ph.D.
Department of Political Science
University of Georgia
7. Posted by Jared on April 5, 2006 @ 10:20 | Permalink
The practitioner rankings may be meaningless in terms of differentiating between schools on quality, but many students plan to seek employment among these practitioners. Therefore, while a school with a good football team may not be any better of a school than one without, if your future employer has heard of the former and not of the latter, then it becomes a "better school" in the sense of helping you acquire the career you desire.
8. Posted by John Steele on April 5, 2006 @ 10:25 | Permalink
As always, good data, usefully organized, and some provocative questions.
1. As for lawyer reputational scoring, I think you're right. As a former hiring partner who also teaches, I've seen that practitioner knowledge often trails by a decade or more, especially with the older practitioners who are influential in the field.
Here's an analogous point that supports the idea that lawyers and judges aren't keeping current. One of my students wrote a paper that surveyed the views of practitioners, judges, and academics on what it meant to be on law review and how students got onto law review. The typical practitioners' and judges' responses were factually correct for about the year 1980. The academics knew the current answers.
2. You ask, "Why does every law school strategic plan, formed in the crucible of USNWR rankings angst, emphasize a plan of more and better scholarship when, empirically, such a strategy is unlikely to produce substantial improvements relative to peer schools? Scholarship is a highly portable asset. Highly productive scholars tend to have tremendous upward mobility. When all schools compete on the basis of scholarship, it drives up costs with no clear social benefit to students, institutions, or society."
I can't help but think of an analogy to biglaw lawyers: why is the solution to so many problems "higher PPPs"? (The old chestnut says that to a hammer every problem looks like a nail.) In other words, supporting more scholarship is supporting the means of personal advancement for each professor. As you note, however, the successful scholars can and typically do move up the rankings chart and take their citation counts with them.
As you note, one strategy for moving up in rankings is to "purchase" higher LSAT students, and just assume (correctly I think) that over time the better scholars will gravitate toward the school as it moves up the ranks.
One other approach, which is probably just a thought experiment, is to think about a state school that offered an undergraduate law degree that qualified the student for sitting for the bar exam. Such a school could attract top students from that state who wanted to leave college with close to zero debt and a professional degree. The school might suddenly find its law school full of the "best and brightest."
9. Posted by William Henderson on April 5, 2006 @ 12:50 | Permalink
As usual, I am outdone by the comments. I think Scott Moss may be on to something re the nationalizing of law school competition, which I think is product of USNWR. In an environment where information costs have plummeted, first by USNWR and then by the Internet, I think students are casting broader nets. For example, the number of applications per applicant is up 1.5 apps over the last decade. Also, at least at IU Law, the proportion of out-of-state applications continues to grow year after year.
Re John Steele's point on buying high LSAT students, that is definitely going on. And insofar as it increases USNWR rankings, the academic reputations variables goes up as well. My colleague Jeff Stake has documented this dynamic. And boy, it would be interesting to see how the market would react to a high quality LLB degree, especially in a place like California.
I agree with Jared that all else equal, it is better to attend a law school affiliated with a great football program.
10. Posted by Royce Barondes on April 6, 2006 @ 18:52 | Permalink
It would appear that you have significant heteroskedasticity here. It would seem that the old scores are highly predictive in Tier 1 schools and not nearly as predictive in other schools. Perhaps this is all refined statistically in your paper.