Precedent and Discretion

This blog has been quiet, at least for my part, because some of my core job duties like grading and article-writing had to take precedence over blogging. But with some deadlines behind me, I now have time to share a short paper for the Supreme Court Review I recently posted on SSRN.

The paper is called Precedent and Discretion, and addresses some emerging theories of precedent on the Supreme Court. Here’s the abstract:

Supreme Court precedent is a topic of perennial prominence. The Court overruled or severely limited multiple precedents last year, just as it did the year before that. Because of our widely-repeated norm of stare decisis, every overruling is criticized. Scholars have then debated whether the Court needs a stronger norm of stare decisis, so that it overrules fewer cases.

This focus is misguided. Rather than worrying about which cases will be cast aside, we should pay more attention to those precedents that are left standing in place. Many of the Court’s questionable precedents nonetheless go unquestioned. The real problem is not that the Court overrules too much, but that it overrules without a theory that explains why it overrules so little.

At last, it seems such theories may be coming. Last term, Justice Thomas (in Gamble v United States) and Justice Alito (in Gundy v United States) each attempted to explain some of their decisions to reject and adhere to precedent. These explanations deserve serious scholarly scrutiny, which they have not yet received.

Unfortunately, these interventions do not solve, and indeed they exacerbate, the problem. What they propose is neither a regime of adherence to precedent, nor a regime without precedent, but rather a regime in which individual Justices have substantial discretion whether to adhere to precedent or not. This turns precedent from a tool to constrain discretion into a tool to expand discretion, and ultimately into a tool to evade more fundamental legal principles.

Part I describes the state of stare decisis in the Court today. Part II discusses Justice Thomas’s theory that precedent must be overruled when it is “demonstrably erroneous.” Part III describes Justice Alito’s theory that precedents ought not be overruled on the basis of “halfway originalism.” Part IV explains why discretionary precedent—of which these theories are examples—are worse than no precedent at all.

Things We Liked This Week

1. This new paper by Julian Mortensen and Nick Bagley purporting to refute the non-delegation doctrine, and this response by Ilan Wurman.

2. The resignation of the President of the Romance Writers of America. (It’s a long story, but you can try to get caught up with this story or by scrolling Courtney Milan’s Twitter feed.)

3. The new LaMP Blog. LaMP is new organization started by leading migration experts dedicated to the idea that “labor mobility that allows workers from poorer countries to access job opportunities in rich countries has the potential of creating billions in income gains to workers, sending countries and receiving countries.” The blog’s first post by Lant Pritchett details the truly alarming trend of dramatically increasing retirement age populations relative to working age populations in all the world’s advanced economies. The ratios are already unprecedented, and are going to get much worse.

4. This essay by Tess Wilkinson-Ryan on “visits” in law school hiring. (We might have more on this later.)

Research as Consumption

The only thing bad about Will’s recent post, Unhelpful Tips for Junior Scholars, is the title. The advice is all helpful. And in addition to giving a lot of great tips that junior scholars should follow, Will also hit on my main piece of advice for people deciding which junior scholars to hire: focus on people that view research as consumption.

Let me explain why this so important. The mission of academic departments is the production and dissemination of knowledge. Even after tenure, you can force people to disseminate knowledge (e.g. teach). But it’s hard to force someone to produce it (e.g. research). This problem is true in all academic departments, but it’s exacerbated by several structural features of law schools: tenure clocks are short and denial rates are low; salary ranges are compressed; and there are non-research, legal related opportunities for faculty to spend their time on that will provide them with money, prestige, and stimulation.

So although it’s critical to hire people that are going to continue to be active researchers and help produce knowledge, it’s hard to compel them to do it for very long. The solution is to focus on hiring scholars that love research and would be doing it regardless of the professional rewards. The kind of people that love nothing more than a day alone in their office, that fantasize about starting new projects, and that love learning about other new research because they find it interesting. These are the people that are worth betting on as being future great scholars when making massive long-run investments in their hiring.

But identifying people that love research isn’t just important because its a good characteristic to focus on when predicting that individual’s own future scholarly output. Having people that love research around is massively important for peer effects. I’m pretty sure there is research on this point, but it’s definitely been my experience anecdotally.

When someone is an active researcher, there are things they’re learning about that few other people in the world know. So, over a meal, I can almost always learn something novel from someone that has a lot of active research interests. Often, the things I learn make me want to start new projects or help me improve current ones. And being around great, productive researchers makes me think I need to push harder to produce good scholarship.

But when I’m at a meal with an academic that is not actively engrossed by the researcher process, I find that that dinner conversations slide toward whatever is currently being covered on the landing page of the NY Times or the TV that people are watching. This might make for a fun evening, but it never does much to improve my projects. And it makes me think I should read more news and watch more TV.

Unhelpful Tips for Junior Scholars

On Twitter, somebody asked for my tips for young scholars. There’s no particular reason to think my tips are reliable (google “survivorship bias”) and it may not be possible to implement these, but these really are what I think are the most important.

1. Have lots of ideas. Many of them can be bad, probably most (see #3 below), but you need these or this whole thing is going to be miserable. Which brings us to ..

2. Enjoy writing about them. Some people find working on research independently enjoyable and fun; others regard it as work that is necessary to get other things they regard as enjoyable and fun (like paychecks or promotions or power). You will be much more successful if you are in the first category, which Adam likes to call “research as consumption.”

3. Fail fast. You need to start working on ideas to see if they are good ones, but you also need to abandon them if they are not. You’ll be a better scholar if you can get to the question whether to abandon them faster, rather than spending a year on a bad project, thus either wasting a lot of time or forcing you to delude yourself and deceive your readers into pretending it’s a good project.

4. Have faith in your good ideas. A corollary of #3. Once you’ve decided an idea is worth pursuing, don’t try to dress it up as some other idea that you think is more important or more marketable or more interesting. Les Green put this point very well in a discussion of “bullshit” dissertation titles, such as (the made-up example): Agency, Structure, and Power: The Milk-Marketing Board of Ruritania, 2007-2009:

   “Never allow doctoral students to use subtitles. Either there is good reason to study three years of decisions of the Milk-Marketing Board or there isn’t. . . . If there is, they should have the courage of their convictions and make the subject their title. If there isn’t, do not allow them to waste their intellectual careers on trivia and then package it up in a bullshit title.”

5. Say no. You shouldn’t say no to everything. You want to engage with others, both for their benefit and for yours. But if your research is successful you will have more invitations than you can accept. And time is scarce, so you can’t afford to give all of yours away. To implement this rule, I highly recommend the wise advice of Sarah Lawsky to have a “no buddy,” a trusted professional friend who reviews all of your new commitments before you accept them, and who has the power and duty to say “no” most of the time. Which brings us to….

6. Have friends you can trust. You can have ideas and enjoy writing about them on your own. But to decide which ideas to abandon and which requests for your time you can meet, you need advice, and the best advice comes from people who know you well and whose judgment you can trust. You want to share drafts or sketches or ideas early enough that it’s not too late to abandon them. You want people who know what you are good at and not so good at. In other words, you need friends.

I call this advice unhelpful because I don’t know what to say about how to do some of these things if you don’t already. But I think they are what successful junior scholars truly need.

This Year's Writing, Adam Edition

Like Will, I also thought this year got away from me. But every year gets away from me. (Overcommitting and then feeling constantly behind is my number one productivity tip.) Also like Will, I published four papers during 2019. But a difference is that all four of my published papers were co-authored empirical articles in peer-reviewed law and economics journals.

The first paper explored the influence that law clerks have on voting at the Supreme Court. In Legal Rasputins? Law Clerk Influence on Voting at the U.S. Supreme Court (Journal of Law, Economics, & Organization), we collected data on the identities and political ideologies of law clerks going back to 1960.[1] We then were able to use the timing of the Supreme Court clerk hiring process to identify that clerks exert modest effects on the votes of the judges’ they work for. But we found that the clerks exert substantial influence in cases that are high-profile, legally significant, or close decisions.

The other three papers all were on comparative antitrust law, and leveraged a series of major datasets that Anu Bradford and I assembled over the last five years or so. We also launched a new website to host the data and associated research. It’s got a lot of great new data for download, so check it out at http://comparativecompetitionlaw.org/.

In Competition Gone Global: Introducing the Comparative Competition Law and Enforcement Datasets (Journal of Empirical Legal Studies), we introduce the two major datasets that we assembled for this project.[2] One of the datasets codes the contents of 700 competition laws adopted by 130 countries and regional organizations since the beginning of competition law, and the other dataset provides information on the resources and enforcement activities of 112 antitrust agencies dating back to 1990. We use the data to illustrate some previously under explored trends in antitrust law, like how Russia has used competition law in a fundamentally different way than other countries.

In Trade Openness and Antitrust Law (Journal of Law and Economics), we use our data to explore a long-running debate in economics.[3] We spent years on this paper, and wanting to write it was one of the reasons we decided to spend years collecting antitrust data. So I’ll just share the whole abstract:

Openness to international trade and adoption of antitrust laws can both curb anticompetitive behavior. But scholars have long debated the relationship between the two. Some argue that greater trade openness makes antitrust unnecessary, while others contend that antitrust laws are still needed to realize the benefits of trade liberalization. Data limitations have made this debate largely theoretical to date. We study the relationship between trade and antitrust empirically using new data on antitrust laws and enforcement activities. We find that trade openness and stringency of antitrust laws are positively correlated from 1950 to 2010 overall, but the positive correlation disappears in the early 1990s as a large number of new countries adopt antitrust laws. However, we find a positive correlation between trade openness and antitrust enforcement resources and activities for both early and late adopters of antitrust regimes during this period.

In The Global Dominance of European Competition Law Over American Antitrust Law (Journal of Empirical Legal Studies), we looked at the diffusion of specific antitrust rules around the world during the post-war period.[4] We show that countries used to follow the American model when they adopted antirust now, but that now countries overwhelmingly follow the European model.

I also published a few short comments and op-eds, and spent a lot of time on projects that I’ll be blogging about in the new year.

———————-

[1] This paper was co-authored with Adam Bonica, Jacob Goldin, Kyle Rozema, and Maya Sen.

[2] This paper was co-authored with Anu Bradford, Christopher Megaw, and Nathaniel Sokol.

[3] This paper was co-authored with Anu Bradford.

[4] This paper was co-authored with Anu Bradford, Katerina Linos, and Alex Weaver.

This Year's Writings

I confess that this year felt like it got away from me, but looking back I am surprised to discover that I published four articles in 2019.

Two of them were pieces on originalist theory co-authored with Steve Sachs: Grounding Originalism, in the Originalism 3.0 edition of the Northwestern Law Review, and Originalism and the Law of the Past in the special originalism issue of the peer-reviewed Law and History Review.

I also published two solo-authored pieces. I recently blogged about the most recent one, The Unconstitutionality of Hugo Black (Texas Law Review).

The other, which I published at the start of the year was an article in the Stanford Law Review called Constitutional Liquidation. Liquidation is something I’ve been working on since I before I officially became a law professor — it’s an attempt to reconstruct a profound aspect of James Madison’s theory of constitutional law, as well as to provide a theory of constitutional precedent: that the practice of the government (not necessarily the courts) can “liquidate” the meaning of ambiguous parts of the Constitution when it is sufficiently deliberate, widespread, and settled. Here’s the abstract:

James Madison wrote that the Constitution’s meaning could be “liquidated” and settled by practice. But the term “liquidation” is not widely known, and its precise meaning is not understood. This Article attempts to rediscover the concept of constitutional liquidation, and thereby provide a way to ground and understand the role of historical practice in constitutional law.
Constitutional liquidation had three key elements. First, there had to be a textual indeterminacy. Clear provisions could not be liquidated, because practice could “expound” the Constitution, but could not “alter” it. Second, there had to be a course of deliberate practice. This required repeated decisions that reflected constitutional reasoning. Third, that course of practice had to result in a constitutional settlement. This settlement was marked by two related ideas: acquiescence by the dissenting side, and “the public sanction” – a real or imputed popular ratification.
While this Article does not provide a full account of liquidation’s legal status at or after the Founding, liquidation is deeply connected to shared constitutional values. It provides a structured way for understanding the practice of departmentalism. It is analogous to Founding-era precedent, and could provide a salutary improvement over the modern doctrine of stare decisis. It is consistent with the core arguments for adhering to tradition. And it is less susceptible to some of the key criticisms against the more capacious use of historical practice.

Besides these articles, I also finally published a short chapter, The Court or the Constitution? in a festschrift for the great Professor Larry Alexander. It’s a short piece, but I’m especially proud of the fact that the chapter merited this comment in James Allan’s review of the festschrift:

As for Baude, let me just say that it is very seldom indeed that I read someone get the better of an argument with Larry Alexander. In my view Baude does just that in his chapter, even after considering Alexander’s reply. Both are a treat, but Baude’s claim that Alexander cannot have both the cake of judicial supremacy while also eating the truth of originalism, convinced me, and I recommend the exchange to all readers.

Finally, I also co-authored two amicus briefs to the Supreme Court, arguing that they should grant certiorari to decide whether to reconsider the doctrine of qualified immunity. (Something I published an article about last year.) One, in a case called Doe v. Woodard, was denied this summer. The other, in a case called Baxter v. Bracey, has been repeatedly rescheduled and is still waiting for the Court to decide whether to grant it.

Things We Liked This Week

1. Longform Best of 2019. It’s a fantastic list of long form magazine articles from the past year that are perfect for reading over winter break.

2. The Spy. A six-part mini-series on Netflix staring Sacha Baron Cohen about the life of Israeli spy Eli Cohen.

3. Agnes Callard on recommendation letters.

4. Tropical islands. Posting will be relatively light between now and the start of the quarter, though we have a couple things in the queue.

Published: The Unconstitutionality of Justice Black

My latest article was just published in the Texas Law Review, and it is called “The Unconstitutionality of Justice Black.” I originally gave it the accurate but completely uninteresting title “Ex Parte Levitt,” the name of the too-widely-forgotten case that inspired the article.

The article is about the constitutional controversy over the appointment of Justice Black. The day that Black was sworn in to the Supreme Court in 1937, an apparent crank tried to orally argue that Black was an unconstitutional usurper. The Court dismissed the case on procedural grounds.

But it turns out that the crank was correct, and might not really have been a crank. Justice Black was unconstitutionally appointed, and while the suit might have had some procedural problems, they weren’t exactly the problems that the Court thought they were.

The piece also discusses the aftermath of the litigation. As you may know, Justice Black sat on the bench for many decades. But during all that time, the Court never actually ruled on the lawfulness of Justice Black’s appointment. Instead, after a while everybody just took it for granted anyway.

As I’ve blogged before elsewhere, I’m generally a fan of Justice Black’s work, so I feel a little sheepish about publishing the piece. But I’ve become convinced that his appointment was unconstitutional. You can read the whole thing (only 30 pages) if you want to see why.

Things We Liked This Week

1. Five Minute Fifth, a new newsletter about the Fifth Circuit from Raffi Melkonian.

2. The two “emergency” podcasts that Trade Talks (@trade__talks) put up over the weekend to explain the announced US-China Phase One deal and the deal House Democrats struck with Trump Administration to revise the USMCA.

3. In Service of the Republic: The Art and Science of Economic Policy, a new book (primarily focused on India) arguing that countries with low state capacity should be “libertarian by necessity” and only focus on fulfilling functions it is absolutely essential the government does.

4. How deep, audacious 3-pointers are taking over the NBA.

Craft Beer and Law Blogging

Last year, Daniel Hemel and I co-taught a seminar on the “Law and Economics of Craft Beer.” The class was as amazing as it sounds. We drank great beer, met with people working in the industry, and learned about the bonkers three-tier system that’s been used to regulate alcohol distribution in the United States since prohibition. (We thought we were innovators starting this class; it turns out there was already a text book and blog about the law of craft beer.)

During the seminar sessions, multiple brewers told our class that a common mistake in their industry is confusing initial surges in demand for signs of a trend. When a new beer hits the market for the first time, hipsters — and law professors masquerading as hipsters — are excited to try it. This initial surge in demand can exhaust the initial supply, which makes getting to try the new beer even more exciting.

When this happens, many small breweries have made the mistake of making capital intensive investments to expand their capacity shortly after they’ve opened. But by the time the brewery has the capacity to meet a large demand, the need for the excess capacity evaporates because there is a new exciting beer to try.

Since we launched this blog one month ago, we’ve gotten 7,500 unique visitors. I’ve never had a blog before, so I’m not sure if that’s a lot or a little. And I’m also not sure if that’s all just an initial surge. But I am excited that people have been reading, and I’ll try to talk Will out of his plans to buy bigger fermentation tanks.

Reforming the Academic Publication Process Should be a First Order Priority

Anup Malani and I hosted a conference last week in New Delhi on finding ways to “Improve the Lives of India’s Urban Poor.” One of the presenters at the conference, Neelanjan Sircar, motivated his presentation with the above slide illustrating India’s disturbingly low female labor force participation (FLFP). As India has gotten richer, women have dropped out of the labor force at a rate higher than other South Asian countries. Neelanjan persuasively argued that understanding the drivers of this trend has massive welfare consequences, and given the size of India, is a question of first order importance.

Of course, a conference on urban poverty in India, FLFP isn’t the only problem of first order importance that came up. After all, India is facing dire consequences caused by climate change; there are frequent cases of violence against women and religious minorities; and there are still hundreds of millions of people in extreme poverty.

But here’s the most persuasive argument I heard last week about what problem should be the highest priority for researchers to tackle: making the academic publication process more efficient. Anup was the person that made this argument, and he was primarily talking about the peer review process in social science. Here’s the case he made.

Right now it frequently takes years for a paper to make it through the peer review process in political science, economics, or law & economics. Although this process sometimes makes the paper better, often it is just years of lateral moves. For instance, if you submit to Journal A, a reviewer may suggest to switch piece of evidence X with piece of evidence Y. If you’re rejected from Journal A, it’s reasonable to take a few months making the changes that the reviewers suggested before you submit to Journal B. After you get the referee reports back from Journal B months later, it’s all too common for a different reviewer to say it makes no sense to use piece of evidence Y instead of X. And then the process repeats.

The result is that authors spend years making tweaks on the same paper instead of starting new projects. If you believe that there is social value to the production of knowledge, this inefficient process is a massive social loss. But it may be the most important problem for academia to tackle, because whatever you think is the most important questions for researchers to study, they could be doing dramatically more of it if we could speed up the peer review process and allow them to move on to new projects. Think Neelanjan is right that FLFP in the developing world is a high priority problem to address? Fix the publication system so that researchers like Neelanjan can write twice as many papers instead of spending their time tweaking existing ones.

This brings me to the AALS proposals to reform the law review publication process that Will blogged about yesterday. The big advantage of the law review publication process is its speed. Established authors can submit their papers to dozens of law reviews in February and know with an extremely high degree of confidence that one of the journals will accept their paper by April. It would be a shame for any law review reform to give up this advantage over peer reviewed journals (although, making the process a month or two longer wouldn’t be a huge deal).

But the big drawback with the law review process is that the editors making publication decisions do not have any expertise in the subject matter of the articles specifically, or a good sense of what makes for a good article more generally. Peer reviewed journals typically don’t let second year graduate students review papers, and those students spend their time taking seminars where academic articles are read and debated. The view that a second year law student could assess the relative merits of papers after a year and a half of classes where they mostly read appellate cases is simply not credible.

The result is that law review placement is not a meaningful signal of article quality. It’s true that articles in the “Prestigious Law Journal” might be better on average than the “Low Ranked Journal”, but there is so much noise in the placement that it’s tough to look at a law professors list of publications and know much of anything other than how frequently they like to submit papers.

This creates its own kind of inefficiencies. It is most difficult for lower placed professors to successfully lateral when they do not have the advantage of their articles having a shot at top journals through blind peer review. The result is that the law professors that are best at producing research have difficultly moving to schools with better resources that would, in turn, allow them to produce more scholarship. Like the peer review process, the law review process is thus unnecessarily reducing the production of knowledge.

The only way to fix this problem is to introduce some form of peer review. Which is why it is nice to see the AALS proposals include a section on a possible peer review pool. Many of their ideas are sensible. But I would go further.

For one, I would require any author that submits an article to write referee reports for three other articles. Only after the author writes three reviews, and their paper has been reviewed three times, would the article be released to journals. Additionally, I would only allow law reviews access to the pool of peer reviews if they commit to not publishing articles that have not gone through the peer review process. The list of journals that have made that commitment would be posted online, and sticklers like me would know to not take any ones placements seriously if they publish in journals that have not made the commitment.

The AALS proposal had a versions of both these ideas, but it did not make them requirements or a cornerstone of their plan. These requirements, combined with some of proposed reforms (for instance, the requirement to take the first placement offered) could help to improve the signal in article placement without having law schools take on all of the problems of full peer review models.

Do Law Journals Need Real Reform?

Brian Galle has posted a discussion draft of A Proposal for Law Journal Reform, which is a project of The AALS Section on Scholarship, Advisory Committee on Law Journal Reform. Here is the introduction:

No one is satisfied with today’s legal publishing. The long-standing tradition of simultaneous submission to student-edited journals has always involved tradeoffs, but the costs of that approach have grown dramatically over the last decade. Where once even top journals faced a relatively manageable task in identifying promising submissions, technological innovation now enables authors to easily submit to hundreds of journals with a few clicks. The result has been enormous practical and even ethical pressures on students and authors. Top journals receive more than 4,000 submissions annually. Selection outcomes are often driven not by merit but by insider knowledge, such as whether an author knows when journals are open to selecting articles or how to “expedite” publication offers to more-preferred journals. Increasingly, top journals are demanding exclusive submission windows, undermining one of the core strengths of the traditional structure. With few clear rules of the road, opportunities for gamesmanship on each “side” are prevalent, and may be mutually reinforcing.

While we believe that legal academia can and should agree on “best practices” to improve how authors and editors conduct themselves, we are realists. No set of idealized norms can succeed in the face of enormous structural pressures. Fundamental reforms are necessary.

Thus, the Section offers two possible paths for reform, each of which can be further tailored. In the simpler path, authors will submit to a small number of journals at a time, and must accept the first offer received. Journals will not extend offers during a “quiet period” of four weeks or so. A more ambitious path involves adoption of a two-round Shapley matching system, better known as the “med school” match. In that path, authors will rank a set of journals from which they would accept offers, and journals will rank those articles that meet their publication threshold. Both paths can be combined with a new peer review pool, as we describe, and additionally AALS Member Schools can adopt and encourage compliance with a set of complementary best practices for authors and editors.

Though we detail the strengths and potential weaknesses of these options in more detail below, we want to emphasize here their overwhelming advantage over the status quo: each would essentially eliminate expedited review. Expedited review is the root cause of nearly all the problems we and other stakeholders have identified with the current approach. It motivates mass submissions and other, even less fortunate, gaming behaviors. It turns many journals into screening editors for journals that are more preferred by authors, greatly increasing both their workloads and frustrations. The time pressures it imposes make meaningful peer review next to impossible. And it systematically rewards authors who are most expert at navigating the system.

An alternative, of course, would be to turn to the exclusive-submission model common in other academic disciplines. We believe that would be too radical a step. It would greatly extend time to acceptance for most authors without alleviating the crushing workload of top-journal editors. Further, many outstanding law journals — although not enough, in our view — already operate under the traditional exclusive-submission/peer-review model of the social sciences. We believe that preserving both paths is important for the discipline.

I know that law professors love to complain about law journals and the law journal process, and I know that my own experiences surely bias my assessment of the system, but my view is that law reviews are not that bad. (As I’ve written before, most law review articles are not good, but I’m not convinced this is anything other than an application of Sturgeon’s Law — “90% of everything is crap.”)

In particular, the proposal focuses on two problems with the current system. The first is that prestige/quality sorting is imperfect, so the best articles are not always published in the most prestigious journals. This is surely true, but I’m not sure how true it is. When I come across new articles I notice a pretty consistent correlation between better articles and better journals — it’s far from perfect, but I think there’s a marked correlation. And so the question is how much the correlation is likely to increase from these new systems. I don’t think we know that, especially if we don’t know what the current correlation coefficient is, or what the causes are.

The second problem is that law review editors have too many articles to read, and therefore spend too little time on most of the articles. I take this problem more seriously, since it (according to the proposal) reflects the consensus view of law review editors. But there are good reasons for the systems we have, and so it’s hard to come up with a superior one that is likely to get any traction.

That leads me to wonder if both of the proposed solutions are too ambitious, since they require a lot of journals and authors to buy into the new regime. What if, instead, we focused on disclosure and promise enforcement on the part of the authors? What if we simply required authors to disclose how many other journals they were currently submitting to? Journals could focus their efforts, if they wanted to, on the authors who were not broadly playing the field. And what if we also allowed authors to promise to accept an offer if they received it? Journals could focus their efforts on these sure-yield articles if they wanted to.

Allowing both of these options would effectively let authors and journals opt in to one of the AALS-proposed systems — in which authors submit only to a small batch of journals and promise to accept any of them. But it would also make it possible to make marginal moves towards that equilibrium without requiring everybody to move at once. And it would leave journals and authors free to make their own judgments, which would let us find out how strong the demand for the proposed equilibrium really is.

Great News: Someone is Reining in the IRB

A few years ago, some collaborators and I wrote a paper trying to assess whether law clerks systematically influence votes at the Supreme Court. The specific research question was: do justices cast more liberal votes in years they have particularly liberal clerks and more conservative votes in years they have particularly conservative clerks?

The tricky part of answering this question is that Justices likely hire clerks that reflect their ideology at the time. For instance, if a justice is trending to the right over time, the ideology of the clerks they hire might trend to the right over time too. Fortunately for us though, the fact that Supreme Court clerks are hired during earlier terms makes it possible to account for this. I’ll spare you the equations, but we could control for the Justices’ ideology in the prior term as a way to account for whether Justices preferences for hiring particularly conservative or liberal clerks was changing.  

Our key assumption was thus that Supreme Court Clerks are hired either during the term before they started or earlier. But it would be a problem for us if most clerks were hired and started working on cases immediately. How did we know that rarely happened? We just do. 

Despite our assertion that we were sure that was the way that it worked, during the peer review process we ran into some referees that didn’t want to take our word for it. So we decided to email a random sample of 10 percent of Supreme Court clerks in our database and ask them when they were hired to clerk. Here’s the email we decided to send to the former clerks:

Dear [Former Clerk],

I am law professor at the University of Chicago Law School, and I am currently conducting research on the Supreme Court. As part of that research, I am trying to understand when Supreme Court clerks were hired for their clerkship. It is my understanding that you are a former Supreme Court clerk. I was hoping you would be willing to answer two short questions for me:

1. What is the month and year that you were offered your Supreme Court clerkship?

2. What is the month and year that you started your Supreme Court clerkship?

If you do not remember, any information would be helpful (as well as simply knowing that you do not remember). Your answers will be kept confidential. I am simply trying to document the average amount of time Supreme Court clerks are hired before they started, and will not in any way reveal personalized information.

Thank you for your time and help,

Adam

But sending out emails like this to get data for research is human subject research, and thus subject to IRB review. (Don’t know what an IRB is? Then this post isn’t for you.) So I submitted the survey to the University of Chicago’s IRB and explained that I wanted to ask 102 of the countries most sophisticated lawyers the date they were offered their clerkship just to confirm something that is widely known in legal circles.

In response, the IRB asked me to first send the former clerks a consent form to get their agreement to participate in our research. I went back and forth with the IRB explaining that requiring a consent form before soliciting answers would ensure that no one responded to my short email. Finally, we reached a compromise. I would include this in every email:

This research has been approved by the University of Chicago Institutional Review Board. If you have any questions about participating in this research, you can contact the Social & Behavioral Sciences Institutional Review Board at the University of Chicago, 1155 E. 60th Street, Room 418, Chicago, IL 60637. Phone: (773) 834-7835. Email: sbs-irb@uchicago.edu. 

Although I’m complaining about it now, getting IRB approval for this project wasn’t a huge deal. But it did it involve filling our forms and several rounds of emails. And, more importantly, it held up the research a few weeks. And this story isn’t an outlier. Talk to any researcher at an American University that collects data in any way, and they’ll have plenty of IRB red tape horror stories.

That’s why I’m excited that my colleague Omri Ben-Shahar just wrote an essay to announce a pilot he is launching to rein in the IRB at the University of Chicago. The basic idea of the pilot is that the federal law exempts many kinds of research from IRB review, but right now IRB across the country have decided they should be the one to make the determination of whether a project falls into one of the exemptions. And when they make that determination, even if the project qualifies as exempt, they can ask for revisions. (That’s what happened in my story: even though the IRB agreed my harmless email to former clerks was exempt, they still had me add the IRB language to the email.)  

In this pilot, researchers will get to make the determination of whether their research qualifies for one of the IRB exemptions for themselves. But that’s not all the pilot is doing: it’s randomizing which professors at the University of Chicago get to be part of the pilot. That way, in a few years, Omri and others can look back and see if the people in the control group (that remained subject to IRB review) committed any few instances of harmful research. In other words, the IRB reform is being done as a research project.

Like with our survey to clerks, I’m pretty sure I already know what Omri’s research will find: there will not be any differences in research misconduct between the professors in the pilot and professors subject to the normal IRB regime. But it’s great that someone is running this experiment to generate the kind of evidence we need to rein in the IRB for everyone.

---- 

Oh, and in case you are in suspense about the results of the survey: the people that responded confirmed they were hired a while before they showed up for their clerkship.

Note: I edited this post because I misspelled “rein” as “reign” and law twitter dragged me for it. On the plus side, I finally found a way to get people to engage with one of my blog posts.

Prizes, How Do They Work?

For multiple reasons, both intellectual and selfish, I obviously agree with Adam that it would be cool to have a major legal scholarship prize that brought attention and prestige to law professors. But this raises the question of what it takes for a prize to actually succeed at these kinds of social goals.

For instance, some law professors do win the kinds of major prizes Adam mentions — Danielle Citron and the MacArthur Award; James Forman and the Pulitzer Prize, to give two recent examples off the top of my head. But these don’t have the effects Adam describes on the entire discipline.

An obvious, if completely impractical, strategy is to try to convince the Nobel Foundation to add a new category for law. But that brand is tightly controlled and I’m not sure the Nobel Foundation would much interest in, or good judgment about, American legal scholarship.

A giant sum of money always helps. With a 20 million dollar endowment, roughly enough to create a 1 million dollar prize, any prize would rapidly be the focal point for the legal academy. But failing that, I’m not sure we know exactly how to establish a focal prize in a field that doesn’t have one. Except, perhaps, through experimentation. For instance, the Georgetown Center for the Constitution has started awarding a Thomas M. Cooley Book Prize (of $50,000) every year. Will it one day have the status of the Pulitzer Prize among law professors?

The Case for a Signature Prize for Legal Scholarship

There was no Pulitzer prize for fiction awarded in 2012. This decision not only upset eligible authors, but it also was terrible for bookstores. By failing to award a prize, the Pulitzer committee had essentially sent the message to the broader public that there weren’t any new novels worth reading that year. As a result, book sales suffered across the board.

I was thinking about this example when I attended a talk a few weeks ago for one of this year’s Nobel Prize winners in economics, Abhijit Banerjee. The audience filled an auditorium that’s probably normally used for plays or concerts (I actually have no idea what usually happens in the auditorium; it took an economics talk being scheduled in the space for me to learn it existed). Although economists are revered here at the University of Chicago, that kind of audience is still extremely unusual.

Abhijit’s talk not only highlighted his own research projects, but also the work of many other economists. It thus not only increased the awareness of the randomized control trials on development that this year’s laureates are now famous for, but the discipline as a whole.

Without his recent publicity for winning the Nobel, however, I’m confident that Abhijit would have spoken in front of a few dozen people instead of a few hundred. (One reason for that confidence: space was not a concern when Oliver Hart presented at our law and economics workshop in a basement seminar room just before he won the Nobel Prize in 2016.)

By failing to have major prizes like other academic disciplines (not to mention literature, music, and movies), legal scholars miss out on the opportunity to highlight our work to the broader public. This not only deprives recognition for the superstars at the top of our profession that would win the award, but also means that the public will not be exposed to legal scholarship more generally. Or, in other words, by not establishing a signature prize, we are sending the message to the broader public that there isn’t any legal scholarship worth learning about. But instead of sending that message one year, we send it every year.

Is There a FedSoc Litmus Test?

I’m being dragged plenty for my post on the Federalist Society, but I nonetheless wanted to return to the topic for a moment because of this recent piece by my colleague Geof Stone. In the course of a review of a forthcoming book by Ruth Marcus, Geof writes:

Another constituency with supreme ambition was the Federalist Society. Founded in 1982 with the active support of such figures as Antonin Scalia and Robert Bork, then professors at the University of Chicago and Yale, respectively, the society’s initial goal was to articulate a coherent conservative approach to constitutional interpretation. Over time, though, and under the leadership of Executive Vice President and Co-Chairman Leo, the ambition of the Federalist Society became less academic and more political. Its supreme ambition was to stack the courts, and especially the Supreme Court, with jurists who would support a rigid right-wing agenda.

Although it tossed out theories like judicial restraint and originalism, over time the true measure of a successful judicial appointment from the perspective of the Federalist Society turned on results. The group sees things this way: Affirmative action is unconstitutional. There is no constitutional right to abortion. The Voting Rights Act is unconstitutional. Gerrymandering is not unconstitutional. Laws limiting campaign contributions and expenditures are unconstitutional. There is no constitutional right to same-sex marriage. Laws regulating guns are unconstitutional. And on and on and on. These outcomes have nothing to do with constitutional theory and everything to do with political goals.

This is the kind of analysis that prompted my original post. “The group” doesn’t have a uniform view on any of these topics. And the individual members of those groups that I know have views that are based on about what you'd expect -- some of the more theoretically minded members follow constitutional theory where it leads them, some of the more politically driven members are likely driven by political goals. But indeed, that's probably true of every group of lawyers or academics.

For what it’s worth, of the seven constitutional positions Geof lists above, and putting aside stare decisis, I think I agree with only about four and a half of them.

Countries Are Sloppy About Signing BITs, revisited

There are countless stories of countries not caring very much about the Bilateral Investment Treaties (BITs) that they sign. Here’s one example I found for a paper I wrote a few years ago:

The United States signed a BIT with Senegal in 1983 and with Morocco in 1985. These BITs were both approved by the United States Senate on October 20, 1988. By 1990, however, these two partners had still not ratified the agreements. During a hearing held by the Senate Committee on Foreign Relations, Senator Sarbanes asked a State Department Official why Senegal and Morocco had not yet ratified the agreements. The assistant Secretary of State replied by saying that “my understanding is that both governments were looking for an auspicious occasion on which to ratify the agreement, some ceremony. I didn't know this until this morning.” In response, Senator Sarbanes asked: “Would a visit by you be deemed such a sufficient occasion, do you think?” In response, McAllister replied: “I would certainly think so. But I would hope we could do it before that.” This is revealing because Senegal and Morocco were willing to wait over three years to ratify agreements that are reportedly designed to attract investment so that they could have a signing ceremony with even a relatively obscure U.S. official.

The literature is filled with these kind of anecdotes, but most of them are about the process of treaties being signed in a sloppy way. A couple of weeks ago, however, an arbitration panel issued a decision in a dispute with a truly bonkers procedural history that shows countries don’t even pay much attention to BITs even after they are signed.

Here’s the story. In 2014, a man from South Africa brought an international investment dispute against Mozambique. The case was ostensibly brought under a BIT between South Africa and Mozambique that provided for investor-state dispute settlement. Five (!!!) years later, after lengthy proceedings, a motion to dismiss filed by Mozambique was granted. The reason? It turns out the BIT between South Africa and Mozambique never went into force.

But it took Mozambique a while to actually figure that out. In 2016, Mozambique filed a merits brief “totaling 160 pages [that] raised multiple objections to jurisdiction (in 90 paragraphs spread over 46 pages). Nowhere in these did Respondent even hint that the BIT had not entered into force.”

And if it isn’t bad enough that a dispute could drag on for years before it was clear there wasn’t even a valid treaty, here’s the conclusion of the decision:

h/t to @MatthewMorantz for highlighting this wild case.

"Chocolate Accelerates Weight Loss!" Or, why to worry about randomization.

In 2008, a program was launched called “Secure Communities” that sent information about anyone arrested by local police departments to the Federal Government so that their immigration status could be checked. The program had been billed as an effort to “secure” our communities by increasing immigration enforcement, and in turn, reducing crime.

In a 2014 paper, Adam Cox and Tom Miles examined whether the Secure Communities program actually reduced crime. The paper leveraged the fact that the program was rolled out gradually county by county to test its effects. The paper made a big splash because it found that increased focus on deportations wasn’t accomplishing much.

Cox and Miles’ paper wasn’t just substantively important — the research design has become influential too. Why? Their paper showed that the roll out was haphazard in a way that made it a quasi-random source of county-level variation in government policy. This kind of variation is what makes causal inference, and thus publication in peer reviewed journals, possible.

So, naturally, other scholars started using the roll out of Secure Communities to study other topics. For instance, Ariel White wrote a paper looking at the effect of Secure Communities on Latino vote turnout; Marcella Alsan and Crystal Yang looked at the effect of Secure Communities on take up of social insurance programs; Nest at al. explored the effect of Secure Communities on employment patterns for low-education native workers; and Dee and Murphy looked at the effect of Secure Communities on school enrollment. The research design was even used by Hines and Peri to study the effect of Secure Communities on crime (which, if you’re thinking that sounds an awful lot like the original Cox and Miles paper, you’d be right).

Why am I bringing this up? The Regulatory Review has been running a series of essays about the very sensible idea of trying to encourage the government to incorporate more randomization into their policy implementation. The hope is that by randomizing—like the way that the roll out of Secure Communities was staggered—it will be possible for scholars to evaluate the effect of programs in a rigorous way.

In general, I’m totally on board with this idea. Randomization makes it possible to do causal inference, and causal inference makes it possible to know if policies are working. But we do need to be worried that the proliferation of studies that will follow will start to produce bogus results. Here’s why.

As I explained in my essay for the Regulatory Review series , when researchers look for the effect of of a policy in a lot of places, it runs the risk of a problem called Multiple Hypothesis Testing (“MHT”). The concern with MHT is that statistically significant results happen randomly 5% of the time, so if we look at the effect of an intervention 20 times, we’re likely to find 1 bogus result.

My favorite example of this is the chocolate weight-loss hoax. To prove that newspapers will publish anything scientific sounding without thinking, a scientist/journalist conducted a study where people were randomly assigned to eat chocolate. The researchers then measured 18 outcomes for the people in the study. The study, predictably, found that one of the 18 variables was statistically significant thanks to random chance. An academic paper was published in a “predatory” journal based on the study, and newspapers around the world published stories about the finding with headlines like “Chocolate Accelerates Weight Loss”.

What does this problem have to do with government randomizing policy? The worry is that researchers are drawn to randomized policy interventions like moths to a flame. So when policies are randomized, people study them from every possible angle. And a lot of people looking for outcomes from the same intervention means we are naturally going to start getting some results due to the multiple hypotheses testing problem.

For instance, if studies keep looking for the effect of Secure Communities in more and more places, some of the results are going to be bogus. Not because the researchers are being nefarious, but just because of random chance.

———

If you’re interested in the topic, check out my essay and the rest of the series being published by the Regulatory Review. And shout out to Colleen Chien for writing the essay that inspired the series and inviting me to contribute. Thanks Colleen!

Restatements Should be Transparent

In a 2015 case, Kansas v. Nebraska, Antonin Scalia wrote a separate opinion to offer a scathing assessment of the value of Restatements:

I write separately to note that modern Restatements—such as the Restatement (Third) of Restitution and Unjust Enrichment (2010), which both opinions address in their discussions of the disgorgement remedy—are of questionable value, and must be used with caution. The object of the original Restatements was “to present an orderly statement of the general common law.” Over time, the Restatements’ authors have abandoned the mission of describing the law, and have chosen instead to set forth their aspirations for what the law ought to be. Restatement sections such as that should be given no weight whatever as to the current state of the law, and no more weight regarding what the law ought to be than the recommendations of any respected lawyer or scholar. And it cannot safely be assumed, without further inquiry, that a Restatement provision describes rather than revises current law. [citations omitted]

Two years later, Will and I wrote a paper with our colleague Anup Malani arguing that anyone trying to make descriptive claims about the state of the law on a given topic should consider conducting a so called “systematic review.”

The basic idea of a systematic review is to transparently lay out the process you used to reach your conclusion. For instance, if I wanted to know Scalia’s opinion was on Restatements, I could either just say “Scalia hated Restatements, see Kansas v. Nebraska.” Or, I could conduct a systematic review by searching every opinion and academic article written by Scalia for mentions of the Restatement, and then document whether he discussed Restatements favorably or not. Obviously, that kind of comprehensive review is a lot more time intensive, but it also reduces error. So if I really cared about Scalia’s opinion of Restatements (which, for the record, I don’t), it might be worth investing the effort.

In the article, we suggested that Restatements are exactly the kind of instance where the costs of conducting systematic reviews are worth it. When writing our paper, we were inspired in part by the methods that the reports for the Restatement of Consumer Contracts — Oren Bar-Gill, Omri Ben-Shahar, and Florencia Marotta-Wurgler — were using to code cases as part of their efforts to accurately describe the law.

We thought that reviews of the law would be more accurate because of this kind of coding, but also that the added transparency would allow replication. Or, as we put it at the time: “being explicit about the method is almost as important as the method itself, because transparency allows others to replicate the review author’s work, ensuring that the review was not manipulated and increasing confidence in the review’s conclusions.”

Since writing that, it’s been exciting to see that kind of replication play our with the Restatement of Consumer Contracts. The reporters were transparent about their process of coding cases, and Gregory Klass and others then sought out to reassess their claims through replication. In doing so, they raised a number of concerns with the original analysis, which in turn lead to a response from Oren, Omri, and Florencia explaining how the replication effort still came to the same substantive conclusions.

And although there has been some controversy over the Restatement of Consumer Contracts, their work demonstrates the kind of concrete academic exchange that is possible when reporters take the time to be rigorous. So although I’m not sure on the answer to every questions Will raised in his post, but I know that whatever reporters do, they should be transparent.

How Should Restatements Restate the Law?

Two weeks ago I spent the day at a meeting for the project to create a Third Restatement of Conflict of Laws. For the reasons I discussed on another blog long ago, I remain unsure that another Restatement of Conflict of Laws is a good idea, but working on the project has made me think about how tricky the project of Restatements is.

One much-discussed problem is the relationship between positive and normative analysis. Most of the time the restatements describe the majority rule, but sometimes they instead recommend a minority rule or a new rule. As I understand it, the general practice is that this is fine, so long as it is explicitly disclosed and normatively justified. But even putting that kind of explicit normative change to the side, a number of confusing problems come up in deciding what a Restatement should be restating:

One issue is the denominator problem. Restatements are very long and have lots and lots of detail. So for any given rule, it’s possible that most states simply haven’t clearly stated what their rule is on that specific topic. If so, the “majority rule” might actually be only a handful of cases. But those cases may or may not be representative of the logic or likely outcomes in most states.

A related issue is the combination problem. Suppose a majority of jurisdictions follow rule X. And a majority of jurisdictions follow rule Y. That doesn’t mean that a majority of jurisdictions follow rule X+Y. Indeed, depending on the denominator problem, it’s possible that none do. And of course this issue scales up across the whole Restatement.

A final issue is the relevance of statutes. Restatements generally focus on common-law topics, but sometimes the state legislatures have adopted statutes changing the common-law rule. Should one simply restate the common-law rule that would have applied in the absence of the statute, or use only the non-statutory states as the denominator? Or should one try to Restate a new common law rule that matches the prevalent statutory rule? My instinct is that the former is better, because it treats the decision whether to pass a statute as meaningful, but I think the latter is common.

Even the purely descriptive parts of a Restatement project can subtly transform the common law.