Is The Second Amendment A Second Class Right?

Volume 68 September 2018
IS THE SECOND AMENDMENT A SECOND CLASS RIGHT

Adam M. Samaha & Roy Germano†. Inez Milholland Professor of Civil Liberties, New York University School of Law; Research Scholar, New York University School of Law. Thanks to Joseph Blocher, Jessica Bulman-Pozen, Emiliano Catan, Phil Cook, Lee Epstein, Jonah Gelbach, Eric Ruben, and Laura Weinrib for their thoughts on this project. Peter Devlin provided invaluable research assistance. Mistakes are ours.

Ten years have passed since the Supreme Court endorsed a new line of constitutional litigation in District of Columbia v. Heller. [1][1]. District of Columbia v. Heller, 554 U.S. 570, 635–36 (2008) (invalidating the District’s ban on handgun possession in the home). Without much delay, the Court extended Heller’s right to keep and bear arms to state and local government action. [2][2]. See McDonald v. City of Chicago, 561 U.S. 742, 791 (2010) (plurality opinion) (addressing handgun restrictions similar to those declared invalid in Heller). This suggested that significant matters of policy could be at stake. [3][3]. See Philip J. Cook, Jens Ludwig & Adam M. Samaha, Gun Control After Heller: Threats and Sideshows from a Social Welfare Perspective, 56 UCLA L. Rev. 1041, 1069–71 (2009). But the Court has done nearly nothing else to elaborate. [4][4]. An arguable exception is Caetano v. Massachusetts, 136 S. Ct. 1027, 1028 (2016) (per curiam) (rejecting three reasons for upholding a stun-gun ban, and remanding). And the Court’s position—probably a work in progress anyway, even without new appointees—was open to unexciting interpretations. [5][5]. See Cook et al., supra note 3, at 1061–64. Perhaps Heller’s slim majority would end up objecting to only the most extreme forms of gun control that happened to survive the usual political opposition.

In the meantime, other people had to handle the ensuing litigation. Lower federal courts and state courts have now fielded more than a thousand Second Amendment claims, as documented in Eric Ruben and Joseph Blocher’s impressive research. [6][6]. See Eric Ruben & Joseph Blocher, From Theory to Doctrine: An Empirical Analysis of the Right to Keep and Bear Arms After Heller, 67 Duke L.J. 1433, 1455 (2018). Those courts have shouldered the tasks of clearing cases and developing doctrine. Not without criticism, of course. For observers who remain unconvinced that there is or ever was a federal constitutional right to keep and bear arms unrelated to militia service, any effort spent in this field of litigation is a dead-weight loss. Any claimant victory is an unearned gratuity, on this view.

The opposite complaint probably receives more attention, however. After all, lower federal and state courts usually reject gun rights claims when they reach the merits. Ruben and Blocher’s study reinforces this impression with numbers. [7][7]. See id. at 1447. Indeed, an allegedly deep judicial opposition to gun rights has become an argument for renewed Supreme Court attention. [8][8]. See id. at 1447–51 (collecting sources). A few Justices have alleged, with memorable phrasing, that judicial mistreatment is turning the Second Amendment into a “second-class right.” [9][9]. E.g., Silvester v. Becerra, 138 S. Ct. 945, 952 (2018) (Thomas, J., dissenting from denial of certiorari) (charging lower courts and the Supreme Court with treating the Second Amendment as a disfavored right, and quoting McDonald v. City of Chicago, 561 U.S. 742, 780 (2010) (asserting that Chicago’s view would effectively convert the Amendment into a “second-class right”)); Peruta v. California, 137 S. Ct. 1995, 1999 (2017) (Thomas, J., joined by Gorsuch, J., dissenting from denial of certiorari) (similar); Friedman v. City of Highland Park, 136 S. Ct. 447, 449 (2015) (Thomas, J., joined by Scalia, J., dissenting from denial of certiorari) (similar). Silvester involved a waiting period, Peruta involved public carry, and Friedman involved an assault-weapons ban.

Is this true? A second-class complaint is normative insofar as it maintains that judges are not giving gun rights adequate protection. [10][10]. Accord Ruben & Blocher, supra note 6, at 1449–50, 1507 (discussing underenforcement complaints). Like Ruben and Blocher, our goal here is not to revisit the normative commitments that underlie that conclusion. We can, nonetheless, work on the empirical element of a second-class complaint. The complaint is factual and relative insofar as it compares how judges treat different subcategories of constitutional claiming.

To be sure, particularists may deny that a subcategory of legal claim can be usefully compared to another subcategory “all else equal.” Subcategories of claims are, by definition, somehow different from each other. Usually we cannot switch the constitutional clause cited and leave everything else about a case the same. But for observers who are comfortable aggregating individual claims into subcategories of litigation—people who must include those making second-class complaints—we can offer some data.

Below we compare five fields of high-profile constitutional litigation in the United States Courts of Appeals: gun rights, abortion rights, affirmative action, establishment of religion, and commercial speech. We do see different rates of success in attracting judge votes across some of these case sets, with gun rights claims at the low end. We are unable to erase or fully explain the relatively poor performance of gun rights claims by introducing several independent variables. And judges appointed by Democrats might contribute more to the disparities than judges appointed by Republicans, although neither set of judges seems to support all claims at equal rates.

Nevertheless, we cannot make much progress on why gun rights claims might fare worse than certain other claims. The second-class complaint points at judges. But plausible explanations are available that have nothing to do with judges disliking gun rights, [11][11]. See Adam M. Samaha & Roy Germano, Are Commercial Speech Cases Ideological? An Empirical Inquiry, 25 Wm. & Mary Bill Rts. J. 827, 860–61 & n.176 (2017) (offering preliminary and possible explanations for different win rates across claim categories). and existing data cannot rule out those alternatives. Nor is it clear that more data like ours will solve the problem of isolating the contribution of judicial preferences to case results across fields of litigation. More fully detailing and explaining the forces at play must await future work, a significant part of which will have to be theoretical, historical, and ethnographic.

I. Comparisons and Complications

Litigation success rates vary across fields of litigation. Researching that much is relatively straightforward, when the fields are defined well enough and the search or sample of cases is adequate. We report some success rates below. But explaining litigation success rates is much harder. Long streams of intertwined events occur before any judge enters the scene of a case, although previous judicial decisions might well influence these events. People develop various policy commitments, policy makers decide whether and how to regulate, lawyers and clients decide whether and how to litigate, judges decide whether and how to guide other actors through doctrine in written opinions. These events shape the set of claims that judges see.

We should be cautious, therefore, in using litigation success rates to draw inferences about judge preferences or decision standards. [12][12]. See Theodore Eisenberg, Testing the Selection Effect: A New Theoretical Framework with Empirical Tests, 19 J. Legal Stud. 337, 337–38, 340 (1990) (discussing selection effects that can defeat inferences about whether the rules are “skewed”). This should be a lesson from the sophisticated literature on selection effects in litigation. A well-known early effort to model litigation and settlement choices is George L. Priest & Benjamin Klein, The Selection of Disputes for Litigation, 13 J. Legal Stud. 1, 20–26 (1984) (indicating that, when the stakes are symmetric, the plaintiff win rate at trial will tend toward 50 percent regardless of the decision standard). Reconsiderations include Eisenberg, supra, at 338–40, 352, 355–56 (finding varying trial win rates in federal court data, without rejecting selection effect theories more generally); Steven Shavell, Any Frequency of Plaintiff Victory at Trial Is Possible, 25 J. Legal Stud. 493 (1996) (warning against the suggested 50-percent tendency under the Priest-Klein model or under an asymmetric information model); Jonah B. Gelbach, The Reduced Form of Litigation Models and the Plaintiff’s Win Rate 16–18 (Univ. of Penn. Inst. For Law & Econ., Research Paper No. 16-22), available at https://ssrn.com/abstract=2837154 [https://perma.cc/8QV5-SU35] (extending Shavell’s warning to a range of litigation models, including some that assume very accurate party information); and Daniel Klerman & Yoon-Ho Alex Lee, Inferences from Litigated Cases, 43 J. Legal Studies 209, 210–11, 214 (2014) (contending that, under certain assumptions and models, one may cautiously draw inferences about the decision standard from the trial win rate). Caution is warranted even if formal law seems vague in a way that invites judicial discretion. [13][13]. We continue to doubt that uncertainty about current law, alone, predicts low success rates for claimants. See Samaha & Germano, supra note 11, at 860–61 n.176. In fact, the litigation success rate could be low, middling, or high while the judicial decision standard for supporting the claims is low, middling, or high. [14][14]. The “decision standard” could be based on extralegal judicial preferences, valid legal sources, or both. We bracket the possibility of judges automatically opposing a given subcategory of claim. Consider gun rights claims before Heller, or abortion rights claims before Roe v. Wade, 410 U.S. 113 (1973). Much depends on how non-judges think and act.

If claimants are willing and able to litigate assertively, pushing the composition of claims toward those that are unlikely to persuade judges yet valuable to the claimants nonetheless, then we should expect a downward pull on the success rate. For example, cash-flush organizations might litigate improbable claims to deter regulatory efforts; clients and cause lawyers might want to voice their positions in litigation even if, or because, judges are likely to disagree; and optimistic parties might hope for rescue from the Supreme Court. [15][15]. See Samaha & Germano, supra note 11, at 861 n.176 (discussing deterrence through litigation and other phenomena); see also Ben Depoorter, The Upside of Losing, 113 Colum. L. Rev. 817, 820–21 (2013) (discussing how unfavorable court outcomes might mobilize political support); cf. James E. Krier & Stewart E. Sterk, An Empirical Study of Implicit Takings, 58 Wm. & Mary L. Rev. 35, 65 (2016) (suggesting asymmetric stakes, deterring government regulators, high settlement costs where government lawyers cannot bind their clients, and asymmetric information to explain low success rates for certain regulatory takings claims in state courts). Likewise, if governments are unwilling or unable to regulate, enforce, or litigate assertively when their odds in court seem long, then we should expect another downward pull on the claimant success rate. Judges would see fewer vulnerable targets. On the flip side, assertive governments and cautious claimants should have an upward pull on that rate.

If this non-judicial behavior were stable and similar across fields, we could draw stronger inferences about judges by comparing litigation success rates. Different rates would be more easily attributed to judges, though we could still wonder about the influence of law’s sources as opposed to extralegal preferences. But of course the relevant behavior might change over time, it might vary across fields of litigation, and it might influence litigation success rates even if all judges are otherwise equally receptive to supporting any subcategory of claim.

Absent an independent measure of claim quality, we can gather suggestive data on judges, claimants, and regulators. But our aspirations regarding explanations must be modest. To date we lack direct measures across case sets for trial-level judgments and settlements; filing and appeal rates; litigation costs and available resources; perceived stakes; severity of regulation and enforcement; the formal content of constitutional law at the time of decision; and so on. Moreover, the plausible influences on case results are complex when taken together, aside from the data collection challenges. No feasible study can account for all potential influences. Instead we can make incremental—and hopefully provocative—advances with available data. [16][16]. We greatly appreciate Ruben and Blocher’s efforts to build a large dataset and to offer initial analysis without overly firm conclusions. Given the plausible causation theories and limited information, however, we are not quite ready to agree that their “data shows that the low rate of success probably has more to do with the claims being asserted than with judicial hostility to the right.” Ruben & Blocher, supra note 6, at 1507.

II. Data

We rely on five existing case sets, [17][17]. See Samaha & Germano, supra note 11, at 843–47 (describing the original sets); Adam M. Samaha, Codebook: Constitutional Case Sets App’x A (Aug. 13, 2018) (on file with the authors and the Duke Law Journal) (describing search queries and grounds for exclusion of cases). which we have extended through the end of 2016. The aim has been to gather the universe of federal circuit court decisions on the merits of various constitutional claims, in the mold of prior research by Lee Epstein, Cass Sunstein, and their teams. [18][18]. See generally Lee Epstein, William M. Landes & Richard A. Posner, The Behavior of Federal Judges (2013); Cass R. Sunstein, David Schkade, Lisa M. Ellman & Andres Sawicki, Are Judges Political? An Empirical Analysis of the Federal Judiciary (2006). Ruben and Blocher graciously helped us round out the gun rights case set, while Sepehr Shahshahani and Lawrence Liu shared data on recent Establishment Clause appeals. [19][19]. See generally Sepehr Shahshahani & Lawrence J. Liu, Religion and Judging on the Federal Courts of Appeals, 14 J. Empirical Legal Stud. 716 (2017). The Establishment Clause case set is less comprehensive than the others: fewer of the unpublished decisions have been gathered, which could leave the success rate for that case set artificially high. [20][20]. Following the protocol of an earlier study, see Gregory C. Sisk & Michael Heise, Ideology “All the Way Down”? An Empirical Study of Establishment Clause Decisions in the Federal Courts, 110 Mich. L. Rev. 1201, 1207–08 & n.27 (2012), we gathered Establishment Clause cases from searches of opinions with digests on Westlaw. Not every unpublished decision has a Westlaw digest. We presume that unpublished decisions reject constitutional claims more often than do published opinions. On the other hand, Shahshahani and Liu, supra note 19, at 723, left in some cases decided on justiciability grounds. The five case sets are broadly similar in scope, nonetheless.

Our analysis examines cases decided during and after 2008, the year in which Heller was issued. We exclude en banc decisions to concentrate on the ordinary three-judge panel situation. [21][21]. On the assumption of random assignment in this setting, see Adam S. Chilton & Marin K. Levy, Challenging the Randomness of Panel Assignment in the Federal Courts of Appeals, 101 Cornell L. Rev. 1, 3–7 & n.4 (2015) (concluding that the circuit courts do not always randomly assign judges to three-judge panels for oral argument, even if they randomly assign cases to panels). We also exclude criminal and habeas corpus appeals to concentrate on civil litigation in which claimants and lawyers tend to be more free to pick their targets. Gun rights cases include a large fraction of criminal-defendant claimants, [22][22]. See Samaha & Germano, supra note 11, at 860. who may have appointed counsel and who often adopt implausible legal positions when facing serious penalties. The other four case sets have nowhere near this fraction of criminal and habeas appeals. [23][23]. Perhaps the criminal cases generate precedent that drags down the civil claims. This effect has been suggested in the Fourth Amendment setting. See Nancy Leong, Improving Rights, 100 Va. L. Rev. 377, 393, 473–74 (2014).

We model the likelihood of a circuit judge voting in favor of a claim or claims in a case. A judge is counted as voting “in favor” if he or she supported at least part of at least one relevant claim in the case at hand. Many cases are understood by judges to present only one constitutional claim, some cases multiple claims. We simplify those differences by leaving claims within single-case packages that are up for one vote, while breaking out individual judge votes on those packages.

In contrast, Ruben and Blocher break out claims from cases. They track court judgments on various gun rights claims, along with the associated content of court opinions, without analyzing the behavior of individual judges. Ideally, we would take the deep coding of opinions in the Blocher and Ruben study, add data on the participating judges, and then compare gun rights with other case sets at both the judge level and the judgment level. Unfortunately, other case sets are not coded like Ruben and Blocher’s extensive dataset, and some of their coding is unique to gun rights doctrine. For now, we persist with our judge-vote approach so that we may begin to make timely comparisons.

III. Models

Our starting point, Model 1, combines observations on judge votes from all five case sets, using dummy variables to mark each set. Subsequent models add control variables related to the judges, the claimants, and the government action under challenge. All models include circuit fixed effects to help control for unmeasured regional variation in regulation, litigation, and judicial practices. Our reported models cluster standard errors on cases, on the theory that votes of judges on the same panel might not be independent. This is a relatively conservative choice for establishing statistical significance in our dataset. In alternative specifications, we have clustered errors on judges.

On the judges, Model 2 adds the party of the appointing president, as a proxy for judge ideology or party identification. For two case sets, we recoded this binary variable so that a 1 consistently indicates expected support for the claim, based on what we take to be conventional understandings of ideological commitments in the United States for this timeframe. [24][24]. Ordinarily, 0 indicates that the judge was nominated by a Democratic president and 1 indicates that the judge was nominated by a Republican. We recoded the abortion rights and Establishment Clause case sets so that Democratic nominees got the value 1 and Republican nominees got the value 0. For simplicity and because our aspirations are modest, we have not interacted judge party with the case set variables.
In unreported models, we have added a variable for partisan panel effects: the fraction of other judges on the panel who were appointed by Republican presidents—or instead, for the abortion rights and Establishment Clause case sets, those appointed by Democratic presidents.
Our models will not reveal whether judicial ideology is a good predictor within each case set or a better predictor in some case sets. Instead, the judge party variable helps control for the possibility that some case sets were assigned a mix of judges who were more likely to evaluate the claims favorably.

On the claimants, Model 3 adds a variable for whether the claimant was the appellant. The courts of appeals tend to affirm. [25][25]. Possible explanations include formal standards of review, deference to trial judge work, trial judge votes being predictive of appellate judge votes, and appellants facing low incremental costs for appeals. See Kevin M. Clermont & Theodore Eisenberg, Litigation Realities, 88 Cornell L. Rev. 119, 150–52 (2002). If claimants in some case sets are more likely to be appellants, they might be more likely to lose, all else equal. And eagerness to appeal might reveal more about the claimants than the judges. That said, trial judges might be especially opposed to some subcategories of claims, which would generate a larger fraction of disappointed claimants who might then appeal. We lack comprehensive data on how trial judges treat constitutional claims and how often parties appeal.

Model 4 includes pro se status. Pro se claimants usually are at a disadvantage, at least if judges do not compensate by lowering their standards for pro se claims, [26][26]. Cf. D. James Greiner & Cassandra Wolos Pattanayak, Randomized Evaluation in Legal Assistance: What Difference Does Representation (Offer and Actual Use) Make?, 121 Yale L.J. 2118, 2174 (2012) (discussing possibilities in the context of administrative adjudication of unemployment benefits claims). and the fraction of people doggedly willing to pursue appeals without a lawyer will vary across fields. Model 4 also includes variables to indicate whether at least one of the constitutional claimants was an individual, a business, or an interest group. Businesses and interest groups might tend to have greater resources for litigation or be more sophisticated at litigation decisionmaking, compared to individuals. Nonetheless, sometimes organizations will be content with a litigation loss that can be converted into future support. As well, we have not distinguished cases in which individuals joined with businesses or interest groups.

On government action, Model 5 adds a variable for whether federal, state, or local action was challenged. A sensible theory is that the federal government will tend to perform better in litigation than state and local governments, and that states will tend to do better than locals. [27][27]. See Samaha & Germano, supra note 11, at 857. One reason is disparity in litigation resources when these governments are litigants, which we are not coding directly. Another reason might be that national and even statewide politics tend to yield policies and practices that are less extreme, more carefully advised, and easier to defend in court. Different levels of government aggregate information and political forces in unique ways. However, it is also possible that claimants and lawyers will select their targets such that any differences across levels of government will not reappear in cases that are filed and appealed. Or the selection effects could vary across case sets.

Model 6 introduces a variable regarding Supreme Court cases. Lower court judges pay attention to messages from the Supreme Court, at least in explaining their decisions and sometimes in reaching them. We collected Supreme Court cases within each case set, coded them as either supporting or rejecting the relevant claim(s), and created a continuous score based on citation counts in the circuit courts. [28][28]. See id. at 854. This variable is not designed to take judges out of the equation, obviously. The attempt is to examine the Supreme Court’s influence on circuit judge voting. The Court’s recent support, opposition, or silence regarding certain constitutional claims might make the success of such claims in the circuit courts less or more impressive.

Finally, we split the sample along party lines. [29][29]. Thanks to Lee Epstein for the suggestion. We estimated our models on the subsample of judges appointed by Democratic presidents, and then separately on the subsample of judges appointed by Republicans. Any differing levels of judicial support across case sets might arise from the voting patterns of Republican appointees, Democratic appointees, or both. However, splitting the sample this way will not identify the influence of unmeasured regulatory and litigation behaviors. Those non-judge behaviors might or might not differ across case sets, whether or not the claim types themselves induce different voting behavior from different judges.

IV. Results

Table 1 presents some characteristics of the claims, claimants, and judges in our dataset. Gun rights claims had the lowest fraction of judge support (18 percent), and they were pursued by the highest fractions of pro se litigants (14 percent) and appellants who lost in the proceedings below (94 percent, remarkably). [30][30]. This exceptionally high fraction of claimants who were appellants is independently interesting and subject to competing explanatory theories—once again including litigant behavior and judicial attitudes. Abortion rights claims are at the opposite ends of those measures. So, perhaps disadvantaged categories of litigants will help explain the apparently different rates of success across claims. Note that we found far fewer cases that resolved abortion rights or anti-affirmative action claims on the merits, compared to the other case sets. The relative rarity and success of these claims might not be a coincidence, though the small numbers limit what we can learn.

Binomial logit estimates are reported in Table 2. In these models, we made gun rights claims our baseline. This allows us to compare the probability of a judge voting in favor of gun rights claims to the probabilities in each of the other four sets of claims. On the whole, these models show that judges are significantly more likely to vote for claims in the other four case sets. This finding is robust to the inclusion of a number of available control variables, including the party of the president who appointed the judge (Model 2), several claimant characteristics (Model 3 and Model 4), whether federal, state, or local government action is being challenged (Model 5), and recent decisions from the Supreme Court (Model 6). The results for commercial speech claims are not fully consistent across models, however. In Model 4 and Model 5, the differences between commercial speech and gun rights claims are not statistically significant. Those results are, in turn, sensitive to how standard errors are clustered. [31][31]. When we cluster standard errors on judges instead of cases, the standard errors are smaller. Consequently, the difference between commercial speech and gun rights claims becomes significant at p < 0.05 or p < 0.01 in all of our models with errors clustered on judges. In any event, gun rights claims generally underperform.

Encouragingly, several control variables are significant in the expected directions. The judge party variable is consistently positive and statistically significant at p < 0.01, while the claimant appellant variable is consistently negative at p < 0.05 or p < 0.01. Interestingly, pro se status is not correlated with judge votes, all else equal. Nor do we find significant associations between judge votes and business claimants or interest group claimants, compared to individual claimants. [32][32]. When we cluster errors on judges, pro se status and interest group claimant participation are significant at p < 0.05 in some of our models. But claims do seem to fare better against local and especially state action, compared to federal action. [33][33]. Cf. Ruben & Blocher, supra note 6, at 1505 (comparing judgments on Second Amendment challenges to federal, state, and local actions in federal and state appellate courts). When we add our panel effects variable, see supra note 24, the local action variable does not reach p < 0.05, while the state action variable remains at p < 0.01. And our Supreme Court variable is positive and significant at p < 0.01. Recent messages from the Supreme Court may therefore affect voting in the Courts of Appeals. Even so, gun rights claims still seem to lag behind. [34][34]. We ran Models 1–6 with abortion rights claims as the baseline and errors clustered on judges, which we have omitted in the interest of brevity. Those results indicate that our independent variables do erase several apparent differences between case sets. Once we introduce the claimant-appellant variable in Model 3, only gun rights claims perform differently (here, worse) compared to abortion rights claims at p < 0.05. In Models 3, 4, and 5, abortion rights claims are not significantly different in their likelihood of attracting judge votes compared to commercial speech, Establishment Clause, or anti-affirmative action claims.

This gun-rights lag might be more attributable to the votes of Democratic appointees. As noted above, we split the sample into Democratic appointees and Republican appointees and reestimated the models separately. In Table 2, we display results for these subsamples using Model 6. Results for the Democratic appointees generally mirror what we found in the full sample: these judges are significantly more likely to support the other four types of claims compared to gun rights claims. Even anti-affirmative action claims are more likely to attract votes from Democratic appointees. [35][35]. As in the full sample, the difference between commercial speech and gun rights claims is not significant in Model 4 or Model 5 for the Democratic appointees, with errors clustered on cases. With errors clustered on judges, that difference is significant at p < 0.05.

Republican appointees seem different. When we look only at the votes of Republican appointees, we find that these judges are not significantly more likely to support other types of claims compared to gun rights claims—with the notable exception of anti-affirmative action claims. Like their Democratic counterparts, Republican appointees seem significantly more likely to support challenges to affirmative action programs than challenges to firearms restrictions. [36][36]. When errors are clustered on judges, there is one model (Model 6) in which Republican appointees are more likely to support Establishment Clause claims than gun rights claims at p < 0.05. Adding a variable for panel effects does not affect these conclusions. [37][37]. See supra note 24. In both subsamples, the same variables are significant (or not) at the p < 0.05 threshold when we add our panel effects variable to Model 6.

V. Reflections

The judicial treatment of gun rights claims is, in some respects, remarkable and a bit mysterious. We have little doubt that gun rights claims have been attracting a lower percentage of votes on the merits from circuit judges in civil cases since Heller, compared to the other four claim types in our dataset. Our case collection effort was thorough enough to assert that much. But we cannot make claims about other case sets or other courts. Low litigation success rates have been reported in other studies, after all. [38][38]. See, e.g., Sunstein et al., supra note 18, at 48–50 (indicating that federal circuit judges voted to support commerce clause challenges less than 10 percent of the time); Krier & Sterk, supra note 15, at 58 (reporting similarly low success rates for regulatory takings claims in state courts); Nancy Leong, Making Rights, 92 B.U. L. Rev. 405, 428–29 & tbl.6 (2012) (reporting success rates for subcategories of Fourth Amendment claims in published circuit court opinions, some of which were around 20 percent); see also Nancy J. King, Non-Capital Habeas Cases After Appellate Review: An Empirical Analysis, 24 Fed. Sentencing Rptr. 308, 317 (2012) (reporting success rates under 1 percent).

Nor can we give a complete explanation for the differences that we do see. We controlled for a limited number of variables. Gun regulation and litigation choices could be distinctive in ways that pull down the probability of litigation success, and in ways that we have not measured. Judicial decision standards certainly could be part of the explanation; our subsample results are provocative in this regard. But finding distinctive voting patterns among Democratic and Republican appointees is closer to the beginning of the investigation than the end.

Moreover, last place doesn’t necessarily mean second class. Conceptual challenges arise when cases are compared across familiar boundaries, as we noted up front. People need not accept that waiting periods for firearms and abortion services are essentially the same phenomenon, or that the bundle of claims against advertising regulations can be equated with any other bundle of constitutional claims. Furthermore, for those who do appreciate comparisons among case sets, differences in success rates might be welcome. Standing alone, differences do not prove that judges are erring. The appropriate response depends on one’s well-grounded normative position on law and the judicial role in these contested fields. And if equalization of success rates is justified, a normative framework is required to tell which claims should have their rates leveled up as opposed to leveled down. Our study is meant to develop hard questions about constitutional litigation, not to prescribe reforms.

To learn more, additional data is welcome. For example, information on the range of parties, amici, and attorneys in trial and appellate litigation would help, as would proxies for the assertiveness of regulation across fields. We should develop a clearer picture of whether, for instance, abortion rights and anti-affirmative action claimants generally are cautious parties who face envelope-pushing officials; and whether gun rights advocates and commercial advertisers generally are deep-pocketed or ideologically committed parties who face shy regulators. Broader and automated data collection will contribute to this learning.

Other kinds of research might be even more valuable at this stage, however. Histories and case studies of litigation decisions and the surrounding forces can advance our theories and inform further efforts. True, the key decisions and motivations often are difficult for outside observers to document and understand, even when few people are involved. Nonetheless, scholars in more than one discipline have produced detailed investigations into various social movements and legal reform efforts. More such work can be done for gun rights and other fields to help us understand when, how, and why people turn to constitutional litigation. As we better understand those upstream forces, we will better understand the judiciary’s reactions.

Conclusion

The Supreme Court will take up another gun rights case at some point and it will have plenty of opportunities to do so. Gun rights claimants aren’t going anywhere. But whatever the Court might want to achieve by granting review, success rates in lower courts are not sensible grounds on which to make decisions unless someone can explain those rates convincingly. Presumably the Court does not want to reward any class of petitioners for flooding lower courts with weak claims, whether those claims are for gun rights, habeas corpus relief, or anything else. Anyway, the Justices probably can make reasonable decisions without determining which sets of petitioners are the most disadvantaged classes in litigation.

Still, for anyone interested in the operation and effects of constitutional litigation, the measurement and explanation of courtroom success rates is an ongoing challenge. The scholarly understanding of gun rights litigation is hardly second class, but it probably isn’t yet first rate. In this paper, we offer some new results, ideas, and ways forward. Ruben and Blocher’s study contributes to the stock of data, too. And, in the years ahead, we will learn even more if we can combine a variety of research methods—relying on a blend of scholars who try to understand not only litigated outcomes but also the confluence of upstream forces that constrain, propel, and shape constitutional claims.

 

Table 1: Characteristics of Claimants, and Judges

Click Image to Enlarge

 

Table 2: Circuit Judge Votes for Claims, in Full or in Part

Click Image to Enlarge


Copyright© 2018 Adam M. Samaha & Roy Germano.

Inez Milholland Professor of Civil Liberties, New York University School of Law; Research Scholar, New York University School of Law. Thanks to Joseph Blocher, Jessica Bulman-Pozen, Emiliano Catan, Phil Cook, Lee Epstein, Jonah Gelbach, Eric Ruben, and Laura Weinrib for their thoughts on this project. Peter Devlin provided invaluable research assistance. Mistakes are ours.

[1] District of Columbia v. Heller, 554 U.S. 570, 635–36 (2008) (invalidating the District’s ban on handgun possession in the home).

[2] See McDonald v. City of Chicago, 561 U.S. 742, 791 (2010) (plurality opinion) (addressing handgun restrictions similar to those declared invalid in Heller).

[3] See Philip J. Cook, Jens Ludwig & Adam M. Samaha, Gun Control After Heller: Threats and Sideshows from a Social Welfare Perspective, 56 UCLA L. Rev. 1041, 1069–71 (2009).

[4] An arguable exception is Caetano v. Massachusetts, 136 S. Ct. 1027, 1028 (2016) (per curiam) (rejecting three reasons for upholding a stun-gun ban, and remanding).

[5] See Cook et al., supra note 3, at 1061–64.

[6] See Eric Ruben & Joseph Blocher, From Theory to Doctrine: An Empirical Analysis of the Right to Keep and Bear Arms After Heller, 67 Duke L.J. 1433, 1455 (2018).

[7] See id. at 1447.

[8] See id. at 1447–51 (collecting sources).

[9] E.g., Silvester v. Becerra, 138 S. Ct. 945, 952 (2018) (Thomas, J., dissenting from denial of certiorari) (charging lower courts and the Supreme Court with treating the Second Amendment as a disfavored right, and quoting McDonald v. City of Chicago, 561 U.S. 742, 780 (2010) (asserting that Chicago’s view would effectively convert the Amendment into a “second-class right”)); Peruta v. California, 137 S. Ct. 1995, 1999 (2017) (Thomas, J., joined by Gorsuch, J., dissenting from denial of certiorari) (similar); Friedman v. City of Highland Park, 136 S. Ct. 447, 449 (2015) (Thomas, J., joined by Scalia, J., dissenting from denial of certiorari) (similar). Silvester involved a waiting period, Peruta involved public carry, and Friedman involved an assault-weapons ban.

[10] Accord Ruben & Blocher, supra note 6, at 1449–50, 1507 (discussing underenforcement complaints).

[11] See Adam M. Samaha & Roy Germano, Are Commercial Speech Cases Ideological? An Empirical Inquiry, 25 Wm. & Mary Bill Rts. J. 827, 860–61 & n.176 (2017) (offering preliminary and possible explanations for different win rates across claim categories).

[12] See Theodore Eisenberg, Testing the Selection Effect: A New Theoretical Framework with Empirical Tests, 19 J. Legal Stud. 337, 337–38, 340 (1990) (discussing selection effects that can defeat inferences about whether the rules are “skewed”). This should be a lesson from the sophisticated literature on selection effects in litigation. A well-known early effort to model litigation and settlement choices is George L. Priest & Benjamin Klein, The Selection of Disputes for Litigation, 13 J. Legal Stud. 1, 20–26 (1984) (indicating that, when the stakes are symmetric, the plaintiff win rate at trial will tend toward 50 percent regardless of the decision standard). Reconsiderations include Eisenberg, supra, at 338–40, 352, 355–56 (finding varying trial win rates in federal court data, without rejecting selection effect theories more generally); Steven Shavell, Any Frequency of Plaintiff Victory at Trial Is Possible, 25 J. Legal Stud. 493 (1996) (warning against the suggested 50-percent tendency under the Priest-Klein model or under an asymmetric information model); Jonah B. Gelbach, The Reduced Form of Litigation Models and the Plaintiff’s Win Rate 16–18 (Univ. of Penn. Inst. For Law & Econ., Research Paper No. 16-22), available at https://ssrn.com/abstract=2837154 [https://perma.cc/8QV5-SU35] (extending Shavell’s warning to a range of litigation models, including some that assume very accurate party information); and Daniel Klerman & Yoon-Ho Alex Lee, Inferences from Litigated Cases, 43 J. Legal Studies 209, 210–11, 214 (2014) (contending that, under certain assumptions and models, one may cautiously draw inferences about the decision standard from the trial win rate).

[13] We continue to doubt that uncertainty about current law, alone, predicts low success rates for claimants. See Samaha & Germano, supra note 11, at 860–61 n.176.

[14] The “decision standard” could be based on extralegal judicial preferences, valid legal sources, or both. We bracket the possibility of judges automatically opposing a given subcategory of claim. Consider gun rights claims before Heller, or abortion rights claims before Roe v. Wade, 410 U.S. 113 (1973).

[15] See Samaha & Germano, supra note 11, at 861 n.176 (discussing deterrence through litigation and other phenomena); see also Ben Depoorter, The Upside of Losing, 113 Colum. L. Rev. 817, 820–21 (2013) (discussing how unfavorable court outcomes might mobilize political support); cf. James E. Krier & Stewart E. Sterk, An Empirical Study of Implicit Takings, 58 Wm. & Mary L. Rev. 35, 65 (2016) (suggesting asymmetric stakes, deterring government regulators, high settlement costs where government lawyers cannot bind their clients, and asymmetric information to explain low success rates for certain regulatory takings claims in state courts).

[16] We greatly appreciate Ruben and Blocher’s efforts to build a large dataset and to offer initial analysis without overly firm conclusions. Given the plausible causation theories and limited information, however, we are not quite ready to agree that their “data shows that the low rate of success probably has more to do with the claims being asserted than with judicial hostility to the right.” Ruben & Blocher, supra note 6, at 1507.

[17] See Samaha & Germano, supra note 11, at 843–47 (describing the original sets); Adam M. Samaha, Codebook: Constitutional Case Sets App’x A (Aug. 13, 2018) (on file with the authors and the Duke Law Journal) (describing search queries and grounds for exclusion of cases).

[18] See generally Lee Epstein, William M. Landes & Richard A. Posner, The Behavior of Federal Judges (2013); Cass R. Sunstein, David Schkade, Lisa M. Ellman & Andres Sawicki, Are Judges Political? An Empirical Analysis of the Federal Judiciary (2006).

[19] See generally Sepehr Shahshahani & Lawrence J. Liu, Religion and Judging on the Federal Courts of Appeals, 14 J. Empirical Legal Stud. 716 (2017).

[20] Following the protocol of an earlier study, see Gregory C. Sisk & Michael Heise, Ideology “All the Way Down”? An Empirical Study of Establishment Clause Decisions in the Federal Courts, 110 Mich. L. Rev. 1201, 1207–08 & n.27 (2012), we gathered Establishment Clause cases from searches of opinions with digests on Westlaw. Not every unpublished decision has a Westlaw digest. We presume that unpublished decisions reject constitutional claims more often than do published opinions. On the other hand, Shahshahani and Liu, supra note 19, at 723, left in some cases decided on justiciability grounds.

[21] On the assumption of random assignment in this setting, see Adam S. Chilton & Marin K. Levy, Challenging the Randomness of Panel Assignment in the Federal Courts of Appeals, 101 Cornell L. Rev. 1, 3–7 & n.4 (2015) (concluding that the circuit courts do not always randomly assign judges to three-judge panels for oral argument, even if they randomly assign cases to panels).

[22] See Samaha & Germano, supra note 11, at 860.

[23] Perhaps the criminal cases generate precedent that drags down the civil claims. This effect has been suggested in the Fourth Amendment setting. See Nancy Leong, Improving Rights, 100 Va. L. Rev. 377, 393, 473–74 (2014).

[24] Ordinarily, 0 indicates that the judge was nominated by a Democratic president and 1 indicates that the judge was nominated by a Republican. We recoded the abortion rights and Establishment Clause case sets so that Democratic nominees got the value 1 and Republican nominees got the value 0. For simplicity and because our aspirations are modest, we have not interacted judge party with the case set variables.
In unreported models, we have added a variable for partisan panel effects: the fraction of other judges on the panel who were appointed by Republican presidents—or instead, for the abortion rights and Establishment Clause case sets, those appointed by Democratic presidents.

[25] Possible explanations include formal standards of review, deference to trial judge work, trial judge votes being predictive of appellate judge votes, and appellants facing low incremental costs for appeals. See Kevin M. Clermont & Theodore Eisenberg, Litigation Realities, 88 Cornell L. Rev. 119, 150–52 (2002).

[26] Cf. D. James Greiner & Cassandra Wolos Pattanayak, Randomized Evaluation in Legal Assistance: What Difference Does Representation (Offer and Actual Use) Make?, 121 Yale L.J. 2118, 2174 (2012) (discussing possibilities in the context of administrative adjudication of unemployment benefits claims).

[27] See Samaha & Germano, supra note 11, at 857.

[28] See id. at 854.

[29] Thanks to Lee Epstein for the suggestion.

[30] This exceptionally high fraction of claimants who were appellants is independently interesting and subject to competing explanatory theories—once again including litigant behavior and judicial attitudes.

[31] When we cluster standard errors on judges instead of cases, the standard errors are smaller. Consequently, the difference between commercial speech and gun rights claims becomes significant at p < 0.05 or p < 0.01 in all of our models with errors clustered on judges.

[32] When we cluster errors on judges, pro se status and interest group claimant participation are significant at p < 0.05 in some of our models.

[33] Cf. Ruben & Blocher, supra note 6, at 1505 (comparing judgments on Second Amendment challenges to federal, state, and local actions in federal and state appellate courts). When we add our panel effects variable, see supra note 24, the local action variable does not reach p < 0.05, while the state action variable remains at p < 0.01.

[34] We ran Models 1–6 with abortion rights claims as the baseline and errors clustered on judges, which we have omitted in the interest of brevity. Those results indicate that our independent variables do erase several apparent differences between case sets. Once we introduce the claimant-appellant variable in Model 3, only gun rights claims perform differently (here, worse) compared to abortion rights claims at p < 0.05. In Models 3, 4, and 5, abortion rights claims are not significantly different in their likelihood of attracting judge votes compared to commercial speech, Establishment Clause, or anti-affirmative action claims.

[35] As in the full sample, the difference between commercial speech and gun rights claims is not significant in Model 4 or Model 5 for the Democratic appointees, with errors clustered on cases. With errors clustered on judges, that difference is significant at p < 0.05.

[36] When errors are clustered on judges, there is one model (Model 6) in which Republican appointees are more likely to support Establishment Clause claims than gun rights claims at p < 0.05.

[37] See supra note 24. In both subsamples, the same variables are significant (or not) at the p < 0.05 threshold when we add our panel effects variable to Model 6.

[38] See, e.g., Sunstein et al., supra note 18, at 48–50 (indicating that federal circuit judges voted to support commerce clause challenges less than 10 percent of the time); Krier & Sterk, supra note 15, at 58 (reporting similarly low success rates for regulatory takings claims in state courts); Nancy Leong, Making Rights, 92 B.U. L. Rev. 405, 428–29 & tbl.6 (2012) (reporting success rates for subcategories of Fourth Amendment claims in published circuit court opinions, some of which were around 20 percent); see also Nancy J. King, Non-Capital Habeas Cases After Appellate Review: An Empirical Analysis, 24 Fed. Sentencing Rptr. 308, 317 (2012) (reporting success rates under 1 percent).

Is The Second Amendment A Second Class Right?

Volume 68 September 2018
IS THE SECOND AMENDMENT A SECOND CLASS RIGHT

Adam M. Samaha & Roy Germano†. Inez Milholland Professor of Civil Liberties, New York University School of Law; Research Scholar, New York University School of Law. Thanks to Joseph Blocher, Jessica Bulman-Pozen, Emiliano Catan, Phil Cook, Lee Epstein, Jonah Gelbach, Eric Ruben, and Laura Weinrib for their thoughts on this project. Peter Devlin provided invaluable research assistance. Mistakes are ours.

Ten years have passed since the Supreme Court endorsed a new line of constitutional litigation in District of Columbia v. Heller. [1][1]. District of Columbia v. Heller, 554 U.S. 570, 635–36 (2008) (invalidating the District’s ban on handgun possession in the home). Without much delay, the Court extended Heller’s right to keep and bear arms to state and local government action. [2][2]. See McDonald v. City of Chicago, 561 U.S. 742, 791 (2010) (plurality opinion) (addressing handgun restrictions similar to those declared invalid in Heller). This suggested that significant matters of policy could be at stake. [3][3]. See Philip J. Cook, Jens Ludwig & Adam M. Samaha, Gun Control After Heller: Threats and Sideshows from a Social Welfare Perspective, 56 UCLA L. Rev. 1041, 1069–71 (2009). But the Court has done nearly nothing else to elaborate. [4][4]. An arguable exception is Caetano v. Massachusetts, 136 S. Ct. 1027, 1028 (2016) (per curiam) (rejecting three reasons for upholding a stun-gun ban, and remanding). And the Court’s position—probably a work in progress anyway, even without new appointees—was open to unexciting interpretations. [5][5]. See Cook et al., supra note 3, at 1061–64. Perhaps Heller’s slim majority would end up objecting to only the most extreme forms of gun control that happened to survive the usual political opposition.

In the meantime, other people had to handle the ensuing litigation. Lower federal courts and state courts have now fielded more than a thousand Second Amendment claims, as documented in Eric Ruben and Joseph Blocher’s impressive research. [6][6]. See Eric Ruben & Joseph Blocher, From Theory to Doctrine: An Empirical Analysis of the Right to Keep and Bear Arms After Heller, 67 Duke L.J. 1433, 1455 (2018). Those courts have shouldered the tasks of clearing cases and developing doctrine. Not without criticism, of course. For observers who remain unconvinced that there is or ever was a federal constitutional right to keep and bear arms unrelated to militia service, any effort spent in this field of litigation is a dead-weight loss. Any claimant victory is an unearned gratuity, on this view.

The opposite complaint probably receives more attention, however. After all, lower federal and state courts usually reject gun rights claims when they reach the merits. Ruben and Blocher’s study reinforces this impression with numbers. [7][7]. See id. at 1447. Indeed, an allegedly deep judicial opposition to gun rights has become an argument for renewed Supreme Court attention. [8][8]. See id. at 1447–51 (collecting sources). A few Justices have alleged, with memorable phrasing, that judicial mistreatment is turning the Second Amendment into a “second-class right.” [9][9]. E.g., Silvester v. Becerra, 138 S. Ct. 945, 952 (2018) (Thomas, J., dissenting from denial of certiorari) (charging lower courts and the Supreme Court with treating the Second Amendment as a disfavored right, and quoting McDonald v. City of Chicago, 561 U.S. 742, 780 (2010) (asserting that Chicago’s view would effectively convert the Amendment into a “second-class right”)); Peruta v. California, 137 S. Ct. 1995, 1999 (2017) (Thomas, J., joined by Gorsuch, J., dissenting from denial of certiorari) (similar); Friedman v. City of Highland Park, 136 S. Ct. 447, 449 (2015) (Thomas, J., joined by Scalia, J., dissenting from denial of certiorari) (similar). Silvester involved a waiting period, Peruta involved public carry, and Friedman involved an assault-weapons ban.

Is this true? A second-class complaint is normative insofar as it maintains that judges are not giving gun rights adequate protection. [10][10]. Accord Ruben & Blocher, supra note 6, at 1449–50, 1507 (discussing underenforcement complaints). Like Ruben and Blocher, our goal here is not to revisit the normative commitments that underlie that conclusion. We can, nonetheless, work on the empirical element of a second-class complaint. The complaint is factual and relative insofar as it compares how judges treat different subcategories of constitutional claiming.

To be sure, particularists may deny that a subcategory of legal claim can be usefully compared to another subcategory “all else equal.” Subcategories of claims are, by definition, somehow different from each other. Usually we cannot switch the constitutional clause cited and leave everything else about a case the same. But for observers who are comfortable aggregating individual claims into subcategories of litigation—people who must include those making second-class complaints—we can offer some data.

Below we compare five fields of high-profile constitutional litigation in the United States Courts of Appeals: gun rights, abortion rights, affirmative action, establishment of religion, and commercial speech. We do see different rates of success in attracting judge votes across some of these case sets, with gun rights claims at the low end. We are unable to erase or fully explain the relatively poor performance of gun rights claims by introducing several independent variables. And judges appointed by Democrats might contribute more to the disparities than judges appointed by Republicans, although neither set of judges seems to support all claims at equal rates.

Nevertheless, we cannot make much progress on why gun rights claims might fare worse than certain other claims. The second-class complaint points at judges. But plausible explanations are available that have nothing to do with judges disliking gun rights, [11][11]. See Adam M. Samaha & Roy Germano, Are Commercial Speech Cases Ideological? An Empirical Inquiry, 25 Wm. & Mary Bill Rts. J. 827, 860–61 & n.176 (2017) (offering preliminary and possible explanations for different win rates across claim categories). and existing data cannot rule out those alternatives. Nor is it clear that more data like ours will solve the problem of isolating the contribution of judicial preferences to case results across fields of litigation. More fully detailing and explaining the forces at play must await future work, a significant part of which will have to be theoretical, historical, and ethnographic.

I. Comparisons and Complications

Litigation success rates vary across fields of litigation. Researching that much is relatively straightforward, when the fields are defined well enough and the search or sample of cases is adequate. We report some success rates below. But explaining litigation success rates is much harder. Long streams of intertwined events occur before any judge enters the scene of a case, although previous judicial decisions might well influence these events. People develop various policy commitments, policy makers decide whether and how to regulate, lawyers and clients decide whether and how to litigate, judges decide whether and how to guide other actors through doctrine in written opinions. These events shape the set of claims that judges see.

We should be cautious, therefore, in using litigation success rates to draw inferences about judge preferences or decision standards. [12][12]. See Theodore Eisenberg, Testing the Selection Effect: A New Theoretical Framework with Empirical Tests, 19 J. Legal Stud. 337, 337–38, 340 (1990) (discussing selection effects that can defeat inferences about whether the rules are “skewed”). This should be a lesson from the sophisticated literature on selection effects in litigation. A well-known early effort to model litigation and settlement choices is George L. Priest & Benjamin Klein, The Selection of Disputes for Litigation, 13 J. Legal Stud. 1, 20–26 (1984) (indicating that, when the stakes are symmetric, the plaintiff win rate at trial will tend toward 50 percent regardless of the decision standard). Reconsiderations include Eisenberg, supra, at 338–40, 352, 355–56 (finding varying trial win rates in federal court data, without rejecting selection effect theories more generally); Steven Shavell, Any Frequency of Plaintiff Victory at Trial Is Possible, 25 J. Legal Stud. 493 (1996) (warning against the suggested 50-percent tendency under the Priest-Klein model or under an asymmetric information model); Jonah B. Gelbach, The Reduced Form of Litigation Models and the Plaintiff’s Win Rate 16–18 (Univ. of Penn. Inst. For Law & Econ., Research Paper No. 16-22), available at https://ssrn.com/abstract=2837154 [https://perma.cc/8QV5-SU35] (extending Shavell’s warning to a range of litigation models, including some that assume very accurate party information); and Daniel Klerman & Yoon-Ho Alex Lee, Inferences from Litigated Cases, 43 J. Legal Studies 209, 210–11, 214 (2014) (contending that, under certain assumptions and models, one may cautiously draw inferences about the decision standard from the trial win rate). Caution is warranted even if formal law seems vague in a way that invites judicial discretion. [13][13]. We continue to doubt that uncertainty about current law, alone, predicts low success rates for claimants. See Samaha & Germano, supra note 11, at 860–61 n.176. In fact, the litigation success rate could be low, middling, or high while the judicial decision standard for supporting the claims is low, middling, or high. [14][14]. The “decision standard” could be based on extralegal judicial preferences, valid legal sources, or both. We bracket the possibility of judges automatically opposing a given subcategory of claim. Consider gun rights claims before Heller, or abortion rights claims before Roe v. Wade, 410 U.S. 113 (1973). Much depends on how non-judges think and act.

If claimants are willing and able to litigate assertively, pushing the composition of claims toward those that are unlikely to persuade judges yet valuable to the claimants nonetheless, then we should expect a downward pull on the success rate. For example, cash-flush organizations might litigate improbable claims to deter regulatory efforts; clients and cause lawyers might want to voice their positions in litigation even if, or because, judges are likely to disagree; and optimistic parties might hope for rescue from the Supreme Court. [15][15]. See Samaha & Germano, supra note 11, at 861 n.176 (discussing deterrence through litigation and other phenomena); see also Ben Depoorter, The Upside of Losing, 113 Colum. L. Rev. 817, 820–21 (2013) (discussing how unfavorable court outcomes might mobilize political support); cf. James E. Krier & Stewart E. Sterk, An Empirical Study of Implicit Takings, 58 Wm. & Mary L. Rev. 35, 65 (2016) (suggesting asymmetric stakes, deterring government regulators, high settlement costs where government lawyers cannot bind their clients, and asymmetric information to explain low success rates for certain regulatory takings claims in state courts). Likewise, if governments are unwilling or unable to regulate, enforce, or litigate assertively when their odds in court seem long, then we should expect another downward pull on the claimant success rate. Judges would see fewer vulnerable targets. On the flip side, assertive governments and cautious claimants should have an upward pull on that rate.

If this non-judicial behavior were stable and similar across fields, we could draw stronger inferences about judges by comparing litigation success rates. Different rates would be more easily attributed to judges, though we could still wonder about the influence of law’s sources as opposed to extralegal preferences. But of course the relevant behavior might change over time, it might vary across fields of litigation, and it might influence litigation success rates even if all judges are otherwise equally receptive to supporting any subcategory of claim.

Absent an independent measure of claim quality, we can gather suggestive data on judges, claimants, and regulators. But our aspirations regarding explanations must be modest. To date we lack direct measures across case sets for trial-level judgments and settlements; filing and appeal rates; litigation costs and available resources; perceived stakes; severity of regulation and enforcement; the formal content of constitutional law at the time of decision; and so on. Moreover, the plausible influences on case results are complex when taken together, aside from the data collection challenges. No feasible study can account for all potential influences. Instead we can make incremental—and hopefully provocative—advances with available data. [16][16]. We greatly appreciate Ruben and Blocher’s efforts to build a large dataset and to offer initial analysis without overly firm conclusions. Given the plausible causation theories and limited information, however, we are not quite ready to agree that their “data shows that the low rate of success probably has more to do with the claims being asserted than with judicial hostility to the right.” Ruben & Blocher, supra note 6, at 1507.

II. Data

We rely on five existing case sets, [17][17]. See Samaha & Germano, supra note 11, at 843–47 (describing the original sets); Adam M. Samaha, Codebook: Constitutional Case Sets App’x A (Aug. 13, 2018) (on file with the authors and the Duke Law Journal) (describing search queries and grounds for exclusion of cases). which we have extended through the end of 2016. The aim has been to gather the universe of federal circuit court decisions on the merits of various constitutional claims, in the mold of prior research by Lee Epstein, Cass Sunstein, and their teams. [18][18]. See generally Lee Epstein, William M. Landes & Richard A. Posner, The Behavior of Federal Judges (2013); Cass R. Sunstein, David Schkade, Lisa M. Ellman & Andres Sawicki, Are Judges Political? An Empirical Analysis of the Federal Judiciary (2006). Ruben and Blocher graciously helped us round out the gun rights case set, while Sepehr Shahshahani and Lawrence Liu shared data on recent Establishment Clause appeals. [19][19]. See generally Sepehr Shahshahani & Lawrence J. Liu, Religion and Judging on the Federal Courts of Appeals, 14 J. Empirical Legal Stud. 716 (2017). The Establishment Clause case set is less comprehensive than the others: fewer of the unpublished decisions have been gathered, which could leave the success rate for that case set artificially high. [20][20]. Following the protocol of an earlier study, see Gregory C. Sisk & Michael Heise, Ideology “All the Way Down”? An Empirical Study of Establishment Clause Decisions in the Federal Courts, 110 Mich. L. Rev. 1201, 1207–08 & n.27 (2012), we gathered Establishment Clause cases from searches of opinions with digests on Westlaw. Not every unpublished decision has a Westlaw digest. We presume that unpublished decisions reject constitutional claims more often than do published opinions. On the other hand, Shahshahani and Liu, supra note 19, at 723, left in some cases decided on justiciability grounds. The five case sets are broadly similar in scope, nonetheless.

Our analysis examines cases decided during and after 2008, the year in which Heller was issued. We exclude en banc decisions to concentrate on the ordinary three-judge panel situation. [21][21]. On the assumption of random assignment in this setting, see Adam S. Chilton & Marin K. Levy, Challenging the Randomness of Panel Assignment in the Federal Courts of Appeals, 101 Cornell L. Rev. 1, 3–7 & n.4 (2015) (concluding that the circuit courts do not always randomly assign judges to three-judge panels for oral argument, even if they randomly assign cases to panels). We also exclude criminal and habeas corpus appeals to concentrate on civil litigation in which claimants and lawyers tend to be more free to pick their targets. Gun rights cases include a large fraction of criminal-defendant claimants, [22][22]. See Samaha & Germano, supra note 11, at 860. who may have appointed counsel and who often adopt implausible legal positions when facing serious penalties. The other four case sets have nowhere near this fraction of criminal and habeas appeals. [23][23]. Perhaps the criminal cases generate precedent that drags down the civil claims. This effect has been suggested in the Fourth Amendment setting. See Nancy Leong, Improving Rights, 100 Va. L. Rev. 377, 393, 473–74 (2014).

We model the likelihood of a circuit judge voting in favor of a claim or claims in a case. A judge is counted as voting “in favor” if he or she supported at least part of at least one relevant claim in the case at hand. Many cases are understood by judges to present only one constitutional claim, some cases multiple claims. We simplify those differences by leaving claims within single-case packages that are up for one vote, while breaking out individual judge votes on those packages.

In contrast, Ruben and Blocher break out claims from cases. They track court judgments on various gun rights claims, along with the associated content of court opinions, without analyzing the behavior of individual judges. Ideally, we would take the deep coding of opinions in the Blocher and Ruben study, add data on the participating judges, and then compare gun rights with other case sets at both the judge level and the judgment level. Unfortunately, other case sets are not coded like Ruben and Blocher’s extensive dataset, and some of their coding is unique to gun rights doctrine. For now, we persist with our judge-vote approach so that we may begin to make timely comparisons.

III. Models

Our starting point, Model 1, combines observations on judge votes from all five case sets, using dummy variables to mark each set. Subsequent models add control variables related to the judges, the claimants, and the government action under challenge. All models include circuit fixed effects to help control for unmeasured regional variation in regulation, litigation, and judicial practices. Our reported models cluster standard errors on cases, on the theory that votes of judges on the same panel might not be independent. This is a relatively conservative choice for establishing statistical significance in our dataset. In alternative specifications, we have clustered errors on judges.

On the judges, Model 2 adds the party of the appointing president, as a proxy for judge ideology or party identification. For two case sets, we recoded this binary variable so that a 1 consistently indicates expected support for the claim, based on what we take to be conventional understandings of ideological commitments in the United States for this timeframe. [24][24]. Ordinarily, 0 indicates that the judge was nominated by a Democratic president and 1 indicates that the judge was nominated by a Republican. We recoded the abortion rights and Establishment Clause case sets so that Democratic nominees got the value 1 and Republican nominees got the value 0. For simplicity and because our aspirations are modest, we have not interacted judge party with the case set variables.
In unreported models, we have added a variable for partisan panel effects: the fraction of other judges on the panel who were appointed by Republican presidents—or instead, for the abortion rights and Establishment Clause case sets, those appointed by Democratic presidents.
Our models will not reveal whether judicial ideology is a good predictor within each case set or a better predictor in some case sets. Instead, the judge party variable helps control for the possibility that some case sets were assigned a mix of judges who were more likely to evaluate the claims favorably.

On the claimants, Model 3 adds a variable for whether the claimant was the appellant. The courts of appeals tend to affirm. [25][25]. Possible explanations include formal standards of review, deference to trial judge work, trial judge votes being predictive of appellate judge votes, and appellants facing low incremental costs for appeals. See Kevin M. Clermont & Theodore Eisenberg, Litigation Realities, 88 Cornell L. Rev. 119, 150–52 (2002). If claimants in some case sets are more likely to be appellants, they might be more likely to lose, all else equal. And eagerness to appeal might reveal more about the claimants than the judges. That said, trial judges might be especially opposed to some subcategories of claims, which would generate a larger fraction of disappointed claimants who might then appeal. We lack comprehensive data on how trial judges treat constitutional claims and how often parties appeal.

Model 4 includes pro se status. Pro se claimants usually are at a disadvantage, at least if judges do not compensate by lowering their standards for pro se claims, [26][26]. Cf. D. James Greiner & Cassandra Wolos Pattanayak, Randomized Evaluation in Legal Assistance: What Difference Does Representation (Offer and Actual Use) Make?, 121 Yale L.J. 2118, 2174 (2012) (discussing possibilities in the context of administrative adjudication of unemployment benefits claims). and the fraction of people doggedly willing to pursue appeals without a lawyer will vary across fields. Model 4 also includes variables to indicate whether at least one of the constitutional claimants was an individual, a business, or an interest group. Businesses and interest groups might tend to have greater resources for litigation or be more sophisticated at litigation decisionmaking, compared to individuals. Nonetheless, sometimes organizations will be content with a litigation loss that can be converted into future support. As well, we have not distinguished cases in which individuals joined with businesses or interest groups.

On government action, Model 5 adds a variable for whether federal, state, or local action was challenged. A sensible theory is that the federal government will tend to perform better in litigation than state and local governments, and that states will tend to do better than locals. [27][27]. See Samaha & Germano, supra note 11, at 857. One reason is disparity in litigation resources when these governments are litigants, which we are not coding directly. Another reason might be that national and even statewide politics tend to yield policies and practices that are less extreme, more carefully advised, and easier to defend in court. Different levels of government aggregate information and political forces in unique ways. However, it is also possible that claimants and lawyers will select their targets such that any differences across levels of government will not reappear in cases that are filed and appealed. Or the selection effects could vary across case sets.

Model 6 introduces a variable regarding Supreme Court cases. Lower court judges pay attention to messages from the Supreme Court, at least in explaining their decisions and sometimes in reaching them. We collected Supreme Court cases within each case set, coded them as either supporting or rejecting the relevant claim(s), and created a continuous score based on citation counts in the circuit courts. [28][28]. See id. at 854. This variable is not designed to take judges out of the equation, obviously. The attempt is to examine the Supreme Court’s influence on circuit judge voting. The Court’s recent support, opposition, or silence regarding certain constitutional claims might make the success of such claims in the circuit courts less or more impressive.

Finally, we split the sample along party lines. [29][29]. Thanks to Lee Epstein for the suggestion. We estimated our models on the subsample of judges appointed by Democratic presidents, and then separately on the subsample of judges appointed by Republicans. Any differing levels of judicial support across case sets might arise from the voting patterns of Republican appointees, Democratic appointees, or both. However, splitting the sample this way will not identify the influence of unmeasured regulatory and litigation behaviors. Those non-judge behaviors might or might not differ across case sets, whether or not the claim types themselves induce different voting behavior from different judges.

IV. Results

Table 1 presents some characteristics of the claims, claimants, and judges in our dataset. Gun rights claims had the lowest fraction of judge support (18 percent), and they were pursued by the highest fractions of pro se litigants (14 percent) and appellants who lost in the proceedings below (94 percent, remarkably). [30][30]. This exceptionally high fraction of claimants who were appellants is independently interesting and subject to competing explanatory theories—once again including litigant behavior and judicial attitudes. Abortion rights claims are at the opposite ends of those measures. So, perhaps disadvantaged categories of litigants will help explain the apparently different rates of success across claims. Note that we found far fewer cases that resolved abortion rights or anti-affirmative action claims on the merits, compared to the other case sets. The relative rarity and success of these claims might not be a coincidence, though the small numbers limit what we can learn.

Binomial logit estimates are reported in Table 2. In these models, we made gun rights claims our baseline. This allows us to compare the probability of a judge voting in favor of gun rights claims to the probabilities in each of the other four sets of claims. On the whole, these models show that judges are significantly more likely to vote for claims in the other four case sets. This finding is robust to the inclusion of a number of available control variables, including the party of the president who appointed the judge (Model 2), several claimant characteristics (Model 3 and Model 4), whether federal, state, or local government action is being challenged (Model 5), and recent decisions from the Supreme Court (Model 6). The results for commercial speech claims are not fully consistent across models, however. In Model 4 and Model 5, the differences between commercial speech and gun rights claims are not statistically significant. Those results are, in turn, sensitive to how standard errors are clustered. [31][31]. When we cluster standard errors on judges instead of cases, the standard errors are smaller. Consequently, the difference between commercial speech and gun rights claims becomes significant at p < 0.05 or p < 0.01 in all of our models with errors clustered on judges. In any event, gun rights claims generally underperform.

Encouragingly, several control variables are significant in the expected directions. The judge party variable is consistently positive and statistically significant at p < 0.01, while the claimant appellant variable is consistently negative at p < 0.05 or p < 0.01. Interestingly, pro se status is not correlated with judge votes, all else equal. Nor do we find significant associations between judge votes and business claimants or interest group claimants, compared to individual claimants. [32][32]. When we cluster errors on judges, pro se status and interest group claimant participation are significant at p < 0.05 in some of our models. But claims do seem to fare better against local and especially state action, compared to federal action. [33][33]. Cf. Ruben & Blocher, supra note 6, at 1505 (comparing judgments on Second Amendment challenges to federal, state, and local actions in federal and state appellate courts). When we add our panel effects variable, see supra note 24, the local action variable does not reach p < 0.05, while the state action variable remains at p < 0.01. And our Supreme Court variable is positive and significant at p < 0.01. Recent messages from the Supreme Court may therefore affect voting in the Courts of Appeals. Even so, gun rights claims still seem to lag behind. [34][34]. We ran Models 1–6 with abortion rights claims as the baseline and errors clustered on judges, which we have omitted in the interest of brevity. Those results indicate that our independent variables do erase several apparent differences between case sets. Once we introduce the claimant-appellant variable in Model 3, only gun rights claims perform differently (here, worse) compared to abortion rights claims at p < 0.05. In Models 3, 4, and 5, abortion rights claims are not significantly different in their likelihood of attracting judge votes compared to commercial speech, Establishment Clause, or anti-affirmative action claims.

This gun-rights lag might be more attributable to the votes of Democratic appointees. As noted above, we split the sample into Democratic appointees and Republican appointees and reestimated the models separately. In Table 2, we display results for these subsamples using Model 6. Results for the Democratic appointees generally mirror what we found in the full sample: these judges are significantly more likely to support the other four types of claims compared to gun rights claims. Even anti-affirmative action claims are more likely to attract votes from Democratic appointees. [35][35]. As in the full sample, the difference between commercial speech and gun rights claims is not significant in Model 4 or Model 5 for the Democratic appointees, with errors clustered on cases. With errors clustered on judges, that difference is significant at p < 0.05.

Republican appointees seem different. When we look only at the votes of Republican appointees, we find that these judges are not significantly more likely to support other types of claims compared to gun rights claims—with the notable exception of anti-affirmative action claims. Like their Democratic counterparts, Republican appointees seem significantly more likely to support challenges to affirmative action programs than challenges to firearms restrictions. [36][36]. When errors are clustered on judges, there is one model (Model 6) in which Republican appointees are more likely to support Establishment Clause claims than gun rights claims at p < 0.05. Adding a variable for panel effects does not affect these conclusions. [37][37]. See supra note 24. In both subsamples, the same variables are significant (or not) at the p < 0.05 threshold when we add our panel effects variable to Model 6.

V. Reflections

The judicial treatment of gun rights claims is, in some respects, remarkable and a bit mysterious. We have little doubt that gun rights claims have been attracting a lower percentage of votes on the merits from circuit judges in civil cases since Heller, compared to the other four claim types in our dataset. Our case collection effort was thorough enough to assert that much. But we cannot make claims about other case sets or other courts. Low litigation success rates have been reported in other studies, after all. [38][38]. See, e.g., Sunstein et al., supra note 18, at 48–50 (indicating that federal circuit judges voted to support commerce clause challenges less than 10 percent of the time); Krier & Sterk, supra note 15, at 58 (reporting similarly low success rates for regulatory takings claims in state courts); Nancy Leong, Making Rights, 92 B.U. L. Rev. 405, 428–29 & tbl.6 (2012) (reporting success rates for subcategories of Fourth Amendment claims in published circuit court opinions, some of which were around 20 percent); see also Nancy J. King, Non-Capital Habeas Cases After Appellate Review: An Empirical Analysis, 24 Fed. Sentencing Rptr. 308, 317 (2012) (reporting success rates under 1 percent).

Nor can we give a complete explanation for the differences that we do see. We controlled for a limited number of variables. Gun regulation and litigation choices could be distinctive in ways that pull down the probability of litigation success, and in ways that we have not measured. Judicial decision standards certainly could be part of the explanation; our subsample results are provocative in this regard. But finding distinctive voting patterns among Democratic and Republican appointees is closer to the beginning of the investigation than the end.

Moreover, last place doesn’t necessarily mean second class. Conceptual challenges arise when cases are compared across familiar boundaries, as we noted up front. People need not accept that waiting periods for firearms and abortion services are essentially the same phenomenon, or that the bundle of claims against advertising regulations can be equated with any other bundle of constitutional claims. Furthermore, for those who do appreciate comparisons among case sets, differences in success rates might be welcome. Standing alone, differences do not prove that judges are erring. The appropriate response depends on one’s well-grounded normative position on law and the judicial role in these contested fields. And if equalization of success rates is justified, a normative framework is required to tell which claims should have their rates leveled up as opposed to leveled down. Our study is meant to develop hard questions about constitutional litigation, not to prescribe reforms.

To learn more, additional data is welcome. For example, information on the range of parties, amici, and attorneys in trial and appellate litigation would help, as would proxies for the assertiveness of regulation across fields. We should develop a clearer picture of whether, for instance, abortion rights and anti-affirmative action claimants generally are cautious parties who face envelope-pushing officials; and whether gun rights advocates and commercial advertisers generally are deep-pocketed or ideologically committed parties who face shy regulators. Broader and automated data collection will contribute to this learning.

Other kinds of research might be even more valuable at this stage, however. Histories and case studies of litigation decisions and the surrounding forces can advance our theories and inform further efforts. True, the key decisions and motivations often are difficult for outside observers to document and understand, even when few people are involved. Nonetheless, scholars in more than one discipline have produced detailed investigations into various social movements and legal reform efforts. More such work can be done for gun rights and other fields to help us understand when, how, and why people turn to constitutional litigation. As we better understand those upstream forces, we will better understand the judiciary’s reactions.

Conclusion

The Supreme Court will take up another gun rights case at some point and it will have plenty of opportunities to do so. Gun rights claimants aren’t going anywhere. But whatever the Court might want to achieve by granting review, success rates in lower courts are not sensible grounds on which to make decisions unless someone can explain those rates convincingly. Presumably the Court does not want to reward any class of petitioners for flooding lower courts with weak claims, whether those claims are for gun rights, habeas corpus relief, or anything else. Anyway, the Justices probably can make reasonable decisions without determining which sets of petitioners are the most disadvantaged classes in litigation.

Still, for anyone interested in the operation and effects of constitutional litigation, the measurement and explanation of courtroom success rates is an ongoing challenge. The scholarly understanding of gun rights litigation is hardly second class, but it probably isn’t yet first rate. In this paper, we offer some new results, ideas, and ways forward. Ruben and Blocher’s study contributes to the stock of data, too. And, in the years ahead, we will learn even more if we can combine a variety of research methods—relying on a blend of scholars who try to understand not only litigated outcomes but also the confluence of upstream forces that constrain, propel, and shape constitutional claims.

 

Table 1: Characteristics of Claimants, and Judges

Click Image to Enlarge

 

Table 2: Circuit Judge Votes for Claims, in Full or in Part

Click Image to Enlarge


Copyright© 2018 Adam M. Samaha & Roy Germano.

Inez Milholland Professor of Civil Liberties, New York University School of Law; Research Scholar, New York University School of Law. Thanks to Joseph Blocher, Jessica Bulman-Pozen, Emiliano Catan, Phil Cook, Lee Epstein, Jonah Gelbach, Eric Ruben, and Laura Weinrib for their thoughts on this project. Peter Devlin provided invaluable research assistance. Mistakes are ours.

[1] District of Columbia v. Heller, 554 U.S. 570, 635–36 (2008) (invalidating the District’s ban on handgun possession in the home).

[2] See McDonald v. City of Chicago, 561 U.S. 742, 791 (2010) (plurality opinion) (addressing handgun restrictions similar to those declared invalid in Heller).

[3] See Philip J. Cook, Jens Ludwig & Adam M. Samaha, Gun Control After Heller: Threats and Sideshows from a Social Welfare Perspective, 56 UCLA L. Rev. 1041, 1069–71 (2009).

[4] An arguable exception is Caetano v. Massachusetts, 136 S. Ct. 1027, 1028 (2016) (per curiam) (rejecting three reasons for upholding a stun-gun ban, and remanding).

[5] See Cook et al., supra note 3, at 1061–64.

[6] See Eric Ruben & Joseph Blocher, From Theory to Doctrine: An Empirical Analysis of the Right to Keep and Bear Arms After Heller, 67 Duke L.J. 1433, 1455 (2018).

[7] See id. at 1447.

[8] See id. at 1447–51 (collecting sources).

[9] E.g., Silvester v. Becerra, 138 S. Ct. 945, 952 (2018) (Thomas, J., dissenting from denial of certiorari) (charging lower courts and the Supreme Court with treating the Second Amendment as a disfavored right, and quoting McDonald v. City of Chicago, 561 U.S. 742, 780 (2010) (asserting that Chicago’s view would effectively convert the Amendment into a “second-class right”)); Peruta v. California, 137 S. Ct. 1995, 1999 (2017) (Thomas, J., joined by Gorsuch, J., dissenting from denial of certiorari) (similar); Friedman v. City of Highland Park, 136 S. Ct. 447, 449 (2015) (Thomas, J., joined by Scalia, J., dissenting from denial of certiorari) (similar). Silvester involved a waiting period, Peruta involved public carry, and Friedman involved an assault-weapons ban.

[10] Accord Ruben & Blocher, supra note 6, at 1449–50, 1507 (discussing underenforcement complaints).

[11] See Adam M. Samaha & Roy Germano, Are Commercial Speech Cases Ideological? An Empirical Inquiry, 25 Wm. & Mary Bill Rts. J. 827, 860–61 & n.176 (2017) (offering preliminary and possible explanations for different win rates across claim categories).

[12] See Theodore Eisenberg, Testing the Selection Effect: A New Theoretical Framework with Empirical Tests, 19 J. Legal Stud. 337, 337–38, 340 (1990) (discussing selection effects that can defeat inferences about whether the rules are “skewed”). This should be a lesson from the sophisticated literature on selection effects in litigation. A well-known early effort to model litigation and settlement choices is George L. Priest & Benjamin Klein, The Selection of Disputes for Litigation, 13 J. Legal Stud. 1, 20–26 (1984) (indicating that, when the stakes are symmetric, the plaintiff win rate at trial will tend toward 50 percent regardless of the decision standard). Reconsiderations include Eisenberg, supra, at 338–40, 352, 355–56 (finding varying trial win rates in federal court data, without rejecting selection effect theories more generally); Steven Shavell, Any Frequency of Plaintiff Victory at Trial Is Possible, 25 J. Legal Stud. 493 (1996) (warning against the suggested 50-percent tendency under the Priest-Klein model or under an asymmetric information model); Jonah B. Gelbach, The Reduced Form of Litigation Models and the Plaintiff’s Win Rate 16–18 (Univ. of Penn. Inst. For Law & Econ., Research Paper No. 16-22), available at https://ssrn.com/abstract=2837154 [https://perma.cc/8QV5-SU35] (extending Shavell’s warning to a range of litigation models, including some that assume very accurate party information); and Daniel Klerman & Yoon-Ho Alex Lee, Inferences from Litigated Cases, 43 J. Legal Studies 209, 210–11, 214 (2014) (contending that, under certain assumptions and models, one may cautiously draw inferences about the decision standard from the trial win rate).

[13] We continue to doubt that uncertainty about current law, alone, predicts low success rates for claimants. See Samaha & Germano, supra note 11, at 860–61 n.176.

[14] The “decision standard” could be based on extralegal judicial preferences, valid legal sources, or both. We bracket the possibility of judges automatically opposing a given subcategory of claim. Consider gun rights claims before Heller, or abortion rights claims before Roe v. Wade, 410 U.S. 113 (1973).

[15] See Samaha & Germano, supra note 11, at 861 n.176 (discussing deterrence through litigation and other phenomena); see also Ben Depoorter, The Upside of Losing, 113 Colum. L. Rev. 817, 820–21 (2013) (discussing how unfavorable court outcomes might mobilize political support); cf. James E. Krier & Stewart E. Sterk, An Empirical Study of Implicit Takings, 58 Wm. & Mary L. Rev. 35, 65 (2016) (suggesting asymmetric stakes, deterring government regulators, high settlement costs where government lawyers cannot bind their clients, and asymmetric information to explain low success rates for certain regulatory takings claims in state courts).

[16] We greatly appreciate Ruben and Blocher’s efforts to build a large dataset and to offer initial analysis without overly firm conclusions. Given the plausible causation theories and limited information, however, we are not quite ready to agree that their “data shows that the low rate of success probably has more to do with the claims being asserted than with judicial hostility to the right.” Ruben & Blocher, supra note 6, at 1507.

[17] See Samaha & Germano, supra note 11, at 843–47 (describing the original sets); Adam M. Samaha, Codebook: Constitutional Case Sets App’x A (Aug. 13, 2018) (on file with the authors and the Duke Law Journal) (describing search queries and grounds for exclusion of cases).

[18] See generally Lee Epstein, William M. Landes & Richard A. Posner, The Behavior of Federal Judges (2013); Cass R. Sunstein, David Schkade, Lisa M. Ellman & Andres Sawicki, Are Judges Political? An Empirical Analysis of the Federal Judiciary (2006).

[19] See generally Sepehr Shahshahani & Lawrence J. Liu, Religion and Judging on the Federal Courts of Appeals, 14 J. Empirical Legal Stud. 716 (2017).

[20] Following the protocol of an earlier study, see Gregory C. Sisk & Michael Heise, Ideology “All the Way Down”? An Empirical Study of Establishment Clause Decisions in the Federal Courts, 110 Mich. L. Rev. 1201, 1207–08 & n.27 (2012), we gathered Establishment Clause cases from searches of opinions with digests on Westlaw. Not every unpublished decision has a Westlaw digest. We presume that unpublished decisions reject constitutional claims more often than do published opinions. On the other hand, Shahshahani and Liu, supra note 19, at 723, left in some cases decided on justiciability grounds.

[21] On the assumption of random assignment in this setting, see Adam S. Chilton & Marin K. Levy, Challenging the Randomness of Panel Assignment in the Federal Courts of Appeals, 101 Cornell L. Rev. 1, 3–7 & n.4 (2015) (concluding that the circuit courts do not always randomly assign judges to three-judge panels for oral argument, even if they randomly assign cases to panels).

[22] See Samaha & Germano, supra note 11, at 860.

[23] Perhaps the criminal cases generate precedent that drags down the civil claims. This effect has been suggested in the Fourth Amendment setting. See Nancy Leong, Improving Rights, 100 Va. L. Rev. 377, 393, 473–74 (2014).

[24] Ordinarily, 0 indicates that the judge was nominated by a Democratic president and 1 indicates that the judge was nominated by a Republican. We recoded the abortion rights and Establishment Clause case sets so that Democratic nominees got the value 1 and Republican nominees got the value 0. For simplicity and because our aspirations are modest, we have not interacted judge party with the case set variables.
In unreported models, we have added a variable for partisan panel effects: the fraction of other judges on the panel who were appointed by Republican presidents—or instead, for the abortion rights and Establishment Clause case sets, those appointed by Democratic presidents.

[25] Possible explanations include formal standards of review, deference to trial judge work, trial judge votes being predictive of appellate judge votes, and appellants facing low incremental costs for appeals. See Kevin M. Clermont & Theodore Eisenberg, Litigation Realities, 88 Cornell L. Rev. 119, 150–52 (2002).

[26] Cf. D. James Greiner & Cassandra Wolos Pattanayak, Randomized Evaluation in Legal Assistance: What Difference Does Representation (Offer and Actual Use) Make?, 121 Yale L.J. 2118, 2174 (2012) (discussing possibilities in the context of administrative adjudication of unemployment benefits claims).

[27] See Samaha & Germano, supra note 11, at 857.

[28] See id. at 854.

[29] Thanks to Lee Epstein for the suggestion.

[30] This exceptionally high fraction of claimants who were appellants is independently interesting and subject to competing explanatory theories—once again including litigant behavior and judicial attitudes.

[31] When we cluster standard errors on judges instead of cases, the standard errors are smaller. Consequently, the difference between commercial speech and gun rights claims becomes significant at p < 0.05 or p < 0.01 in all of our models with errors clustered on judges.

[32] When we cluster errors on judges, pro se status and interest group claimant participation are significant at p < 0.05 in some of our models.

[33] Cf. Ruben & Blocher, supra note 6, at 1505 (comparing judgments on Second Amendment challenges to federal, state, and local actions in federal and state appellate courts). When we add our panel effects variable, see supra note 24, the local action variable does not reach p < 0.05, while the state action variable remains at p < 0.01.

[34] We ran Models 1–6 with abortion rights claims as the baseline and errors clustered on judges, which we have omitted in the interest of brevity. Those results indicate that our independent variables do erase several apparent differences between case sets. Once we introduce the claimant-appellant variable in Model 3, only gun rights claims perform differently (here, worse) compared to abortion rights claims at p < 0.05. In Models 3, 4, and 5, abortion rights claims are not significantly different in their likelihood of attracting judge votes compared to commercial speech, Establishment Clause, or anti-affirmative action claims.

[35] As in the full sample, the difference between commercial speech and gun rights claims is not significant in Model 4 or Model 5 for the Democratic appointees, with errors clustered on cases. With errors clustered on judges, that difference is significant at p < 0.05.

[36] When errors are clustered on judges, there is one model (Model 6) in which Republican appointees are more likely to support Establishment Clause claims than gun rights claims at p < 0.05.

[37] See supra note 24. In both subsamples, the same variables are significant (or not) at the p < 0.05 threshold when we add our panel effects variable to Model 6.

[38] See, e.g., Sunstein et al., supra note 18, at 48–50 (indicating that federal circuit judges voted to support commerce clause challenges less than 10 percent of the time); Krier & Sterk, supra note 15, at 58 (reporting similarly low success rates for regulatory takings claims in state courts); Nancy Leong, Making Rights, 92 B.U. L. Rev. 405, 428–29 & tbl.6 (2012) (reporting success rates for subcategories of Fourth Amendment claims in published circuit court opinions, some of which were around 20 percent); see also Nancy J. King, Non-Capital Habeas Cases After Appellate Review: An Empirical Analysis, 24 Fed. Sentencing Rptr. 308, 317 (2012) (reporting success rates under 1 percent).