The Society for Empirical Legal Studies (SELS) was created less than a decade ago to create a forum for scientific research on the law itself, and the Society has grown each year, with now hundreds of submissions from all over the world for its annual conference and flagship journal. Although there are many strands of such research, a primary research question is whether any particular law works to achieve its end, and if so how? Does the death penalty reduce crime? Does medical malpractice reform promote patient safety or lower costs? Do restrictions on the practice of medicine promote health?
Even before SELS was created, scholars in many fields were looking at the law as an independent variable, and looking at various dependent variables that could be used to access their success or failure. Health outcomes present an obvious dependent variable, given its importance for overall welfare and given the rich data available in this sector. Five years ago, the Robert Wood Johnson Foundation created the Public Health Law Research Program (PHLR), and appointed law professor Scott Burris as its director. PHLR is dedicated to “building the evidence base for laws that improve public health. PHLR funds research, improves research methods, and makes evidence more accessible to policy-makers, the media, and the public.” PHLR has helped to create a rich multidisciplinary field of scholars and practitioners engaged in this sort of research to understand the impact of law on health.
This empirical turn in legal scholarship—drawing other scholarly disciplines into the law and developing empirical capacities among legal experts—creates a wonderful opportunity to improve the law itself. As the methods become more rigorous and the breadth of the work grows, it will be possible for lawmakers to make more intelligent decisions about which laws to enact and which to repeal, based on the empirical evidence as to whether they in fact work. In this way, evidence can supplant ideology in the law, just as evidence has supplanted superstition in medicine. In the grand scheme of things, this approach begins to fulfill the aspirations of philosophers like Francis Bacon, Jeremy Bentham, John Dewey—who all sought to bring intelligent evidence into the domain of social policy.
With this background, I turn to the excellent new work of Evan Anderson and Scott Burris. In this paper, Anderson and Burris begin to explore the question of how empirical evidence can feed into the lawmaking process. Assuming that the science has been performed, and that it is good, how can policymakers incorporate those findings to actually improve the laws on the books and the laws as enforced? This inquiry is analogous to the question of “translation” in medicine, the movement of scientific findings from the bench to the bedside.
Anderson and Burris first note that there is a problem of synthesis: it does little good for there to be dozens of research studies performed on a given question, if they are not all published, not all found, and not all commensurable, given their divergent datasets, methods, and interpretations. Thus, the authors are correct to note that there is an urgent need for more synthesis. We have a model for doing so, again from medicine. As Anderson and Burris explain:
The Cochrane Collaborative grew out of Archibald Cochrane’s observations in the early 1970s about the fragmented and often misleading state of knowledge on the effectiveness of maternal health interventions (Cochrane 1972). The Collaborative quickly became an international force in the promotion of rigorous and methodologically consistent reviews. There are currently more than 5,000 systematic reviews covering a wide range of health interventions in the Cochrane collection, each following a standard protocol detailing how the research question was defined, the relevant studies were identified and the overall body of evidence was characterized (Cochrane Collaborative 2013).
In my view, there is an unmet opportunity for legal scholars to be trained in these methods and begin incorporating them systematically. We also need to get the right outlets and incentives for producing this sort of systematic synthesis. For law professors, just as much effort should be directed to systematizing scientific evidence about whether the law works, as is currently directed towards systematizing legal doctrine (and rhetoric) about what the law is. A systematic account of judicial superstitions provides little guidance as to whether the law works, and what the law should be.
That said, the movement towards bringing evidence into lawmaking is fraught with the danger of bias, and junk science, a problem that is exacerbated by the role of self-interested funders and investigators. Some of this biased research will skew the systematic reviews. In particular, much of the recent development in empirical legal methods has focused on adopting more and more sophisticated mathematical methods (drawing especially from the field of econometrics). In my view, this is often a zero-sum game. As the methods become more complex, it simply creates more opportunities for analysts and consumers to exercise discretion in how they model and interpret the data.
Anderson and Burris acknowledge another deep problem with this agenda of using evidence to shape policy. “For all the value of these efforts, a practical paradox confronts exponents of evidence-based public health law: if a legal intervention is truly innovative, there will not yet be direct evidence of its impact. There will be no studies for systematic reviews and syntheses to digest.” In my view, the answer is experimentation, first in the laboratory (using vignettes or other forms of simulation), and then in the field (with the cooperation of actual policymakers or their proxies exercising law-like power in private settings). Aside from its power to test truly novel interventions, experiments often allow robust causal inference, due to random assignment, without the need for complex and discretionary modeling choices. Instead the discretionary choices are more transparently made in the design of the experiment, and those choices are made behind a veil of ignorance, before the data is known.
We should much more often collaborate with policymakers to conduct randomized rollouts of policy interventions, and similar mechanisms, which allow gold-standard scientific inference about the impact of laws on their expressed goals. And today, we see this agenda gaining steam, as the United States Government, following Britain, is implementing randomized controlled trials to test the efficacy of federal policies. Unfortunately, however, experimentation is woefully under-represented in empirical legal scholarship, and policymakers sometimes seem to conceive the enactment of laws as the fulfillment of ideologies (ends-in-themselves), rather than as mere means for promoting welfare or liberty.
One of the greatest contributions of empirical legal research is to simply train analysts and policymakers to ask the right questions. Anderson and Burris demonstrate the importance of careful causal inquiry through “causal maps,” which are simply graphical depictions of pathways for causation, which help distinguish and clarify the potential relationships. Simply seeing these potential pathways can give rise to important research questions and hypotheses. In particular, using the example of traumatic brain injuries in sports and potential regulatory solutions, Anderson and Burris provide a compelling example of how causal mapping can make explicit the empirical questions and enhance our ability to conceive solutions. The causal map allows the analyst to reject the obvious reforms in favor of real solutions that get to the root causes. In this way, training analysts and policymakers to be careful about causal inference about data can also help us understand how policy reforms actually work.
This is an important paper, laying the groundwork for future empirical research and the translation of that research into smarter law and policy. Anderson and Burris help us understand how laws can work to improve the world and how evidence can enhance the chances that laws will actually do so.
A version of this paper appears as “Researchers and Research Knowledge in Evidence-Informed Policy Innovation” in Regulating Tobacco, Alcohol and Unhealthy Foods: The Legal Issues (Tania Voon, Andrew Mitchell & Jonathan Liberman, eds., 2014).
