The Trajectory of Forensics

 

Volume 69 September 2019

Reflection

The Trajectory of Forensics

Peter Neufeld Co-founder of and Special Counsel to the Innocence Project; Partner, Neufeld Scheck & Brustin, LLP. This Reflection piece is adapted from remarks made at a March 6, 2019 conference at Duke Law School, made possible by the Innocence Project, Duke Law School, and the Center for Statistics and Applications to Forensic Evidence (CSAFE), which is funded through Cooperative Agreement #70NANB15H176 between the National Institute of Standards and Technology and Iowa State University, and which includes activities carried out at Carnegie Mellon University, Duke University, University of California, Irvine, and University of Virginia.
PDF

To appreciate the significance of the 2009 report by the National Academy of Sciences addressing fundamental weaknesses in forensic science, it is useful to understand its historical context.[1][1]. Comm. on Identifying the Needs of the Forensic Sci. Cmty., Nat’l Research Council of the Nat’l Acads., Strengthening Forensic Science in the United States: A Path Forward (2009). That context necessitates an interdisciplinary inquiry since there are sociological, psychological, and political dynamics in play. That historical context better equips us to understand both the obstacles that lie ahead and how to respond to them. In this essay, I discuss the state of forensics before the DNA revolution of the late 1980s and 1990s. I then describe how DNA changed our thinking about forensics more broadly. I describe the immediate backdrop before the National Academy of Sciences Committee on Identifying the Needs of the Forensic Science Community was convened and its impact on policy and litigation in the United States.

I. Forensics in the United States Before 2009

Having litigated criminal cases in the late 1970s and the 1980s at the intersection of science and law, first, I hope to give you a sense of the lay of the land at that period.

Forensic Lab Reports.
The content of forensic lab reports consisted of a “thumb up” or a “thumb down.” Typical lab reports would state in sum and substance, “The crime scene latent lift matches the defendant’s print. He is the source”; “To a reasonable scientific certainty, the defendant is the source of the bite on the victim’s leg”; or “That is the defendant’s hair on the victim’s bed.” They were often a paragraph or two in length.

Litigation
. Few, if any, courts in America seriously considered the validity and reliability of the actual methods used in forensics. How often an expert had been accepted by courts was more important than the accuracy of the expert’s method. If bite-mark evidence was proffered, a judge might reason that the expert used a ruler to measure the distances and used a microscope to look at the marks more carefully. Those instruments were well-described in the literature, therefore bite-mark comparison was science despite a paucity of research estimating the accuracy of forensic dentists. That is what decisions were like, not just for bite mark evidence, but for other pattern and impression forensic disciplines as well.

Regulation and legislation
. With the exception of forensic DNA typing, there simply was not any regulation or legislation governing the forensic sciences in America, either at the state level or at the federal level.

Quality assurance and quality control
. Neither quality assurance nor quality control existed in any meaningful way in crime lab settings in the late 1970s or 80s. Although the National Institute of Justice supported some studies looking at proficiency tests, there was no regulation, and there was no legislation. The meager accreditation that existed was voluntary and did not attempt to limit labs to using scientifically valid methods. Why was that the reality? Comparing the state of forensics with that of clinical laboratories at the same time underscores the problem. By the 1970s and early 80s, we already had a well-regulated system of clinical laboratories in America with routine proficiency testing requirements for analysts. [2][2]. For an overview of the creation of the Clinical Laboratories Improvement Act (CLIA), which regulates clinical laboratories at the federal level, see Brandon L. Garrett & Gregory Mitchell, The Proficiency of Experts, 166 U. Pa. L. Rev. 901, 951 (2018) (detailing the creation of the CLIA). See also 42 U.S.C. § 263 (2012) (describing the process for certifying laboratories). Why did we have that federal system of regulation in clinical laboratories but not crime laboratories? I think the reason has to do with the constituency. For clinical medicine, the entire population is a constituent, including, critically, the white middle and upper class. We all care about our health, and, therefore, we have much greater input into legislation, regulation, quality control in clinical medicine, and tests involving middle class people’s health. But in forensic science, the primary constituents were “criminals.” Political actors considered them dispensable, so there was no need to have independent external bodies review the quality of forensics. I do think that the historical disparity is almost that simple.

II. The DNA Revolution

The DNA revolution was transformative in two ways. First, all of a sudden, DNA as a discipline was introduced in the courts, with a foundation in basic and applied research, extensive peer-reviewed publications, and with measurement of uncertainty that could be articulated. There were two National Academy of Science study groups, one in 1992 and the second in 1996. The first set standards for the bench science,[3][3]. See Nat’l Research Council, DNA Technology in Forensic Science 8–9 (1992) (listing recommendations for sound scientific use of DNA analysis in forensic science). and the second set standards for assessing the probative value of the evidence and dealing with statistical issues. [4][4] Nat’l Research Council, The Evaluation of Forensic DNA Evidence 23–25 (1996) (describing guidelines on quality control and quality assurance). For those of us in the legal profession who had no background in science, all of a sudden, we could see this disparity between the scientific validation and rigor of forensic DNA testing as opposed to all other pattern, impression, and trace evidence. It was a real eye-opener.

The second transformative consequence arose from what DNA taught us about the causes of wrongful convictions. Every time we exonerated a previously convicted person by submitting the preserved, old biological evidence to modern DNA testing, we deconstructed the cases to find out what went wrong in the initial criminal investigation and prosecution that contributed to the conviction of an innocent person. [5][5]. Brandon L. Garrett & Peter J. Neufeld, Invalid Forensic Science Testimony and Wrongful Convictions, 95 Va. L. Rev. 1, 1 (2009) (exploring “the forensic science testimony by prosecution experts in the trials of innocent persons, all convicted of serious crimes, who were later exonerated by post-conviction DNA testing”). We learned that in 45 percent of those wrongful conviction cases, other traditional forensic methods were used at the original criminal investigation and trial, which, along with misleading results, contributed in large part to securing those wrongful convictions. [6][6]. Overturning Wrongful Convictions Involving Misapplied Forensics, Innocence Project, https://www.innocenceproject.org/causes/misapplication-forensic-science [https://perma.cc/K22Z-H6BX].

These two consequences of the DNA revolution made many observers think, “We have to look critically at other forensic methods.” Whereas before there was no constituency, people began to care more about the quality of justice and the reliability of our fact-finding, including in plea and trial adjudications. People in the broader scientific community, people in the legal communities, and laypeople all of a sudden started caring about forensic science.

That historical setting then gave rise in 2007 to the National Academy of Sciences creating the Committee. The Committee then undertook the effort to produce a report examining broadly the legitimacy of many non-DNA forensic methods. It would not have happened but for the extraordinary impact that DNA had on criminal justice.

Other participants in this conference discussed how the 2009 Report contributed to systemic and systematic change. [7][7]. See, e.g., Karen Kafadar, Statistics and the Impact of the 2009 NAS Report, 69 Duke L.J. Online 6 (2019). It created strategic opportunities because it created an opening to talk about these issues to the press, in the courts, and in the legislatures. The Innocence Project has a dynamic policy department, working in dozens of states, pressing for reforms designed to reduce the frequency of wrongful convictions. We have created a model statute for “change in science,” which allows a person to get back into court by demonstrating that the scientific community’s understanding of a particular forensic method has changed since that method was used at the original trial or because the testifying expert has learned that the opinions and conclusions originally offered are no longer supported. Because the vulnerability and misapplication of so many forensic methods has been so well established, five states in the last few years have, either through statute or judicial order, created a “change in science” law, so that people can go back into court—even without new DNA testing—to challenge convictions that were secured with unreliable forensics. [8][8]. Id. Texas, California, Wyoming, and Connecticut each enacted statutes, and in Michigan, the vehicle was created through a Supreme Court judicial rule making. [9][9]. Id.

DNA and the 2009 Report have effectively eroded the legal doctrine of finality. That doctrine poses often insurmountable obstacles to convicted persons seeking to get back into court and secure a new trial many years after a conviction. The doctrine is premised on the notion that after many years, witnesses die, evidence is lost or destroyed, and memories fade, and thus, it is unlikely that a second trial would be more reliable than the first. However, recent DNA exonerations demonstrate that newly discovered biological evidence exculpating convicted persons is far more reliable than the eyewitness, confession, jailhouse informant, or lesser forensic evidence used to convict.

Many members of the forensic community have also changed their thinking about the rigor of the work they did in the past. Of course, there are still some who do not know about the Report or, actually worse, do not recognize its legitimacy. But that is not unexpected, because all great movements and all great changes happen gradually. You take two steps forward and a step backward. People have alluded to the need, ultimately, for changing the culture. [10][10]. See Kafadar, supra note 7, at 11 (“We still have to change the culture. It is not a sign of weakness to say you are not 100 percent accurate. It is far better to quantify the realistic errors in procedures than to pretend they are perfect.”). Legislation can accomplish only so much, litigation has limited utility, and the voluntary cooperation of police and prosecutors is not enforceable. Gradually, we have to win the hearts and minds of people working in forensics, in criminal justice, and just people in general. Once those attitudes change, structural reform and more just outcomes will follow. We are now at the beginning of that trajectory, and the National Academy of Sciences Report played a central role in moving us forward.


Copyright © 2019 Peter Neufeld.

Co-founder of and Special Counsel to the Innocence Project; Partner, Neufeld Scheck & Brustin, LLP. This Reflection piece is adapted from remarks made at a March 6, 2019 conference at Duke Law School, made possible by the Innocence Project, Duke Law School, and the Center for Statistics and Applications to Forensic Evidence (CSAFE), which is funded through Cooperative Agreement #70NANB15H176 between the National Institute of Standards and Technology and Iowa State University, and which includes activities carried out at Carnegie Mellon University, Duke University, University of California, Irvine, and University of Virginia.

[1] Comm. on Identifying the Needs of the Forensic Sci. Cmty., Nat’l Research Council of the Nat’l Acads., Strengthening Forensic Science in the United States: A Path Forward (2009).

[2] For an overview of the creation of the Clinical Laboratories Improvement Act (CLIA), which regulates clinical laboratories at the federal level, see Brandon L. Garrett & Gregory Mitchell, The Proficiency of Experts, 166 U. Pa. L. Rev. 901, 951 (2018) (detailing the creation of the CLIA). See also 42 U.S.C. § 263 (2012) (describing the process for certifying laboratories).

[3] See Nat’l Research Council, DNA Technology in Forensic Science 8–9 (1992) (listing recommendations for sound scientific use of DNA analysis in forensic science).

[4] Nat’l Research Council, The Evaluation of Forensic DNA Evidence 23–25 (1996) (describing guidelines on quality control and quality assurance).

[5] Brandon L. Garrett & Peter J. Neufeld, Invalid Forensic Science Testimony and Wrongful Convictions, 95 Va. L. Rev. 1, 1 (2009) (exploring “the forensic science testimony by prosecution experts in the trials of innocent persons, all convicted of serious crimes, who were later exonerated by post-conviction DNA testing”).

[6] Overturning Wrongful Convictions Involving Misapplied Forensics, Innocence Project, https://www.innocenceproject.org/causes/misapplication-forensic-science [https://perma.cc/K22Z-H6BX].

[7] See, e.g., Karen Kafadar, Statistics and the Impact of the 2009 NAS Report, 69 Duke L.J. Online 6 (2019).

[8] Id.

[9] Id.

[10] See Kafadar, supra note 7, at 11 (“We still have to change the culture. It is not a sign of weakness to say you are not 100 percent accurate. It is far better to quantify the realistic errors in procedures than to pretend they are perfect.”).

The Trajectory of Forensics

 

Volume 69 September 2019

Reflection

The Trajectory of Forensics

Peter Neufeld Co-founder of and Special Counsel to the Innocence Project; Partner, Neufeld Scheck & Brustin, LLP. This Reflection piece is adapted from remarks made at a March 6, 2019 conference at Duke Law School, made possible by the Innocence Project, Duke Law School, and the Center for Statistics and Applications to Forensic Evidence (CSAFE), which is funded through Cooperative Agreement #70NANB15H176 between the National Institute of Standards and Technology and Iowa State University, and which includes activities carried out at Carnegie Mellon University, Duke University, University of California, Irvine, and University of Virginia.
PDF

To appreciate the significance of the 2009 report by the National Academy of Sciences addressing fundamental weaknesses in forensic science, it is useful to understand its historical context.[1][1]. Comm. on Identifying the Needs of the Forensic Sci. Cmty., Nat’l Research Council of the Nat’l Acads., Strengthening Forensic Science in the United States: A Path Forward (2009). That context necessitates an interdisciplinary inquiry since there are sociological, psychological, and political dynamics in play. That historical context better equips us to understand both the obstacles that lie ahead and how to respond to them. In this essay, I discuss the state of forensics before the DNA revolution of the late 1980s and 1990s. I then describe how DNA changed our thinking about forensics more broadly. I describe the immediate backdrop before the National Academy of Sciences Committee on Identifying the Needs of the Forensic Science Community was convened and its impact on policy and litigation in the United States.

I. Forensics in the United States Before 2009

Having litigated criminal cases in the late 1970s and the 1980s at the intersection of science and law, first, I hope to give you a sense of the lay of the land at that period.

Forensic Lab Reports.
The content of forensic lab reports consisted of a “thumb up” or a “thumb down.” Typical lab reports would state in sum and substance, “The crime scene latent lift matches the defendant’s print. He is the source”; “To a reasonable scientific certainty, the defendant is the source of the bite on the victim’s leg”; or “That is the defendant’s hair on the victim’s bed.” They were often a paragraph or two in length.

Litigation
. Few, if any, courts in America seriously considered the validity and reliability of the actual methods used in forensics. How often an expert had been accepted by courts was more important than the accuracy of the expert’s method. If bite-mark evidence was proffered, a judge might reason that the expert used a ruler to measure the distances and used a microscope to look at the marks more carefully. Those instruments were well-described in the literature, therefore bite-mark comparison was science despite a paucity of research estimating the accuracy of forensic dentists. That is what decisions were like, not just for bite mark evidence, but for other pattern and impression forensic disciplines as well.

Regulation and legislation
. With the exception of forensic DNA typing, there simply was not any regulation or legislation governing the forensic sciences in America, either at the state level or at the federal level.

Quality assurance and quality control
. Neither quality assurance nor quality control existed in any meaningful way in crime lab settings in the late 1970s or 80s. Although the National Institute of Justice supported some studies looking at proficiency tests, there was no regulation, and there was no legislation. The meager accreditation that existed was voluntary and did not attempt to limit labs to using scientifically valid methods. Why was that the reality? Comparing the state of forensics with that of clinical laboratories at the same time underscores the problem. By the 1970s and early 80s, we already had a well-regulated system of clinical laboratories in America with routine proficiency testing requirements for analysts. [2][2]. For an overview of the creation of the Clinical Laboratories Improvement Act (CLIA), which regulates clinical laboratories at the federal level, see Brandon L. Garrett & Gregory Mitchell, The Proficiency of Experts, 166 U. Pa. L. Rev. 901, 951 (2018) (detailing the creation of the CLIA). See also 42 U.S.C. § 263 (2012) (describing the process for certifying laboratories). Why did we have that federal system of regulation in clinical laboratories but not crime laboratories? I think the reason has to do with the constituency. For clinical medicine, the entire population is a constituent, including, critically, the white middle and upper class. We all care about our health, and, therefore, we have much greater input into legislation, regulation, quality control in clinical medicine, and tests involving middle class people’s health. But in forensic science, the primary constituents were “criminals.” Political actors considered them dispensable, so there was no need to have independent external bodies review the quality of forensics. I do think that the historical disparity is almost that simple.

II. The DNA Revolution

The DNA revolution was transformative in two ways. First, all of a sudden, DNA as a discipline was introduced in the courts, with a foundation in basic and applied research, extensive peer-reviewed publications, and with measurement of uncertainty that could be articulated. There were two National Academy of Science study groups, one in 1992 and the second in 1996. The first set standards for the bench science,[3][3]. See Nat’l Research Council, DNA Technology in Forensic Science 8–9 (1992) (listing recommendations for sound scientific use of DNA analysis in forensic science). and the second set standards for assessing the probative value of the evidence and dealing with statistical issues. [4][4] Nat’l Research Council, The Evaluation of Forensic DNA Evidence 23–25 (1996) (describing guidelines on quality control and quality assurance). For those of us in the legal profession who had no background in science, all of a sudden, we could see this disparity between the scientific validation and rigor of forensic DNA testing as opposed to all other pattern, impression, and trace evidence. It was a real eye-opener.

The second transformative consequence arose from what DNA taught us about the causes of wrongful convictions. Every time we exonerated a previously convicted person by submitting the preserved, old biological evidence to modern DNA testing, we deconstructed the cases to find out what went wrong in the initial criminal investigation and prosecution that contributed to the conviction of an innocent person. [5][5]. Brandon L. Garrett & Peter J. Neufeld, Invalid Forensic Science Testimony and Wrongful Convictions, 95 Va. L. Rev. 1, 1 (2009) (exploring “the forensic science testimony by prosecution experts in the trials of innocent persons, all convicted of serious crimes, who were later exonerated by post-conviction DNA testing”). We learned that in 45 percent of those wrongful conviction cases, other traditional forensic methods were used at the original criminal investigation and trial, which, along with misleading results, contributed in large part to securing those wrongful convictions. [6][6]. Overturning Wrongful Convictions Involving Misapplied Forensics, Innocence Project, https://www.innocenceproject.org/causes/misapplication-forensic-science [https://perma.cc/K22Z-H6BX].

These two consequences of the DNA revolution made many observers think, “We have to look critically at other forensic methods.” Whereas before there was no constituency, people began to care more about the quality of justice and the reliability of our fact-finding, including in plea and trial adjudications. People in the broader scientific community, people in the legal communities, and laypeople all of a sudden started caring about forensic science.

That historical setting then gave rise in 2007 to the National Academy of Sciences creating the Committee. The Committee then undertook the effort to produce a report examining broadly the legitimacy of many non-DNA forensic methods. It would not have happened but for the extraordinary impact that DNA had on criminal justice.

Other participants in this conference discussed how the 2009 Report contributed to systemic and systematic change. [7][7]. See, e.g., Karen Kafadar, Statistics and the Impact of the 2009 NAS Report, 69 Duke L.J. Online 6 (2019). It created strategic opportunities because it created an opening to talk about these issues to the press, in the courts, and in the legislatures. The Innocence Project has a dynamic policy department, working in dozens of states, pressing for reforms designed to reduce the frequency of wrongful convictions. We have created a model statute for “change in science,” which allows a person to get back into court by demonstrating that the scientific community’s understanding of a particular forensic method has changed since that method was used at the original trial or because the testifying expert has learned that the opinions and conclusions originally offered are no longer supported. Because the vulnerability and misapplication of so many forensic methods has been so well established, five states in the last few years have, either through statute or judicial order, created a “change in science” law, so that people can go back into court—even without new DNA testing—to challenge convictions that were secured with unreliable forensics. [8][8]. Id. Texas, California, Wyoming, and Connecticut each enacted statutes, and in Michigan, the vehicle was created through a Supreme Court judicial rule making. [9][9]. Id.

DNA and the 2009 Report have effectively eroded the legal doctrine of finality. That doctrine poses often insurmountable obstacles to convicted persons seeking to get back into court and secure a new trial many years after a conviction. The doctrine is premised on the notion that after many years, witnesses die, evidence is lost or destroyed, and memories fade, and thus, it is unlikely that a second trial would be more reliable than the first. However, recent DNA exonerations demonstrate that newly discovered biological evidence exculpating convicted persons is far more reliable than the eyewitness, confession, jailhouse informant, or lesser forensic evidence used to convict.

Many members of the forensic community have also changed their thinking about the rigor of the work they did in the past. Of course, there are still some who do not know about the Report or, actually worse, do not recognize its legitimacy. But that is not unexpected, because all great movements and all great changes happen gradually. You take two steps forward and a step backward. People have alluded to the need, ultimately, for changing the culture. [10][10]. See Kafadar, supra note 7, at 11 (“We still have to change the culture. It is not a sign of weakness to say you are not 100 percent accurate. It is far better to quantify the realistic errors in procedures than to pretend they are perfect.”). Legislation can accomplish only so much, litigation has limited utility, and the voluntary cooperation of police and prosecutors is not enforceable. Gradually, we have to win the hearts and minds of people working in forensics, in criminal justice, and just people in general. Once those attitudes change, structural reform and more just outcomes will follow. We are now at the beginning of that trajectory, and the National Academy of Sciences Report played a central role in moving us forward.


Copyright © 2019 Peter Neufeld.

Co-founder of and Special Counsel to the Innocence Project; Partner, Neufeld Scheck & Brustin, LLP. This Reflection piece is adapted from remarks made at a March 6, 2019 conference at Duke Law School, made possible by the Innocence Project, Duke Law School, and the Center for Statistics and Applications to Forensic Evidence (CSAFE), which is funded through Cooperative Agreement #70NANB15H176 between the National Institute of Standards and Technology and Iowa State University, and which includes activities carried out at Carnegie Mellon University, Duke University, University of California, Irvine, and University of Virginia.

[1] Comm. on Identifying the Needs of the Forensic Sci. Cmty., Nat’l Research Council of the Nat’l Acads., Strengthening Forensic Science in the United States: A Path Forward (2009).

[2] For an overview of the creation of the Clinical Laboratories Improvement Act (CLIA), which regulates clinical laboratories at the federal level, see Brandon L. Garrett & Gregory Mitchell, The Proficiency of Experts, 166 U. Pa. L. Rev. 901, 951 (2018) (detailing the creation of the CLIA). See also 42 U.S.C. § 263 (2012) (describing the process for certifying laboratories).

[3] See Nat’l Research Council, DNA Technology in Forensic Science 8–9 (1992) (listing recommendations for sound scientific use of DNA analysis in forensic science).

[4] Nat’l Research Council, The Evaluation of Forensic DNA Evidence 23–25 (1996) (describing guidelines on quality control and quality assurance).

[5] Brandon L. Garrett & Peter J. Neufeld, Invalid Forensic Science Testimony and Wrongful Convictions, 95 Va. L. Rev. 1, 1 (2009) (exploring “the forensic science testimony by prosecution experts in the trials of innocent persons, all convicted of serious crimes, who were later exonerated by post-conviction DNA testing”).

[6] Overturning Wrongful Convictions Involving Misapplied Forensics, Innocence Project, https://www.innocenceproject.org/causes/misapplication-forensic-science [https://perma.cc/K22Z-H6BX].

[7] See, e.g., Karen Kafadar, Statistics and the Impact of the 2009 NAS Report, 69 Duke L.J. Online 6 (2019).

[8] Id.

[9] Id.

[10] See Kafadar, supra note 7, at 11 (“We still have to change the culture. It is not a sign of weakness to say you are not 100 percent accurate. It is far better to quantify the realistic errors in procedures than to pretend they are perfect.”).