Posts Tagged ‘National Academy of Sciences’

Following up on my last post, in which I asked why there were still no national standards for forensic science five years after the National Academy of Sciences’2009 report Strengthening Forensic Science in the United States, and with scandal after scandal in U.S. crime labs all over the country, there may be light on the horizon.  On January 10, the U.S. Department of Justice (DOJ)  and the National Institute for Standards and Technology (NIST) announced the formation of the National Commission on Forensic Science.

According to the announcement issued by DOJ and NIST:

Members of the commission will work to improve the practice of forensic science by developing guidance concerning the intersections between forensic science and the criminal justice system. The commission also will work to develop policy recommendations for the U.S. Attorney General, including uniform codes for professional responsibility and requirements for formal training and certification.

John P. Holdren, Assistant to the President for Science and Technology and Director of the White House Office of Science and Technology Policy, said that the Commission “will help ensure that the forensic sciences are supported by the most rigorous standards available—a foundational requirement in a nation built on the credo of ‘justice for all.’ ”

The formation of the Commission could be the a significant milestone in the march toward the use of real science and defensible national standards in forensic labs.  But it may be limited in what it can achieve just by its creation and structure: it is not a body created by Congress with power to come up with and implement standards or to regulate anything.  Rather it is a federal advisory committee, formed under the Federal Advisory Committee Act of 1972.  (A quick primer on the Act is here.)   It investigates and debate designated topics, and then reports its recommendations to the relevant federal department(s) that formed it (in this case, the DOJ and NIST).  Those agencies could choose to embrace and follow, or could choose to reject, some, all, or none of the Commission’s suggestions.

Still, this is a hopeful sign that we might be heading in the right direction.  At the very least, we will see a national conversation between the very large number of Committee members; they come from a variety of backgrounds in government, science, the legal system, and elsewhere.  See the list of more than thirty Commission members at the bottom of this announcement.

I hope readers will weigh in on the following question: realistically, what will come from the Committee?  Will the government adopt these recommendations?  Will the recommendations include national standards to regulate forensic testing, assure quality control, and the like? In the end, will the work that you foresee coming from the Commission improve the U.S.’s largely unregulated system?

Here we are, more than a month after chemist Annie Dookhan, formerly of the Massachusetts State Drug Laboratory, entered a guilty plea to producing fraudulent forensic testing results, and went to prison.  The scandal, potentially involving tens of thousands of cases, has resulted in the release of hundreds of convicted persons. All of this has reportedly cost the state of Massachusetts more than 8 million dollars, and the state has budgeted almost 9 million more for the continuing damage.    Readers have seen coverage of the Massachusetts scandal, and several others, here and here and here.

But Dhookan and the Massachusetts Drug Laboratory are far from alone.  According to a report by National Public Radio, there have been  twelve major crime lab scandals in the U.S. in just the last two years.  With all of this damage  – to individual cases and defendants, to state and local budgets, and to the public trust — occurring all over the country, how have policy makers at the national level responded?

Well, they haven’t.  At least not yet.

Almost five years after the release of the National Academy of Sciences’ report “Strengthening Forensic Science in the U.S.: A Path Forward,” which recommended (among other things) the establishment of national industry standards for forensic labs and a National Institute of Forensic Science,  as well as the independence of every crime lab from police agencies and prosecution offices, none of this has happened.  Nor is accreditation of laboratories required.

According to National Public Radio, Senators John Cornyn of Texas and Patrick Leahy of Vermont “are working to introduce legislation this year” which could address some of these problems.  But nearly five years after the NAS report, and with the parade of crime lab scandals  continuing without let up, why has it taken five years to get to this very preliminary point?

Readers, would mandatory national standards help?  Are they appropriate?  What about requiring accreditation?

If you are from outside the U.S., does your country set mandatory national standards for crime labs?  Is accreditation required?

The time has long since passed for us to do something about this set of problems in the U.S.  We just can’t afford the damage to the credibility of our criminal justice system and the costs of  reviewing cases and releasing convicted prisoners — some of whom may very well be guilty, but whose cases are tainted.

In a newly-published study, researchers find that forensic experts rendering opinions may unconsciously bias their results toward the parties that employ them.  The results have potentially broad application across a wide variety of traditional forensic disciplines.

The journal Psychological Science has posted the study, called “Are Forensic Experts Biased by the Side That Retained Them?”  (I thank The Crime Report, from John Jay College of Criminal Justice, for bringing it to my attention.)  The authors, Daniel C. Murrie, Marcus T. Boccaccini, Lucy A. Guarnera, and Katrina A. Rufino, tested the idea that forensic experts called upon to evaluate evidence in an adversarial legal proceeding might respond differently, depending solely upon which party asked them to perform the evaluation.  Here’s the abstract to the study:

How objective are forensic experts when they are retained by one of the opposing sides in an adversarial legal proceeding? Despite long-standing concerns from within the legal system, little is known about whether experts can provide opinions unbiased by the side that retained them. In this experiment, we paid 108 forensic psychologists and psychiatrists to review the same offender case files, but deceived some to believe that they were consulting for the defense and some to believe that they were consulting for the prosecution. Participants scored each offender on two commonly used, well-researched risk-assessment instruments. Those who believed they were working for the prosecution tended to assign higher risk scores to offenders, whereas those who believed they were working for the defense tended to assign lower risk scores to the same offenders; the effect sizes (d) ranged up to 0.85. The results provide strong evidence of an allegiance effect among some forensic experts in adversarial legal proceedings.

If these results stand the test of time, the implications are potentially great,  As discussed in the National Academy of Sciences’ landmark 2009 report “Strengthening Forensic Sciences in the United States: A Path Forward,” most forensic disciplines (outside of DNA identification and those that use traditional chemical analysis) rely to a significant degree on human interpretation to generate conclusions.  The results of the Murrie et al. work suggest that all of these disciplines may be influenced by a very important piece of context — who pays for their work — regardless of which side of the case that is.  The study would seem to support one of the central recommendations of the National Academy of Sciences report: “removing all public forensic laboratories and facilities from the administrative control of law enforcement agencies or prosecutors’ offices” (p. 24).

When a former high-ranking Justice Department official speaks of a “revolution” in criminal justice, with the whole field turning toward science, could it mean less failed evidence in the future?  What does it mean for those concerned with faulty forensic science?

Laurie Robinson served as Assistant Attorney General in both the Clinton and Obama Justice Departments, where she oversaw the Office of Justice Programs (OJP), the research, statistics and criminal justice assistance arm Justice.  That made her remarks to the Delaware Center for Justice the other day worth noticing.  According to the Wilmington News Journal, “[w]e’re seeing something akin to a revolution in criminal justice in this country,” said Robinson, now on the faculty of George Mason University. “We’re at an important crossroads, one where ideology has taken a back seat, and science and pragmatism have come to the fore.”

Robinson’s web page at George Mason says her tenure at OJP  “was marked by a focus on science and evidence-based programming.”  She was in Delaware to discuss the state’s re-entry programs  and other initiatives to reduce recidivism, and few would disagree that those important programs need scientific and statistical support.   But I wonder whether Robinson would be as optimistic about science’s role in forensic methods, which have played a role in about half of all wrongful convictions across the U.S.  Surely, forensic science needs “science and evidence-based” support and examination — badly.

 It has now been more than four years since the release of the landmark 2009 National Academy of Sciences report Strengthening Forensic Sciences in the United States: A Path Forward, which found that except for DNA and chemical analysis, most of what we think of as forensic science isn’t science at all.  In that time, little seems to have changed; scandals in crime labs continue to pile up in jurisdictions across the country (see my posts about lab scandals just this past year in Massachusetts  and Minnesota,  for example).  In a particularly compelling piece of writing in the Huffington Post, Radley Balko discusses the long-running crime lab scandal in Mississippi, and puts it in context: Mississippi’s scandal “is just the latest in a long, sad line of such stories” that Balko has already chronicled.

What’s to be done?  As a start, I argued in chapter 7 of  Failed Evidence, all federal money that goes to law enforcement should carry with it a requirement for compliance with best practices in police investigation and forensic science.  Not all law enforcement agencies run their own forensic labs (and as the NAS report said in Chapter 6, “Improving Methods, Practice, and Performance in Forensic Science,” labs should be independent of law enforcement.).  But for those that do, compliance with standards that would avoid systematic error, human biases, fraud, and improper scientific testing and testimony should be mandatory.

That would start a revolution right there.  Because what law enforcement agency could afford to just turn down federal funding, in budgetary times like these?

 

Of all of the methods studied in the 2009 National Academy of Sciences’ report on forensic science, forensic odontology — the analysis of bite marks to identify the perpetrators of crimes — has come under some of the harshest criticism.  And now an effort is underway to keep it out of courtrooms for good.

This article by the Associated Press examined “decades of court records, archives, news reports and filings by the Innocence Project in order to compile the most comprehensive count to date of those exonerated after being convicted or charged based on bite mark evidence.”   Typically, the evidence consisted of testimony by forensic dentists identifying marks on the bodies of victims (usually in cases of murder or sexual assault) as coming from the defendant’s teeth.  According to the AP, since 2000, DNA identification has overturned one after another of these cases, throwing the entire “discipline” into question, and rendering it nearly obsolete.

Critics make two main claims against forensic odontology.  First, there’s no scientific proof that a bite mark can be matched to any particular set of teeth.  There is no data, no experimental evidence — nothing — on which to base the idea that forensic odontology can make reliable identifications.  Second, human skin — almost always the site of the bite mark in question — does not reliably “record” bite marks, since the skin itself changes over time, even after death.  Because the skin changes shape, consistency, color, and even size  after the mark is made, this makes bite marks, and the method itself, inherently unstable.

Yet the dentists, who belong to the American Board of Forensic Odontology, insist that their methods of making identifications of the source of bite marks are reliable; it’s just that some forensic dentists are, well, not too good or biased.  “The problem lies in the analyst or the bias,” said Dr. Frank Wright, a forensic dentist in Cincinnati. “So if the analyst is … not properly trained or introduces bias into their exam, sure, it’s going to be polluted, just like any other scientific investigation. It doesn’t mean bite mark evidence is bad.”  It’s the familiar refrain: it’s just a few bad apples, not the whole barrel.  But according to the AP:

Only about 100 forensic dentists are certified by the odontology board, and just a fraction are actively analyzing and comparing bite marks. Certification requires no proficiency tests. The board requires a dentist to have been the lead investigator and to have testified in one current bite mark case and to analyze six past cases on file — a system criticized by defense attorneys because it requires testimony before certification…The consequences for being wrong are almost nonexistent. Many lawsuits against forensic dentists employed by counties and medical examiner’s offices have been thrown out because as government officials, they’re largely immune from liability.  Only one member of the American Board of Forensic Odontology has ever been suspended, none has ever been decertified, and some dentists still on the board have been involved in some of the most high-profile and egregious exonerations on record.

It’s worth noting that there’s nothing wrong with other uses of forensic dentistry, such as identifying human remains from teeth by matching them to existing dental records; that type of forensic work is generally rock solid and remains unchallenged by critics.  But on identification of perpetrators through bite mark analysis, the question is different: when a forensic “science” has as dismal a record as forensic odontology does, and no scientific proof of its validity exists, can anything justify allowing the use of bite mark evidence to convict a person of a crime?

When I’m in Cincinnati for talks on Failed Evidence tonight, April 4, at 7:00 pm at the Clifton Cultural Arts Center and tomorrow, April 5, at noon at the University of Cincinnati School of Law, one topic sure to come up is an article from the April 3 New York Times, “Advances in Science of Fire Free a Convict After 42 Years. ”   Louis Taylor was serving 28 life sentences for the deaths caused by a fire in a Tucson, Arizona hotel in December of 1970.  Taylor, then just 16 years old, was convicted of arson based on faulty forensic science.

Mr. Taylor has been release from prison.  He is now 58 years old.

The story highlights the state of arson investigation, past and present.

A few years ago, the National Academy of Sciences turned its attention to the misuse of science in courtrooms, saying that pseudoscientific theories had been used to convict people of crimes they may not have committed. By then, a small group of fire engineers had already begun to discredit many of the assumptions employed in fire investigations, like the practice of using the amount of heat radiated by a fire to assess if an accelerant had been used.

Unlike DNA evidence, which can exonerate one person and sometimes incriminate another, the evidence collected in some arson investigations does not yield precise results. Often much of the evidence has been lost or destroyed. In the case of the hotel fire here, all that is left are photographs, reports and chemical analysis, all of them assembled to prove arson.

As a result, “we can’t definitely say what really caused the fire,” said John J. Lentini, a veteran fire investigator who wrote a report on Mr. Taylor’s case. “But what we can do is discredit the evidence” used to support the charge.

The case recalls the story of the trial and execution of Cameron Todd Willingham in Texas, executed in 2004 for the deaths of his children in a fire.  Experts call the arson in that case terribly flawed — just as in Mr. Taylor’s case.

The science surrounding investigation is light years ahead of where it used to be, even a decade ago.  It’s time that all of the old cases in which verdicts depended on outmoded and discredited methods of arson investigation be re-examined, and if necessary overturned.

Four years after the National Academy of Science’s 2009 report Strengthening Forensic Science in the United States: A Path Forward called for basic changes in the forensic sciences, U.S. Department of Justice and U.S. Department of Commerce’s National Institute of Standards and Technology have announced they will create a national commission on forensic science.  The commission will have 30 members — forensic science practitioners, researchers, prosecutors, defense attorneys, and judges — who will develop policy recommendations for the Attorney General.  According to the Department of Justice announcement:

The commission will have responsibility for developing guidance concerning the intersections between forensic science and the courtroom and developing policy recommendations, including uniform codes for professional responsibility and requirements for training and certification.  The new initiative provides a framework for coordination across forensic disciplines under federal leadership, with state and local participation. The Department of Justice, through its involvement in the commission, will take an active role in developing policy recommendations and coordinating implementation.

For many who looked for action in the wake of the National Academy of Science’s 2009 report and saw very little, the creation of the commission will comes as a welcome step forward.   The National District Attorneys Association (NDAA), which took a fairly negative view of the 2009 report and its recommendations, is now reacting with a wait and see attitude.  The NDAA has not yet put out a formal statement in reaction to the announcement of the commission; according to Scott Burns, the Executive Director of the NDAA, the organization will do that once it gets the details on the commission, especially its composition.  So far, Burns told me, the NDAA is  “encouraged” by the fact that prosecutors will be part of the commission, though he stresses that he hopes to see more state and local prosecutors than federal ones.  Burns said that if the commission begins with the attitude that “the system is broken” and that its basics — fingerprints, tool marks, and the like — must be fixed, the NDAA will not look on it favorably.  On the other hand, if the commission starts with the attitude that “we can improve”  the system but that it basically functions well, that would be welcome.

I will write further on this as the story develops.

 

Good news: Failed Evidence: Why Law Enforcement Resists Science is the Feb. 4 selection by delanceyplace.com, a service that highlights and quotes new works for a large community of readers.  Delancyplace.com provides daily subscribers with “an excerpt or quote we view as interesting or noteworthy, offered with commentary to provide context. There is no theme, except that most excerpts will come from a non-fiction work, primarily historical in focus, and will occasionally be controversial. Finally, we hope that the selections will resonate beyond the subject of the book from which they were excerpted.”  Other recent selections have included Jared Diamond, The World Until Yesterday: What Can We Learn from Traditional Societies; Gordon Wood, Empire of Liberty: A History of the Early Republic, 1789-1815; and Ray Kurzweil and Terry Grossman, Transcend: Nine Steps to Living Well Forever.

 

Today I’ll be giving a talk on Failed Evidence at the University of Houston Law Center, 4800 Calhoun Road, Houston, 77004, at noon in room BLB 240.  The talk is free and open to the public.  I’ll be discussing the book and my thoughts about how we can move toward a future in which the existing scientific work on eyewitness identification, interrogation of suspects, and basic (i.e., non-DNA) forensics will make for better, more accurate investigation and prosecution of crime.  I’ll be speaking to law students, faculty, members of the university community, attorneys, and interested members of the public.

Details on the event are here.

Houston is a particularly interesting place to have this discussion.  Over the past ten years, the crime lab in Houston has had repeated problems.  After all of this, the authorities decided to try something they had not done before: they are removing the crime lab from the jurisdiction of law enforcement and putting it under the control of an independent body, the Houston Forensic Science Local Government Corporation.  I wrote about this in an op-ed for the Houston Chronicle on Saturday, which you can see here.  This move puts Houston’s efforts to deal with forensic reform ahead of the  curve, and implements one of the main recommendations of the National Academy of Sciences 2009 report, Strengthening Forensic Science in the U.S.: A Path Forward.

 

 

 

I’ve written a number of times (here and here an here, for example) about the problems with forensic science laboratories in this country.  Just in the last few months, we’ve seen scandals hit labs in Massachusetts, St. Paul, Minnesota, and in Mississippi.  It seems that the parade might never end.

But today, news emerged that indicates that, just maybe, forensic reform might be on the national agenda.

The new Congress will, of course, be preoccupied with budget and fiscal matters, and also with the President’s efforts on gun control and an expected push for immigration reform.  But Senator Patrick Leahy of Vermont, chair of the Senate Judiciary Committee, has announced that he intends to put forensic reform onto the long list of issues he will examine.  According to The BLT (the Blog of the Legal Times, which covers law and government in Washington), Leahy’s committee will be working on an ambitious agenda: immigration, national security and civil liberties issues (including the use of drones in both foreign and domestic contexts), and gun control policy, but that isn’t all.  “The committee will also focus on promoting national standards and oversight for forensic labs and practitioners,” BLT says.

This is a welcome development.  People can disagree about whether we should have national standards (I think yes) or a “national institute of forensic science,” (again, I say yes) as proposed in the National Academy of Sciences’ 2009 report Strengthening Forensic Science in the United States: A Path Forward.  But it’s hard to argue that we should not hold the current situation up to the light for some long-overdue scrutiny and discussion of  higher standards and better oversight.  With the never-ending parade of state and local scandals in crime labs, a little federal look-see could actually help.