In a newly-published study, researchers find that forensic experts rendering opinions may unconsciously bias their results toward the parties that employ them. The results have potentially broad application across a wide variety of traditional forensic disciplines.
The journal Psychological Science has posted the study, called “Are Forensic Experts Biased by the Side That Retained Them?” (I thank The Crime Report, from John Jay College of Criminal Justice, for bringing it to my attention.) The authors, Daniel C. Murrie, Marcus T. Boccaccini, Lucy A. Guarnera, and Katrina A. Rufino, tested the idea that forensic experts called upon to evaluate evidence in an adversarial legal proceeding might respond differently, depending solely upon which party asked them to perform the evaluation. Here’s the abstract to the study:
How objective are forensic experts when they are retained by one of the opposing sides in an adversarial legal proceeding? Despite long-standing concerns from within the legal system, little is known about whether experts can provide opinions unbiased by the side that retained them. In this experiment, we paid 108 forensic psychologists and psychiatrists to review the same offender case files, but deceived some to believe that they were consulting for the defense and some to believe that they were consulting for the prosecution. Participants scored each offender on two commonly used, well-researched risk-assessment instruments. Those who believed they were working for the prosecution tended to assign higher risk scores to offenders, whereas those who believed they were working for the defense tended to assign lower risk scores to the same offenders; the effect sizes (d) ranged up to 0.85. The results provide strong evidence of an allegiance effect among some forensic experts in adversarial legal proceedings.
If these results stand the test of time, the implications are potentially great, As discussed in the National Academy of Sciences’ landmark 2009 report “Strengthening Forensic Sciences in the United States: A Path Forward,” most forensic disciplines (outside of DNA identification and those that use traditional chemical analysis) rely to a significant degree on human interpretation to generate conclusions. The results of the Murrie et al. work suggest that all of these disciplines may be influenced by a very important piece of context — who pays for their work — regardless of which side of the case that is. The study would seem to support one of the central recommendations of the National Academy of Sciences report: “removing all public forensic laboratories and facilities from the administrative control of law enforcement agencies or prosecutors’ offices” (p. 24).
Failed Evidence 
I think the study shows that risk assessments in the mental health community may involve cognitive bias in subjective interpretations, and that it would be inappropriate to generalize the results of this study to all forensic experts. Did anyone compare the ratings of the psychologists to the psychiatrists? Were the evaluator groups equally balanced with psychologists and psychiatrists? How about having a control group that did not know which side retained them? Did the authors assess the amount of experience of each evaluator? From what population(s) were the 108 reviewers drawn? Not exactly the “tightest” study design seen in the “hard sciences.”
I think most experts determine their “feelings” about a case before they accept it, not after.
When I call to report an oral opinion, I frequently ask the attorney, which side are you representing? My words will be the same.
Everything I have read or experienced supports the notion that people have intrinsic bias. Large scale adaptations to overcome biases may or may not be practical, or effective. There are typically second and third order effects that are almost impossible to predict. On the other hand, being aware that I have personal bias (even if I don’t ‘feel’ bias) and humble around the issue, generally allows mine to be balanced by the biases of others.
Fortunately, our society has several levels of checks and balances built in. All one has to do is listen to the rhetoric from one side of a hotly contested issue to be extremely thankful for the tension and balance brought by the other side of the issue.
I am not saying systemic changes are not worth considering and may be needed, I am just encouraging us not to be suckered into a ‘fool’s choice’ – status quo OR draconian changes.