Some notes on How Cross-Examination on Subjectivity and Bias Affects Jurors' Evaluations of Forensic Science Evidence
I recently read Bill Thompson and Nicholas Scurich’s forthcoming article How Cross-Examination on Subjectivity and Bias Affects Jurors' Evaluations of Forensic Science Evidence. I think it’s an important addition to our understanding about the effectiveness of cross-examination in conveying knowledge about the limits (and risks) of forensic science evidence. This is especially important because courts often advert to the possibility of thorough and effective cross-examination as a reason to admit dubious evidence.
In the authors’ study, they provided jury-eligible participants with a forensic odonotologist’s testimony, in which he declared a match “to a scientific certainty”. The cross-examination attacks this in two ways (depending on the condition the participant was in): (1) the subjectivity of the analysis and (2) exposure to task-irrelevant information.
In the condition in which the witness is crossed on subjectivity, the following exchange occurs:
Q: You decided that Stanley Wilson’s teeth are consistent with the bite mark?
A: Yes.
Q: But that required some interpretation, didn’t it?
A: Well, yes, there is always an element of interpretation.
Q: It wasn’t a perfect match, was it?
A: There were eight clear points of comparison on which there was consistency.
Q: But there were differences between the measurements you made on the bite mark and the measurements you made on the model of Mr. Wilson’s teeth?
A: There are always some minor differences when you compare a model to a bite mark, but in my judgment the differences were not meaningful differences.
Q: You measured something you call the inter-canine distance on the bite mark?
A: Yes, that is the distance in millimeters between the marks made by the upper canine teeth.
Q: But when you measured the inter-canine distance on Mr. Wilson’s teeth it was four millimeters shorter?
A: Yes, but in my judgment that could have been caused by stretching of Mr. Johnson’s skin during the attack, so it is still consistent.
Q: And you measured the angle of the tooth impressions made by the incisors?
A: Yes.
Q: And again, there were differences?
A: As I said, one must make allowances for the elasticity of the skin. And some of the differences in measurement were due, I believe, to distortions caused by biting through the fabric of the bandana. But a number of the measurements were identical.
Q: There aren’t any formal rules in your field about what constitutes a match? You just rely on your own personal judgment?
A: There is no formula for what constitutes a match because each case is unique. Like all forensic odontologists, I rely on my knowledge, training and experience in the field to make the right judgment.
Q: So it’s a match because you say it’s a match?
A: Yes. I’m a qualified expert in this field.
In short, the defence brings out the fact that there aren’t hard and fast rules within bitemark analysis and the examiner tends to agree, although downplaying that limit somewhat.
The conditions having to do with contextual bias are a bit more complicated. Those conditions revolved around the accused being associated with a gang and being suspected of other crimes but always avoiding charges. This information does not seem relevant to the examiner’s job and could bias him. Several experimental conditions were related to this information: the expert was blind to this information, the expert saw it but said he ignored it, the expert expressly relied on this information, and the control, the expert saw this information but it was never acknowledged in the testimony.
As to the results, it’s a bit hard to say because the SSRN copy doesn’t have the full tables included. It does appear though that both flaws with the testimony, when brought to light, reduced both the credibility ascribed to the expert and guilty verdicts.
These are interesting findings, especially because (at least in Australia and Canada), many experts do not disclose the significance of subjectivity and contextual bias, and their evidence is often unchallenged. If Thompson and Scurich are right, and this study generalizes beyond this particular context, then we have to find ways to get this knowledge to the factfinder.
As a quibble, I do wish they had preregistered this study and put their data online (e.g., on the OSF). It’d be a lot easier to assess, especially at this prepublication stage.