Rethinking scientific communication in courts
On November 17, 2023, the ANU College of Law convened a day-long workshop (see the program here) about science that affects law and policy, and how it is communicated.
Mehera San Roque, ‘Communicating psychological evidence in criminal trials: biased judgements and expert readings‘
Anna Heavey, ‘Understanding “error” in forensic science: a new perspective on quality issues‘
This workshop started with the big picture, the values and norms inherent to science. It then spanned outwards to current issues in scientific communication, revolutions in forensic science – a field designed to address applied legal problems – and finally the implications of the pardon of Kathleen Folbigg after 20 years spent in Australia’s penal system.
Short descriptions of each talk and the associated slides (when available) are below.
Please watch this space for updates about a special issue of Forensic Science International: Synergy from many of the speakers. I want to once again thank those wonderful speakers, the attendees, the ANU College of Law, and its wonderful events team.
Session 1. Values, trust, and the legal system
Rachael L. Brown, ANU School of Philosophy, Unmasking the Values in Science: Beyond the Myth of Value-Free Inquiry
From outright fabrication, plagiarism and manipulation of data, we all know of high-profile cases where bias and values have negatively influenced science. Discussion around these cases and public discourse around scientific practice in general typically assumes that good scientific practice is value-free. In this paper I show why the ideal of value-free science is an illusion and how embracing the influence of values reshapes our understanding of evidence and expert testimony.
Rachael’s podcast on values in science
An interview with Rachael on values in science
Simine Vazire, Melbourne School of Psychological Sciences, Where are the self-correcting mechanisms in science?
We often hear the self-correcting mechanisms in science invoked as a reason to trust science, but it is not always clear what these mechanisms are. Some quality control mechanisms, such as peer review for journals, or vetting for textbooks or for public dissemination, have recently been found not to provide much of a safeguard against invalid claims. Instead, I argue that we should look for visible signs of a scientific community's commitment to self-correction. These signs include transparency in the research and peer review process, investment in error detection and quality control, and an emphasis on calibration rather than popularization. We should trust scientific claims more to the extent that they were produced by communities that have these hallmarks of credibility. Fields that are more transparent, rigorous, and calibrated should earn more trust. Metascience can provide scientists and the public with valuable information in assessing the credibility of scientific fields.
Gary Edmond, UNSW Law and Justice, Courts and the sciences: Problems with ‘autopoetic’ systems
This paper will explore some of the complexities around the relations and interactions between the relatively autonomous systems of law and the sciences. Though critical of Law’s failings with respect to systematic engagement with scientific knowledge and understanding its practices and their effects empirically, it seeks to provide explanation for the partial and tentative embrace of ‘science’ and ‘knowledge’.
Session 2. Synthesising and communicating science
Matthew J. Page, Monash University School of Public Health and Preventive Medicine, Cochrane Collaboration, The role of systematic reviews and meta-analyses in evidence-based decision-making: opportunities and challenges
Individuals seeking to make health decisions informed by evidence face an unprecedented deluge of literature, with an estimated 4,000 health-related articles published each day. This ‘information overload’ is countered by systematic reviews, an increasingly popular research design in which researchers use systematic and explicit methods to identify, select, critically appraise and synthesise findings of studies that address a particular question. In this presentation I will describe the role systematic reviews have played in evidence-based medicine, highlighting both the opportunities and challenges that have arisen, and implications for legal scholars and practitioners seeking to use synthesised evidence in their deliberations.
Kristy A. Martire, UNSW School of Psychology, Certain concerns: the importance of transparent evidentiary statements in the wake of the Queensland DNA inquiry
The accurate and transparent communication of forensic science opinions is vital for the fair and effective administration of justice. However, concerns have been raised about both the content of forensic science reports and the methods used to communicate the strength of expert opinions. This presentation will explore these key concerns using the 2022 Sofronoff Inquiry Report into “certain evidentiary statements” as an illustrative example. The Sofronoff report expresses concerns about the use of categorical conclusions, the use of institutional jargon as expert testimony, and the failure to disclose the appropriate caveats and qualifications. In doing so, the Sofronoff Inquiry Report provides a sobering demonstration of the reputational, emotional and financial costs that can flow on to professionals, institutions, victims and the public as a result of ignoring best practice recommendations for improving the communication of forensic science opinions.
Kristy A. Martire
Kaye Ballantyne, Chief Forensic Scientist, Victoria Police Forensic Services Department, Transparent forensic science reporting: what impact does it have?
Victoria Police Forensic Services Department has, in line with scientific best practice, impartiality expectations and the Victorian Supreme Court Practice Note on Expert Evidence in Criminal Trials, implemented fully transparent forensic reporting for all expert opinions provided to Victorian Courts. All statements in every discipline contain information regarding the fundamental principles of the scientific methodology utilised, the empirical validation testing status, indicative error rates where available, key limitations and assumptions, cognitive factors, quality assurance and reference to scientific controversies and authoritative reports. The impact of this transparency will be discussed, with consideration of the change our enhanced transparency has brought to forensic science practice and criminal justice proceedings.
Session 3. New challenges in science made for law
Anna Heavey, PathWest Laboratory Medicine WA + Curtin University, Understanding “error” in forensic science: a new perspective on quality issues
Forensic science is a high risk, high consequence field where system failures can result in catastrophic outcomes for individuals, facilities and entire disciplines. For many years the call for transparency with regards to error rates and critical incidents has been prevalent from both end users of forensic information and from within the field itself.
Clear communication of critical issues in forensic science has benefits not only to investigators and courts in understanding the weight of forensic information, but also as an opportunity to identify trends and opportunities for research and innovation to support continuous improvement of the field. The key to understanding the nature of critical issues in forensic science may lie in the quality management systems embedded in accredited forensic facilities world-wide where records of non-conforming work and critical issues are recorded as standard practice. This presentation outlines the current state of forensic quality management systems, the nature of critical issues detected within those systems and how consistent and transparent communication of those issues can improve understandings of “error” as it relates to forensic science.
Max Houck, Florida International University, Global Forensic and Justice Center, Strange Bedfellows: Why is Forensic Science Inside Law Enforcement Organizations?
Government uses science to make better decisions to serve and aid its citizens. Most governmental scientific endeavors have their own agency, like the U.S. Geological Survey or the National Forest Service, with their own remit and some level of control over their missions. Why then are forensic science organizations housed within law enforcement agencies? It is well-known that Edmund Locard created the first forensic laboratory in 1910 under the administration of the Lyon Police Department. But why there? Why not at a university, like at the University of Lyon where he trained and worked cases under Andre Lacassagne? Locard’s decision, and his later successes, fixed the foundation for forensic organizations for decades. But what are the modern and organizational implications when non-scientists oversee and control scientific pursuits? Why has independence of forensic science been so difficult to pursue, let alone achieve? This presentation will discuss the history of forensic science provision and offer an opinion on the reasons for forensic science being subsumed under law enforcement agencies.
Max Houck
Session 4. Improving expert evidence: Implications from the pardon of Kathleen Folbigg
Mehera San Roque, UNSW Law and Justice, Communicating psychological evidence in criminal trials: biased judgements and expert readings
In May 2023, Kathleen Folbigg received a pardon in respect of her five convictions arising from the deaths of her four children. It is generally assumed that the discovery and communication of new genetic evidence to a second Inquiry into her convictions grounded the decision to pardon Ms Folbigg. But perhaps equally significant was a shift in the way that the second Inquiry received and accepted expert evidence about the content and meaning of her diaries. Using this recent (re)reading and (re)evaluation of the Folbigg diaries as a starting point, I will discuss the evolution in the justifications offered and frameworks provided for including (expert) psychological evidence about human behaviour within the criminal trial, with a focus on its use in relation to women as defendants. The inclusion of (expert) evidence about what has sometimes been termed ‘counterintuitive’ behaviour can be characterised as a progressive move; but as a strategy for addressing gender bias it also carries risks.
Jason M. Chin, College of Law, Australian National University, Transparent research summaries to increase the accuracy and procedural fairness of the justice system
The conviction and pardon of Kathleen Folbigg underscore the urgent need for more efficient, transparent, and reliable mechanisms to bring scientific knowledge into courts. High quality systematic reviews of commonly used forensic practices and areas of study offer a promising way forward. Indeed, reviews by the National Academies (‘Strengthening Forensic Science in the United States: A Path Forward’) and Royal Society (the ‘primers for courts’ series) have provided accused people with a better opportunity to understand and test the scientific case levied against them. But even these reviews have their limits. In this talk, I will discuss possibilities and pragmatics of bringing science’s transparent research synthesis movement to law.
Anna-Maria Arabia, Chief Executive, Australian Academy of Science, Creating a more science sensitive justice system: learnings from the wrongful conviction of Kathleen Folbigg
Anna-Maria’s leadership saw the Academy appointed as an independent scientific adviser to the Second Inquiry into the Convictions of Kathleen Folbigg. It is believed to be the first time worldwide that a Learned Academy has played such a role in a judicial inquiry and offers a unique example to examine approaches that could be used by justice systems in Australia in their consideration, evaluation, and management of scientific evidence and expert witnesses. The Academy’s role also provides a springboard to examine law-reform opportunities to create a more science sensitive and evidence-informed justice system.
In 2018, Anna-Maria helped establish an annual join symposium with the Australian Academy of Law which explores topics that intersect science and the law. This year will see a lecture series delivered in Australia by the Royal Society of London’s Executive Director, Dame Julie Maxton DBE.
The DOI for this workshop and its underlying materials is https://doi.org/10.17605/OSF.IO/2FKJQ