Objective The Executive Committee of the American Psychology-Law Society (Division 41 of the American Psychological Association) appointed a subcommittee to update the influential 1998 scientific review paper on guidelines for eyewitness identification procedures. Method This was a collaborative effort by six senior eyewitness researchers, who all participated in the writing process. Feedback from members of AP-LS and the legal communities was solicited over an 18-month period. Results The results yielded nine recommendations for planning, designing, and conducting eyewitness identification procedures. Four of the recommendations were from the 1998 article and concerned the selection of lineup fillers, prelineup instructions to witnesses, the use of double-blind procedures, and collection of a confidence statement. The additional five recommendations concern the need for law enforcement to conduct a prelineup interview of the witness, the need for evidence-based suspicion before conducting an identification procedure, video-recording of the entire procedure, avoiding repeated identification attempts with the same witness and same suspect, and avoiding the use of showups when possible and improving how showups are conducted when they are necessary. Conclusions The reliability and integrity of eyewitness identification evidence is highly dependent on the procedures used by law enforcement for collecting and preserving the eyewitness evidence. These nine recommendations can advance the reliability and integrity of the evidence. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Policy and procedure recommendations for the collection and preservation of eyewitness identification evidence.

Cognitive bias research in forensic science: A systematic review.

Exposure to irrelevant contextual information prompts confirmation-biased judgments of forensic science evidence ( Kassin, Dror, & Kukucka, 2013 ). Nevertheless, some forensic examiners appear to believe that blind testing is unnecessary. To assess forensic examiners’ beliefs about the scope and nature of cognitive bias, we surveyed 403 experienced examiners from 21 countries. Overall, examiners regarded their judgments as nearly infallible and showed only a limited understanding and appreciation of cognitive bias. Most examiners believed they are immune to bias or can reduce bias through mere willpower, and fewer than half supported blind testing. Furthermore, many examiners showed a bias blind spot ( Pronin, Lin, & Ross, 2002 ), acknowledging bias in other domains but not their own, and in other examiners but not themselves. These findings underscore the necessity of procedural reforms that blind forensic examiners to potentially biasing information, as is commonplace in other branches of science.

Cognitive Bias and Blindness: A Global Survey of Forensic Science Examiners

Expert performance can be quantified by examining reliability and biasability between and within experts, and teasing apart their observations from their conclusions. I utilize these parameters to develop a Hierarchy of Expert Performance (HEP) that includes eight distinct levels. Using this hierarchy I evaluate and quantify the performance of forensic experts, a highly specialized domain that plays a critical role in the criminal justice system. Evaluating expert performance within HEP enables the identification of weaknesses in expert performance, and enables the comparison of experts across domains. HEP also provides theoretical and applied insights into expertise.

A Hierarchy of Expert Performance

The evaluation of results of forensic genetic analyses given activity level propositions is an emerging discipline in forensic genetics. Although it is a topic with a long history, it has never been considered to be such a critically important topic for the field, as today. With the increasing sensitivity of analysis techniques, and advances in data interpretation using probabilistic models ('probabilistic genotyping'), there is an increasing demand on forensic biologists to share specialised knowledge to help recipients of expert information address mode and timing of transfer and persistence of traces in court. Scientists thereby have a critical role in the assessment of their findings in the context of the case. This helps the judiciary with activity level inferences in a balanced, robust and transparent way, when based on (1) proper case assessment and interpretation respecting the hierarchy of propositions (supported by, for example, the use of Bayesian networks as graphical models), (2) use of appropriate data to inform probabilities, and (3) reporting guidelines by international bodies. This critical review of current literature shows that with certain prerequisites for training and quality assurance, there is a solid foundation for evidence interpretation when propositions of interest are at the 'activity level'.

/pdf/evaluation-of-forensic-genetics-findings-given-activity-2h5gv6l919.pdf

Evaluation of forensic genetics findings given activity level propositions: A review

J Forensic Sci, July 2015, Vol. 60, No. 4 doi: 10.1111/1556-4029.12805 Available online at: onlinelibrary.wiley.com Letter to the Editor— Context Management Toolbox: A Linear Sequential Unmasking (LSU) Approach for Minimizing Cognitive Bias in Forensic Decision Making Sir, The 2009 NAS report (1) criticized forensic scientists for making insufficient efforts to reduce their vulnerability to cogni- tive and contextual bias. Over the past few years, however, the field has begun to take steps to address this issue. There have been major workshops on cognitive bias, and the Organization of Scientific Area Committees (OSAC), 1 as well as the National Commission on Forensic Science, have created committees on Human Factors that are specifically charged with examining this issue. 2 A number of tools and methods for minimizing bias are under consideration. Some of these tools have already been imple- mented in a few forensic laboratories. In general, these tools are designed to protect and enhance the independence of mind of forensic examiners, particularly those who rely on subjective judgment to make their decisions. Several types of contextual information are of concern, as illus- trated in Fig. 1. We organize them into a taxonomy of five levels (based on a four-level taxonomy suggested by Stoel et al. [2]). The five-level taxonomy differentiates task-irrelevant information that may be conveyed to an analyst by the trace evidence itself (Level 1), the reference samples (Level 2), the case information (Level 3), examiners’ base rate expectations that arise from their experience (e.g., when the examiner expects a particular result— Level 4), and organizational and cultural factors (Level 5). A variety of tools are available for addressing cognitive bias. Different tools are useful for managing exposure to each level of task-irrelevant information. For example, case managers (3,4) is a straightforward tool for dealing with bias from case informa- tion (Level 3). In general, these procedures are designed to pre- vent contextual bias by protecting the examiner from exposure to task-irrelevant information. However, it is important to note that some types of information, while potentially biasing, may also be task relevant (5). These types of biasing information are more difficult to deal with. For example, in some instances, evidence that analysts must examine to perform their duties may contain information that is potentially biasing. This can pertain to cases in which Level 1 information, the trace evidence being evaluated, contains contextual informa- tion (e.g., blood spatter patterns that contain information about the nature of the crime, or handwriting and voice samples in which the meaning of the words is potentially biasing). Reference samples are another example of relevant material that is also potentially biasing (Level 2). These samples are clearly relevant because the analyst must compare them to trace evidence samples to determine whether they are similar enough to conclude that they come from the same source. But it is pos- sible that an analyst’s interpretation of the trace evidence might inadvertently be influenced by knowing the characteristics of the See: http://www.nist.gov/forensics/osac/hfc.cfm. The authors of this letter are comprised primarily from members of the OSAC Human Factors Committee, the Human Factors Subcommittee of the National Commission on Forensic Science, and authors of the original Sequential Unmasking. © 2015 American Academy of Forensic Sciences Fig. 1––A taxonomy of the different levels at which task-irrelevant and potentially biasing contextual information may reach a forensic scientist. reference samples—a form of bias arising from circular reason- ing. To manage this type of bias, one must look to other tools in the context management toolbox. The most widely discussed tool for this purpose is sequential unmasking (6). In contrast to other context management tools (3,5), the sequential unmasking approach does not prevent exposure to biasing relevant information. However, it does mandate that this information be presented as late as possible in the exami- nation process and only when it is necessary. A critical ele- ment of sequential unmasking is that the forensic examiner must first examine and document the trace evidence from the crime scene (Level 1), before being exposed to the known reference material (Level 2). This approach is linear in the sense that one must begin with the trace evidence before being exposed to and working with the reference material, thus working from the evidence to the sus- pect, rather than from the suspect to the evidence. The trace evi- dence should be interpreted the same in this initial step, regardless of any suspect who is considered as a possible (or even likely) source of the trace evidence. However, according to sequential unmasking (6), examiners are permitted to revisit, as well as change, their initial analysis of the trace evidence once they have reviewed the reference material, provided that they document these changes. Sequential unmasking allows unlimited and unrestricted changes to the evidence once exposed to the reference material. We believe it is important to impose limits and restrictions for when examiners are permitted to revisit and alter their initial analysis of trace evidence. The analysis of traces is most objec- tive when the examination is “context free”—that is, prior to exposure to the known reference samples. However, seeing the reference samples could alert the examiner to a possible over- sight, error, or misjudgment in the analysis of the trace evidence. Here, we seek to strike a balance between restrictive proce- dures that forbid analysts from changing their opinion and those that allow unlimited and unrestricted changes. The requirement that changes be documented does not eliminate the possibility that such changes arose from bias—it only makes that possibility more transparent.

/pdf/letter-to-the-editor-context-management-toolbox-a-linear-3qvxylpqqy.pdf

Letter to the Editor- Context Management Toolbox: A Linear Sequential Unmasking (LSU) Approach for Minimizing Cognitive Bias in Forensic Decision Making.

Sir, In his review of George ‘‘Woody’’ Clarke’s book Justice and Science: Trials and Triumphs of DNA Evidence (1), Paul Colman quotes, with apparent approval, a passage in which Clarke falls victim to the well-known prosecutor’s fallacy (2–5). The passage says that DNA evidence allows analysts to reach statistically compelling conclusions, such as ‘‘the chance that someone else could have left that bloodstain is approximately 1 in a quintillion’’ (emphasis added). As every forensic scientist should know, the quoted statement is not a conclusion that a forensic scientist can properly draw from DNA evidence. DNA evidence can, at best, establish the conditional probability of observing a particular DNA profile if it came from someone other than the suspect. DNA evidence cannot, by itself, establish the chances that someone other than the suspect could have left a bloodstain (2–5). I assumed at first that Colman must have misquoted Clarke, but a quick review of the book showed otherwise. Clarke’s book is rife with instances of the prosecutor’s fallacy, including the following:

J.D. William C. Thompson Ph.D.

Papers

Letter to the Editor- Context Management Toolbox: A Linear Sequential Unmasking (LSU) Approach for Minimizing Cognitive Bias in Forensic Decision Making.

Letter to the Editor--The prosecutor's fallacy in George Clarke's Justice and Science: Trials and Triumphs of DNA Evidence.