LAUSR

Editor, We thank the correspondents for reviewing instances of poor comparison performance that we reported for a fraction of the firearms examiners participating in our accuracy study [1]. Unfortunately, the inferences that they draw fail to consider the fact that quality control measures normally operational in case work were expressly excluded from the research study. It is not possible in any field to guarantee that a selected individual practitioner is proficient but extant data indicate that most firearms examiners are proficient, based on the convenience samples we must rely on. Our study, together with numerous other studies involving different examiners and test materials, shows that the majority of examiners can perform firearms comparisons with high accuracy [1– 19]. We reported that most errors were committed by a minority of examiners: 6 examiners account for ~30% of all errors and 13 account for ~50% of errors. Not all errors are of equal consequence. Longstanding legal precepts accept that falsenegative errors unfortunately may acquit the guilty but false-positive errors are acutely consequential. The letter therefore rightly concentrates on the number of false positives reported. Table 4 of our article, reproduced below, summarizes errors of both types [1]. No incorrect identifications or eliminations were made by 139/137 examiners (for bullets/cartridge cases), or roughly 80% of the 173 total number of examiners. For bullet comparisons, 10 examiners (5.8% of 173) made 20 falsepositive errors, while for cartridge cases, 18 examiners (10.4%) made 26 falsepositive errors. We calculated error as incorrect identifications or eliminations relative to all comparisons conducted. Reflective of our design decisions to provide challenging specimens in an openset format [20], many comparisons resulted in an inconclusive decision, particularly for nonmatching sets (65.4% for bullets, 50.6% for cartridge cases). By excluding inconclusive decisions from the denominator, the correspondents generate much higher error rates (their Tables 1 and 2) than we tabulated. The letter posits that, because certain participants were particularly error prone, and because nearly all the participants are employed by an accredited laboratory, therefore laboratory accreditation is an unreliable indicator of quality work by its forensic staff. The correspondents further extend this reasoning with the proposal that firearms testimony not be admitted in court proceedings, as adequate individual proficiency testing is lacking. We remind the correspondents that our study gave explicit direction to the participants that all comparisons were to be conducted independently, without consultation with colleagues, and outside of their laboratory's quality system. The most important consequence of the latter restriction is the absence of any verification process, which is normally part of the quality assurance protocols required to attain and maintain laboratory accreditation. With verification in place, the likelihood of a false positive or false negative passing into a final laboratory report becomes negligible. Results from our repeatability and reproducibility study support the efficacy of verification to detect and defeat infrequent errors. Interexaminer comparisons of the same 1886 nonmatching bullet sets resulted in two falsepositive errors (0.1%), while among 1913 paired comparisons of nonmatching cartridge case sets there were