LAUSR

In their article, ‘Responsibility, Second Opinions, and PeerDisagreement— Ethical and Epistemological Challenges of Using AI in Clinical Diagnostic Contexts,’ Kempt and Nagel argue for a ‘rule of disagreement’ for the integration of diagnostic AI in healthcare contexts. The type of AI in question is a ‘decision support system’ (DSS), the purpose of which is to augment human judgement and decisionmaking in the clinical context by automating or supplementing parts of the cognitive labor. Under the authors’ proposal, artificial decision support systems (AIDSS) which produce automated diagnoses should serve chiefly as confirmatory tools; so long as the physician and AI agree, the matter is settled, and the physician’s initial judgement is considered epistemically justified. If, however, the AIDSS and physician disagree, then a second physician’s opinion is called on to resolve the dispute. While the cognitive labour of the decision is shared between the physicians and AI, the final decision remains at the discretion of the first physician, and with it the moral and legal culpability. The putative benefits of this approach are twofold: (1) healthcare administration can improve diagnostic performance by introducing AIDSS without the unintended byproduct of a responsibility gap, and (2) assuming the physician and AI disagree less than the general rate of requested second opinions, and the AI’s diagnostic accuracy supersedes or at least matches human performance, the result should be a streamlined clinical flow requiring less second opinions and reducing redundancy. Central to the authors’ relegation of AIDSS to a confirmatory role, as opposed to fully substituting as a second opinion, is their contention that an ‘equal views principle’ cannot apply to heterogeneous agents (ie, AI and humans). An AI can contradict a physician but cannot explain itself or enter into a ‘peer disagreement’ because it cannot engage in dialectic or reasongiving the way a physician giving a second opinion can. In this way, the physician–AI relationship creates an asymmetry not present in an ordinary physician–physician peer relationship by burdening the physician with the responsibility of interpreting the AIs outputs. The authors’ view has several advantages as noted, but it seems to rely on an idealised picture of highly conscientious and rigorous physician–physician dialectic being the norm for second opinions, and this premise may be brought into question. The authors themselves acknowledge that the process of requesting a second opinion ‘is most often of informal nature’ and ‘a part of everyday clinical life’. Informality or routinisation do not alone imply a lack of diligence, but without empirical study, it seems difficult to know that the content of second opinion dialogue is uniformly oriented towards reasongiving and evidential argumentation, as opposed to an exchange of clinical gestalt. Moreover, if the second physician cites a subjectively derived or insufficiently operationalised criteria for their opinion (ie, ‘patient frailty’), it is not clear that their input is any less of a black box than the AI.