Hallucinations and Constraints : Regulating surgical workflow recognition beyond accuracy

Abstract

Hallucinations are a major concern for the integration of artificial intelligence into medicine, although less explored in the realm of medical image processing. Unlike problems in natural text understanding and reasoning therewith, determining whether or not predictions derived from biomedical images and signals is less intuitively clear. This article suggests that topological errors could constitute hallucinations in a way that can be more readily measured and thus regulated. Certain of these properties for certain types of problems, such as biomedical signal segmentation, can be rephrased as linear temporal logic predicates, a number of which can be explicitly enforced using probabilistic graphical models. Our simulations show the potential of these explicitly constrained predicates for the case of automatic surgical phase recognition in robot-assisted hysterectomy, improving accuracy by approximately 10% while removing the vast majority of topological errors, suggesting that mathematical guarantees of correctness can supplement other empirical forms of regulating machine learning in medical image computing and computer-assisted interventions.