The inferences of most machine-learning models powering medical artificial intelligence are difficult to interpret. Here we report a general framework for model auditing that combines insights from medical experts with a highly expressive form of explainable artificial intelligence. Specifically, we leveraged the expertise of dermatologists for the clinical task of differentiating melanomas from melanoma ‘lookalikes’ on the basis of dermoscopic and clinical images of the skin, and the power of generative models to render ‘counterfactual’ images to understand the ‘reasoning’ processes of five medical-image classifiers. By altering image attributes to produce analogous images that elicit a different prediction by the classifiers, and by asking physicians to identify medically meaningful features in the images, the counterfactual images revealed that the classifiers rely both on features used by human dermatologists, such as lesional pigmentation patterns, and on undesirable features, such as background skin texture and colour balance. The framework can be applied to any specialized medical domain to make the powerful inference processes of machine-learning models medically understandable.

大多数支持医疗人工智能的机器学习模型的推论很难解释。在这里，我们报告了一个模型审计的通用框架，它将医学专家的见解与高度表达的可解释人工智能形式相结合。具体来说，我们利用皮肤科医生的专业知识来完成基于皮肤镜和皮肤临床图像区分黑色素瘤和“相似”黑色素瘤的临床任务，以及生成模型渲染“反事实”图像以理解“推理”的能力五个医学图像分类器的过程。通过改变图像属性来生成引起分类器不同预测的类似图像，并要求医生识别图像中具有医学意义的特征，反事实图像显示分类器依赖于人类皮肤科医生使用的特征，例如病变色素沉着图案，以及不需要的特征，例如背景皮肤纹理和色彩平衡。该框架可以应用于任何专业医学领域，使机器学习模型的强大推理过程在医学上易于理解。