Nuclei detection and segmentation in hematoxylin and eosin-stained (H&E) tissue images are important clinical tasks and crucial for a wide range of applications. However, it is a challenging task due to nuclei variances in staining and size, overlapping boundaries, and nuclei clustering. While convolutional neural networks have been extensively used for this task, we explore the potential of Transformer-based networks in combination with large scale pre-training in this domain. Therefore, we introduce a new method for automated instance segmentation of cell nuclei in digitized tissue samples using a deep learning architecture based on Vision Transformer called CellViT. CellViT is trained and evaluated on the PanNuke dataset, which is one of the most challenging nuclei instance segmentation datasets, consisting of nearly 200,000 annotated nuclei into 5 clinically important classes in 19 tissue types. We demonstrate the superiority of large-scale in-domain and out-of-domain pre-trained Vision Transformers by leveraging the recently published and a ViT-encoder pre-trained on 104 million histological image patches — achieving state-of-the-art nuclei detection and instance segmentation performance on the PanNuke dataset with a mean panoptic quality of 0.50 and an -detection score of 0.83. The code is publicly available at .

中文翻译：

---

CellViT：用于精确细胞分割和分类的视觉转换器

苏木精和伊红染色 (H&E) 组织图像中的细胞核检测和分割是重要的临床任务，对于广泛的应用至关重要。然而，由于细胞核染色和大小、重叠边界和细胞核聚类的差异，这是一项具有挑战性的任务。虽然卷积神经网络已广泛用于此任务，但我们探索基于 Transformer 的网络与该领域的大规模预训练相结合的潜力。因此，我们引入了一种新方法，使用基于 Vision Transformer 的深度学习架构（称为 CellViT）对数字化组织样本中的细胞核进行自动实例分割。 CellViT 在 PanNuke 数据集上进行训练和评估，该数据集是最具挑战性的细胞核实例分割数据集之一，由近 200,000 个带注释的细胞核组成，分为 19 种组织类型的 5 个临床重要类别。我们利用最近发布的 ViT 编码器以及在 1.04 亿个组织学图像块上进行预训练的 ViT 编码器，展示了大规模域内和域外预训练 Vision Transformer 的优越性 - 实现了最先进的PanNuke 数据集上的核检测和实例分割性能，平均全景质量为 0.50，检测得分为 0.83。该代码可在 公开获取。