Can I read Handling Geometric Domain Shifts in Semantic Segmentation of Surgical RGB and Hyperspectral Images on EtoBox?

Handling Geometric Domain Shifts in Semantic Segmentation of Surgical RGB and Hyperspectral Images by Seidlitz, Silvia; Sellner, Jan; Studier-Fischer, Alexander; Motta, Alessandro; Özdemir, Berkin; Müller-Stich, Beat P.; Nickel, Felix; Maier-Hein, Lena is a scholarly article available to read on EtoBox.

What is Handling Geometric Domain Shifts in Semantic Segmentation of Surgical RGB and Hyperspectral Images about?

Robust semantic segmentation of intraoperative image data holds promise for enabling automatic surgical scene understanding and autonomous robotic surgery. While model development and validation are primarily conducted on idealistic scenes, geometric domain shifts, such as occlusions of the situs, are common in real-world open surgeries. To close this gap, we (1) present the first analysis of state-of-the-art (SOA) semantic segmentation models when faced with geometric out-of-distribution (OOD) data, and (2) propose an augmentation technique called "Organ Transplantation", to enhance generalizability. Our comprehensive validation on six different OOD datasets, comprising 600 RGB and hyperspectral imaging (HSI) cubes from 33 pigs, each annotated with 19 classes, reveals a large performance drop in SOA organ segmentation models on geometric OOD data. This performance decline is observed not only in conventional RGB data (with a dice similarity coefficient (DSC) drop of 46 %) but also in HSI data (with a DSC drop of 45 %), despite the richer spectral information content. The performance decline increases with the spatial granularity of the input data. Our augmentation technique improv

Author

Seidlitz, Silvia; Sellner, Jan; Studier-Fischer, Alexander; Motta, Alessandro; Özdemir, Berkin; Müller-Stich, Beat P.; Nickel, Felix; Maier-Hein, Lena

Published

2024

Language

EN