Can I read Surgical-DeSAM: Decoupling SAM for Instrument Segmentation in Robotic Surgery on EtoBox?

Surgical-DeSAM: Decoupling SAM for Instrument Segmentation in Robotic Surgery by Sheng, Yuyang; Bano, Sophia; Clarkson, Matthew J.; Islam, Mobarakol is a scholarly article available to read on EtoBox.

What is Surgical-DeSAM: Decoupling SAM for Instrument Segmentation in Robotic Surgery about?

Purpose: The recent Segment Anything Model (SAM) has demonstrated impressive performance with point, text or bounding box prompts, in various applications. However, in safety-critical surgical tasks, prompting is not possible due to (i) the lack of per-frame prompts for supervised learning, (ii) it is unrealistic to prompt frame-by-frame in a real-time tracking application, and (iii) it is expensive to annotate prompts for offline applications. Methods: We develop Surgical-DeSAM to generate automatic bounding box prompts for decoupling SAM to obtain instrument segmentation in real-time robotic surgery. We utilise a commonly used detection architecture, DETR, and fine-tuned it to obtain bounding box prompt for the instruments. We then empolyed decoupling SAM (DeSAM) by replacing the image encoder with DETR encoder and fine-tune prompt encoder and mask decoder to obtain instance segmentation for the surgical instruments. To improve detection performance, we adopted the Swin-transformer to better feature representation. Results: The proposed method has been validated on two publicly available datasets from the MICCAI surgical instruments segmentation challenge EndoVis 2017 and 2018. T

Author

Sheng, Yuyang; Bano, Sophia; Clarkson, Matthew J.; Islam, Mobarakol

Published

2024

Language

EN