Semantic-SuPer: A Semantic-aware Surgical Perception Framework for Endoscopic Tissue Identification, Reconstruction, and Tracking

Shan Lin,Albert J. Miao,Jingpei Lu,Shunkai Yu,Zih-Yun Chiu,Florian Richter,Michael C. Yip,Shan Lin,Albert J. Miao,Jingpei Lu,Shunkai Yu,Zih-Yun Chiu,Florian Richter,Michael C. Yip

Accurate and robust tracking and reconstruction of the surgical scene is a critical enabling technology toward autonomous robotic surgery. Existing algorithms for 3D perception in surgery mainly rely on geometric information, while we propose to also leverage semantic information inferred from the endoscopic video using image segmentation algorithms. In this paper, we present a novel, comprehensiv...