The power of Artificial Intelligence (AI) combined with the surgeons’ expertise leads to breakthroughs in surgical care, bringing new hope to patients. Utilizing deep learning-based computer vision techniques in surgical procedures will enhance the healthcare industry. Laparoscopic surgery holds excellent potential for computer vision due to the abundance of real-time laparoscopic recordings captured by digital cameras containing significant unexplored information. Furthermore, with computing power resources becoming increasingly accessible and Machine Learning methods expanding across various industries, the potential for AI in healthcare is vast. There are several objectives of AI’s contribution to laparoscopic surgery; one is an image guidance system to identify anatomical structures in real-time. However, few studies are concerned with intraoperative anatomy recognition in laparoscopic surgery. This study provides a comprehensive review of the current state-of-the-art semantic segmentation techniques, which can guide surgeons during laparoscopic procedures by identifying specific anatomical structures for dissection or avoiding hazardous areas. This review aims to enhance research in AI for surgery to guide innovations towards more successful experiments that can be applied in real-world clinical settings. This AI contribution could revolutionize the field of laparoscopic surgery and improve patient outcomes.

Ahmed, Shams Forruque, Md. Sakib Bin Alam, Maruf Hassan, Mahtabin Rodela Rozbu, Taoseef Ishtiak, Nazifa Rafa, M. Mofijur, A. B. M. Shawkat Ali, and Amir H. Gandomi. 2023. “Deep Learning Modelling Techniques: Current Progress, Applications, Advantages, and Challenges.” Artificial Intelligence Review 56(11):13521–617. doi: 10.1007/s10462-023-10466-8.

Alazab, Mamoun, and MingJian Tang. 2019. Deep Learning Applications for Cyber Security. Springer. doi: 10.1007/978-3-030-13057-2

Amiri, Zahra, Arash Heidari, Nima Jafari Navimipour, Mansour Esmaeilpour, and Yalda Yazdani. 2024. “The Deep Learning Applications in IoT-Based Bio- and Medical Informatics: A Systematic Literature Review.” Neural Computing and Applications 36(11):5757–97. doi: 10.1007/s00521-023-09366-3.

Anteby, Roi, Nir Horesh, Shelly Soffer, Yaniv Zager, Yiftach Barash, Imri Amiel, Danny Rosin, Mordechai Gutman, and Eyal Klang. 2021. “Deep Learning Visual Analysis in Laparoscopic Surgery: A Systematic Review and Diagnostic Test Accuracy Meta-Analysis.” Surgical Endoscopy 35(4):1521–33. doi: 10.1007/s00464-020-08168-1.

Badgery, Henry, Yuning Zhou, Athanasia Siderellis, Matthew Read, and Catherine Davey. 2022. “Machine Learning in Laparoscopic Surgery.” Pp. 175–90 in Artificial Intelligence in Medicine: Applications, Limitations and Future Directions. Springer. doi: 10.1007/978-981-19-1223-8_8.

Bamba, Yoshiko, Shimpei Ogawa, Michio Itabashi, Hironari Shindo, Shingo Kameoka, Takahiro Okamoto, and Masakazu Yamamoto. 2021. “Object and Anatomical Feature Recognition in Surgical Video Images Based on a Convolutional Neural Network.” International Journal of Computer Assisted Radiology and Surgery 16(11):2045–54. doi: 10.1007/s11548-021-02434-w.

Beyersdorffer, Patrick, Wolfgang Kunert, Kai Jansen, Johanna Miller, Peter Wilhelm, Oliver Burgert, Andreas Kirschniak, and Jens Rolinger. 2021. “Detection of Adverse Events Leading to Inadvertent Injury during Laparoscopic Cholecystectomy Using Convolutional Neural Networks.” 66(4):413–21. doi: doi:10.1515/bmt-2020-0106.

Cao, Yue, Thomas Andrew Geddes, Jean Yee Hwa Yang, and Pengyi Yang. 2020. “Ensemble Deep Learning in Bioinformatics.” Nature Machine Intelligence 2(9):500–508. doi: 10.1038/s42256-020-0217-y.

Carass, Aaron, Snehashis Roy, Adrian Gherman, Jacob C. Reinhold, Andrew Jesson, Tal Arbel, Oskar Maier, Heinz Handels, Mohsen Ghafoorian, and Bram Platel. 2020. “Evaluating White Matter Lesion Segmentations with Refined Sørensen-Dice Analysis.” Scientific Reports 10(1):8242. doi.org/10.1038/s41598-020-64803-w.

Eelbode, Tom, Jeroen Bertels, Maxim Berman, Dirk Vandermeulen, Frederik Maes, Raf Bisschops, and Matthew B. Blaschko. 2020. “Optimization for Medical Image Segmentation: Theory and Practice When Evaluating with Dice Score or Jaccard Index.” IEEE Transactions on Medical Imaging 39(11):3679–90. doi: 10.1109/TMI.2020.3002417.

Esteva, Andre, Alexandre Robicquet, Bharath Ramsundar, Volodymyr Kuleshov, Mark DePristo, Katherine Chou, Claire Cui, Greg Corrado, Sebastian Thrun, and Jeff Dean. 2019. “A Guide to Deep Learning in Healthcare.” Nature Medicine 25(1):24–29. doi: 10.1038/s41591-018-0316-z.

Gibson, Eli, Maria R. Robu, Stephen Thompson, P. Eddie Edwards, Crispin Schneider, Kurinchi Gurusamy, Brian Davidson, David J. Hawkes, Dean C. Barratt, and Matthew J. Clarkson. 2017. “Deep Residual Networks for Automatic Segmentation of Laparoscopic Videos of the Liver.” Pp. 423–28 in Medical Imaging 2017: Image-Guided Procedures, Robotic Interventions, and Modeling. Vol. 10135. SPIE. doi.org/10.1117/12.2255975

Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT press.

Gu, Jiuxiang, Zhenhua Wang, Jason Kuen, Lianyang Ma, Amir Shahroudy, Bing Shuai, Ting Liu, Xingxing Wang, Gang Wang, and Jianfei Cai. 2018. “Recent Advances in Convolutional Neural Networks.” Pattern Recognition 77:354–77. https://doi.org/10.1016/j.patcog.2017.10.013

Guo, Kangwei, Haisu Tao, Yilin Zhu, Baihong Li, Chihua Fang, Yinling Qian, and Jian Yang. 2023. “Current Applications of Artificial Intelligence-Based Computer Vision in Laparoscopic Surgery.” Laparoscopic, Endoscopic and Robotic Surgery. doi: https://doi.org/10.1016/j.lers.2023.07.001.

Hashimoto, Daniel A., Guy Rosman, Daniela Rus, and Ozanan R. Meireles. 2018. “Artificial Intelligence in Surgery: Promises and Perils.” Annals of Surgery 268(1):70. https://doi.org/10.1097/SLA.0000000000002693.

Hashimoto, Daniel A., Thomas M. Ward, and Ozanan R. Meireles. 2020. “The Role of Artificial Intelligence in Surgery.” Advances in Surgery 54:89–101. https://doi.org/10.1016/j.yasu.2020.05.010.

Igaki, Takahiro, Daichi Kitaguchi, Shigehiro Kojima, Hiro Hasegawa, Nobuyoshi Takeshita, Kensaku Mori, Yusuke Kinugasa, and Masaaki Ito. 2022. “Artificial Intelligence-Based Total Mesorectal Excision Plane Navigation in Laparoscopic Colorectal Surgery.” Diseases of the Colon & Rectum 65(5):e329–33. https://doi.org/10.1097/DCR.0000000000002393

Jauhar, Sunil Kumar, Praveen Vijaya Raj Pushpa Raj, Sachin Kamble, Saurabh Pratap, Shivam Gupta, and Amine Belhadi. 2024. “A Deep Learning-Based Approach for Performance Assessment and Prediction: A Case Study of Pulp and Paper Industries.” Annals of Operations Research 332(1):405–31. doi: 10.1007/s10479-022-04528-3.

Jordan, Michael I., and Tom M. Mitchell. 2015. “Machine Learning: Trends, Perspectives, and Prospects.” Science 349(6245):255–60. https://doi.org/aaa8415

Kitaguchi, Daichi, Nobuyoshi Takeshita, Hiro Hasegawa, and Masaaki Ito. 2022. “Artificial Intelligence‐based Computer Vision in Surgery: Recent Advances and Future Perspectives.” Annals of Gastroenterological Surgery 6(1):29–36. https://doi.org/10.1002/ags3.12513

Kitaguchi, Daichi, Nobuyoshi Takeshita, Hiroki Matsuzaki, Hiro Hasegawa, Ryoya Honda, Koichi Teramura, Tatsuya Oda, and Masaaki Ito. 2021. “Computer-Assisted Real-Time Automatic Prostate Segmentation during TaTME: A Single-Center Feasibility Study.” Surgical Endoscopy 35:2493–99. https://doi.org/10.1007/s00464-020-07659-5

Kitaguchi, Daichi, Nobuyoshi Takeshita, Hiroki Matsuzaki, Hiroaki Takano, Yohei Owada, Tsuyoshi Enomoto, Tatsuya Oda, Hirohisa Miura, Takahiro Yamanashi, and Masahiko Watanabe. 2020. “Real-Time Automatic Surgical Phase Recognition in Laparoscopic Sigmoidectomy Using the Convolutional Neural Network-Based Deep Learning Approach.” Surgical Endoscopy 34(11):4924–31. https://doi.org/10.1007/s00464-019-07281-0

Kitaguchi, Daichi, Nobuyoshi Takeshita, Hiroki Matsuzaki, Takahiro Igaki, Hiro Hasegawa, Shigehiro Kojima, Kensaku Mori, and Masaaki Ito. 2022. “Real-Time Vascular Anatomical Image Navigation for Laparoscopic Surgery: Experimental Study.” Surgical Endoscopy 36(8):6105–12. https://doi.org/10.1007/s00464-022-09384-7

Kojima, Shigehiro, Daichi Kitaguchi, Takahiro Igaki, Kei Nakajima, Yuto Ishikawa, Yuriko Harai, Atsushi Yamada, Younae Lee, Kazuyuki Hayashi, Norihito Kosugi, Hiro Hasegawa, and Masaaki Ito. 2023. “Deep-Learning-Based Semantic Segmentation of Autonomic Nerves from Laparoscopic Images of Colorectal Surgery: An Experimental Pilot Study.” International Journal of Surgery 109(4). https://doi.org/10.1097/JS9.0000000000000317

Kolbinger, Fiona R., Franziska M. Rinner, Alexander C. Jenke, Matthias Carstens, Stefanie Krell, Stefan Leger, Marius Distler, Jürgen Weitz, Stefanie Speidel, and Sebastian Bodenstedt. 2023. “Anatomy Segmentation in Laparoscopic Surgery: Comparison of Machine Learning and Human Expertise–an Experimental Study.” International Journal of Surgery 10–1097. https://doi.org/10.1097/JS9.0000000000000595

Kolbinger, Fiona R., Sebastian Bodenstedt, Matthias Carstens, Stefan Leger, Stefanie Krell, Franziska M. Rinner, Thomas P. Nielen, Johanna Kirchberg, Johannes Fritzmann, and Jürgen Weitz. 2023. “Artificial Intelligence for Context-Aware Surgical Guidance in Complex Robot-Assisted Oncological Procedures: An Exploratory Feasibility Study.” European Journal of Surgical Oncology 106996. https://doi.org/https://doi.org/10.1016/j.ejso.2023.106996

Laplante, Simon, Babak Namazi, Parmiss Kiani, Daniel A. Hashimoto, Adnan Alseidi, Mauricio Pasten, L. Michael Brunt, Sujata Gill, Brian Davis, Matthew Bloom, Luise Pernar, Allan Okrainec, and Amin Madani. 2023. “Validation of an Artificial Intelligence Platform for the Guidance of Safe Laparoscopic Cholecystectomy.” Surgical Endoscopy 37(3):2260–68. doi: 10.1007/s00464-022-09439-9.

LeCun, Yann, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. “Gradient-Based Learning Applied to Document Recognition.” Proceedings of the IEEE 86(11):2278–2324. https://doi.org/10.1109/5.726791

LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. 2015. “Deep Learning.” Nature 521(7553):436–44. doi: 10.1038/nature14539.

Lee, Daeeop, Giha Lee, Seongwon Kim, and Sungho Jung. 2020. “Future Runoff Analysis in the Mekong River Basin under a Climate Change Scenario Using Deep Learning.” Water 12(6):1556. https://doi.org/10.3390/w12061556

Liu, Xiangbin, Liping Song, Shuai Liu, and Yudong Zhang. 2021. “A Review of Deep-Learning-Based Medical Image Segmentation Methods.” Sustainability 13(3):1224. https://doi.org/10.3390/su13031224

Madad Zadeh, Sabrina, Tom Francois, Lilian Calvet, Pauline Chauvet, Michel Canis, Adrien Bartoli, and Nicolas Bourdel. 2020. “SurgAI: Deep Learning for Computerized Laparoscopic Image Understanding in Gynaecology.” Surgical Endoscopy 34(12):5377–83. doi: 10.1007/s00464-019-07330-8.

Madani, Amin, Babak Namazi, Maria S. Altieri, Daniel A. Hashimoto, Angela Maria Rivera, Philip H. Pucher, Allison Navarrete-Welton, Ganesh Sankaranarayanan, L. Michael Brunt, and Allan Okrainec. 2022. “Artificial Intelligence for Intraoperative Guidance: Using Semantic Segmentation to Identify Surgical Anatomy during Laparoscopic Cholecystectomy.” Annals of Surgery. https://doi.org/10.1097/SLA.0000000000004594

Madani, Amin, Melina C. Vassiliou, Yusuke Watanabe, Becher Al-Halabi, Mohammed S. Al-Rowais, Dan L. Deckelbaum, Gerald M. Fried, and Liane S. Feldman. 2017. “What Are the Principles That Guide Behaviors in the Operating Room?” Annals of Surgery 265(2):255–67. https://doi.org/10.1097/SLA.0000000000001962

Madani, Amin, Yusuke Watanabe, Liane S. Feldman, Melina C. Vassiliou, Jeffrey S. Barkun, Gerald M. Fried, and Rajesh Aggarwal. 2015. “Expert Intraoperative Judgment and Decision-Making: Defining the Cognitive Competencies for Safe Laparoscopic Cholecystectomy.” Journal of the American College of Surgeons 221(5):931–40. https://doi.org/10.1097/SLA.0000000000004736

Maier-Hein, Lena, Matthias Eisenmann, Duygu Sarikaya, Keno März, Toby Collins, Anand Malpani, Johannes Fallert, Hubertus Feussner, Stamatia Giannarou, and Pietro Mascagni. 2022. “Surgical Data Science–from Concepts toward Clinical Translation.” Medical Image Analysis 76:102306. https://doi.org/10.1016/j.media.2021.102306

Mascagni, Pietro, Armine Vardazaryan, Deepak Alapatt, Takeshi Urade, Taha Emre, Claudio Fiorillo, Patrick Pessaux, Didier Mutter, Jacques Marescaux, and Guido Costamagna. 2022. “Artificial Intelligence for Surgical Safety: Automatic Assessment of the Critical View of Safety in Laparoscopic Cholecystectomy Using Deep Learning.” Annals of Surgery 275(5):955–61. https://doi.org/10.1097/SLA.0000000000004351

Mascagni, Pietro, Deepak Alapatt, Takeshi Urade, Armine Vardazaryan, Didier Mutter, Jacques Marescaux, Guido Costamagna, Bernard Dallemagne, and Nicolas Padoy. 2021. “A Computer Vision Platform to Automatically Locate Critical Events in Surgical Videos: Documenting Safety in Laparoscopic Cholecystectomy.” Annals of Surgery 274(1):e93–95. https://doi.org/10.1097/SLA.0000000000004736

Müller, Dominik, Iñaki Soto-Rey, and Frank Kramer. 2022. “Towards a Guideline for Evaluation Metrics in Medical Image Segmentation.” BMC Research Notes 15(1):210. doi: 10.1186/s13104-022-06096-y.

Narihiro, Satoshi, Daichi Kitaguchi, Hiro Hasegawa, Nobuyoshi Takeshita, and Masaaki Ito. 2024. “Deep Learning–Based Real-Time Ureter Identification in Laparoscopic Colorectal Surgery.” Diseases of the Colon & Rectum 67(10). DOI: 10.1097/DCR.0000000000003335.

Padoy, Nicolas. 2019. “Machine and Deep Learning for Workflow Recognition during Surgery.” Minimally Invasive Therapy & Allied Technologies 28(2):82–90. https://doi.org/10.1080/13645706.2019.1584116

Quero, Giuseppe, Pietro Mascagni, Fiona R. Kolbinger, Claudio Fiorillo, Davide De Sio, Fabio Longo, Carlo Alberto Schena, Vito Laterza, Fausto Rosa, Roberta Menghi, Valerio Papa, Vincenzo Tondolo, Caterina Cina, Marius Distler, Juergen Weitz, Stefanie Speidel, Nicolas Padoy, and Sergio Alfieri. 2022. “Artificial Intelligence in Colorectal Cancer Surgery: Present and Future Perspectives.” Cancers 14(15). doi: 10.3390/cancers14153803.

Rahman, Anichur, Tanoy Debnath, Dipanjali Kundu, Md Saikat Islam Khan, Airin Afroj Aishi, Sadia Sazzad, Mohammad Sayduzzaman, and Shahab S. Band. 2024. “Machine Learning and Deep Learning-Based Approach in Smart Healthcare: Recent Advances, Applications, Challenges and Opportunities.” AIMS Public Health 11(1):58. doi: 10.3934/publichealth.2024004

Rasp, Stephan, Michael S. Pritchard, and Pierre Gentine. 2018. “Deep Learning to Represent Subgrid Processes in Climate Models.” Proceedings of the National Academy of Sciences 115(39):9684–89. doi: 10.1073/pnas.1810286115.

Rivas, Homero, and Daniela Díaz-Calderón. 2013. “Present and Future Advanced Laparoscopic Surgery.” Asian Journal of Endoscopic Surgery 6(2):59–67. doi: 10.1111/ases.12028. https://doi.org/10.1111/ases.12028

Ross, Tobias, Annika Reinke, Peter M. Full, Martin Wagner, Hannes Kenngott, Martin Apitz, Hellena Hempe, Diana Mindroc-Filimon, Patrick Scholz, and Thuy Nuong Tran. 2021. “Comparative Validation of Multi-Instance Instrument Segmentation in Endoscopy: Results of the ROBUST-MIS 2019 Challenge.” Medical Image Analysis 70:101920. https://doi.org/10.1016/j.media.2020.101920

Russell, Stuart J., and Peter Norvig. 2016. Artificial Intelligence: A Modern Approach. Pearson.

Safarov, Furkat, Alpamis Kutlimuratov, Akmalbek Bobomirzaevich Abdusalomov, Rashid Nasimov, and Young-Im Cho. 2023. “Deep Learning Recommendations of E-Education Based on Clustering and Sequence.” Electronics 12(4):809. https://doi.org/10.3390/electronics12040809.

Sarker, Iqbal H. 2021. “Deep Cybersecurity: A Comprehensive Overview from Neural Network and Deep Learning Perspective.” SN Computer Science 2(3):154. doi: 10.1007/s42979-021-00535-6.

Scheikl, Paul Maria, Stefan Laschewski, Anna Kisilenko, Tornike Davitashvili, Benjamin Müller, Manuela Capek, Beat P. Müller-Stich, Martin Wagner, and Franziska Mathis-Ullrich. 2020. “Deep Learning for Semantic Segmentation of Organs and Tissues in Laparoscopic Surgery.” 6(1). doi: doi:10.1515/cdbme-2020-0016.

Sengun, Berke, Yalin Iscan, Gozde A. Tataroglu Ozbulak, Nida Kumbasar, Emre Egriboz, Ismail C. Sormaz, Nihat Aksakal, Sencer M. Deniz, Mehmet Haklidir, and Fatih Tunca. 2023. “Artificial Intelligence in Minimally Invasive Adrenalectomy: Using Deep Learning to Identify the Left Adrenal Vein.” Surgical Laparoscopy Endoscopy & Percutaneous Techniques 33(4):327–31. https://doi.org/10.1097/SLE.0000000000001185.

Sengupta, Saptarshi, Sanchita Basak, Pallabi Saikia, Sayak Paul, Vasilios Tsalavoutis, Frederick Atiah, Vadlamani Ravi, and Alan Peters. 2020. “A Review of Deep Learning with Special Emphasis on Architectures, Applications and Recent Trends.” Knowledge-Based Systems 194:105596. doi: https://doi.org/10.1016/j.knosys.2020.105596.

Shin, Hyung-Sup, and Ayoung Woo. 2024. “Analyzing the Effects of Walkable Environments on Nearby Commercial Property Values Based on Deep Learning Approaches.” Cities 144:104628. doi: https://doi.org/10.1016/j.cities.2023.104628.

Silva, B., B. Oliveira, P. Morais, L. R. Buschle, J. Correia–Pinto, E. Lima, and J. L. Vilaça. 2022. “Analysis of Current Deep Learning Networks for Semantic Segmentation of Anatomical Structures in Laparoscopic Surgery.” Pp. 3502–5 in 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). https://doi.org/10.1109/EMBC48229.2022.9871583.

Soffer, Shelly, Avi Ben-Cohen, Orit Shimon, Michal Marianne Amitai, Hayit Greenspan, and Eyal Klang. 2019. “Convolutional Neural Networks for Radiologic Images: A Radiologist’s Guide.” Radiology 290(3):590–606. https://doi.org/10.1148/radiol.2018180547.

Soori, Mohsen, Behrooz Arezoo, and Roza Dastres. 2023. “Artificial Intelligence, Machine Learning and Deep Learning in Advanced Robotics, a Review.” Cognitive Robotics 3:54–70. doi: https://doi.org/10.1016/j.cogr.2023.04.001.

Spencer, Frederick. 1978. “Teaching and Measuring Surgical Techniques: The Technical Evaluation of Competence.” Bull Am Coll Surg 63(3):9–12.

Suliburk, James W., Quentin M. Buck, Chris J. Pirko, Nader N. Massarweh, Neal R. Barshes, Hardeep Singh, and Todd K. Rosengart. 2019. “Analysis of Human Performance Deficiencies Associated with Surgical Adverse Events.” JAMA Network Open 2(7):e198067–e198067. https://doi.org /10.1001/jamanetworkopen.2019.8067

Taha-Mehlitz, Stephanie, Silvio Däster, Laura Bach, Vincent Ochs, Markus von Flüe, Daniel Steinemann, and Anas Taha. 2022. “Modern Machine Learning Practices in Colorectal Surgery: A Scoping Review.” Journal of Clinical Medicine 11(9):2431. https://doi.org/10.3390/jcm11092431

Taher, H., V. Grasso, S. Tawfik, and A. Gumbs. 2022. “The Challenges of Deep Learning in Artificial Intelligence and Autonomous Actions in Surgery: A Literature Review.” Art Int Surg 2:144–58. https://doi.org/10.20517/ais.2022.11

Tomimaru, Yoshito, Kozo Noguchi, Shunji Morita, Hiroshi Imamura, Takashi Iwazawa, and Keizo Dono. 2018. “Is Intraoperative Blood Loss Underestimated in Patients Undergoing Laparoscopic Hepatectomy?” World Journal of Surgery 42:3685–91. https://doi.org/10.1007/s00268-018-4655-1

Twinanda, Andru P., Pramita Winata, Afshin Gangi, M. D. Mathelin, and N. Padoy. 2016. “Multi-Stream Deep Architecture for Surgical Phase Recognition on Multi-View RGBD Videos.” Pp. 1–8 in Proc. M2CAI Workshop MICCAI.

Yamazaki, Yuta, Shingo Kanaji, Takeru Matsuda, Taro Oshikiri, Tetsu Nakamura, Satoshi Suzuki, Yuta Hiasa, Yoshito Otake, Yoshinobu Sato, and Yoshihiro Kakeji. 2020. “Automated Surgical Instrument Detection from Laparoscopic Gastrectomy Video Images Using an Open Source Convolutional Neural Network Platform.” Journal of the American College of Surgeons 230(5):725–32. https://doi.org/10.1016/j.jamcollsurg.2020.01.037

Ziogas, Ioannis A., and Georgios Tsoulfas. 2017. “Advances and Challenges in Laparoscopic Surgery in the Management of Hepatocellular Carcinoma.” World Journal of Gastrointestinal Surgery 9(12):233–45. https://doi.org/10.4240/wjgs.v9.i12.233