Multispectral Imaging Method in Laparoscopy

Introduction. At present, video data acquired in narrow spectral bands are widely used to improve the efficiency of diagnostics in various medical fields, laparoscopy in particular. Conventional laparoscopy uses images obtained in the white light. Images obtained in the visible range suitably depict the color and textural features of tissues. However, it is difficult for a physician to use visible images for distinguishing between lesion areas and normal tissues, largely due to their proximity in color and texture. The efficiency of lesion detection can be improved using fluorescence images, which clearly differentiate lesion areas from normal tissues. However, the use multispectral data implies the need to present the images obtained both in the white and fluorescence light to the physician. In this paper, we propose an image composition method based on visible and fluorescence images, which facilitates their analysis by physicians. A distinctive feature of the method is the use of CIEDE 2000 metric for image fusion, which takes the properties of human vision into account.

Aim. Development of a method for multispectral data visualization, which provides a physician with an image that combines white light data and a color-highlighted area of lesions.

Materials and methods. The proposed method consists of the following steps: preprocessing of images obtained in visible and fluorescence light; segmentation of the lesion area in the fluorescence images; generation of a pseudo-color image of the segmented lesion area; and fusion of the pseudo-color image with the image obtained in the white light.

Results. The proposed method forms an image that includes an image of the operation area obtained in the white light and a separated lesion area based on fluorescence information in the near infrared range. The image composite takes the properties of human vision into account. An experimental study of the method was carried out on actual laparoscopic images, involving endoscopists who were experts in subjective evaluation of video images. The method of paired comparisons was used to evaluated the presented images. The majority of experts preferred the fused image formed by the proposed method to those visualized simultaneously in the white and fluorescence light.

Conclusion. The developed method ensures generation of images with an increased diagnostic value.

1. . Aoki T., Murakami M., Koizumi T., Matsuda K., Fujimori A., Kusano T., Enami Y., Goto S., Watanabe M., Otsuka K. Determination of the Surgical Margin in Laparoscopic Liver Resections Using Infrared Indocyanine Green Fluorescence. Langenbeck's Archives of Surgery. 2018, vol. 403, pp. 671–680. doi: 10.1007/s00423-018-1685-y

2. Schaafsma B. E., Mieog S., Hutteman M., van der Vorst J. R., Kuppen P. J. K., Löwik C. W. G. M., Frangioni J. V., van de Velde C. J. H., Vahrmeijer A. L. The Clinical Use of Indocyanine Green as a Near‐ Infrared Fluorescent Contrast Agent for Image‐Guided Oncologic Surgery. J. of Surgical Oncology. 2011, vol. 104, iss. 3, pp. 323–332. doi: 10.1002/jso.21943

3. Boni L., David G., Mangano A., Dionigi G., Rausei S., Spampatti S., Cassinotti E., Fingerhut A. Clinical Applications of Indocyanine Green (ICG) Enhanced Fluorescence in Laparoscopic Surgery. Surgical Endoscopy. 2015, vol. 29, pp. 2046–2055. doi: 10.1007/s00464-014-3895-x

4. Nishino H., Hatano E., Seo S., Nitta T., Saito T., Nakamura M., Hattori K., Takatani M., Fuji H., Taura K., Uemoto S. Real-Time Navigation for Liver Surgery Using Projection Mapping with Indocyanine Green Fluorescence: Development of the Novel Medical Imaging Projection System. Annals of surgery. 2018, vol. 267, no. 6, pp. 1134–1140. doi: 10.1097/SLA.0000000000002172

5. Ambe P. C., Plambeck J., Fernandez-Jesberg V., Zarras K. The Role of Indocyanine Green Fluoroscopy for Intraoperative Bile Duct Visualization during Laparoscopic Cholecystectomy: an Observational Cohort Study in 70 Patients. Patient Safety in Surgery. 2019, vol. 13, pp. 1–7. doi: 10.1186/s13037-019-0182-8

6. Salem N., Malik H., Shams A. Medical Image Enhancement Based on Histogram Algorithms. Procedia Computer Science. 2019, vol. 163, pp. 300–311. doi: 10.1016/j.procs.2019.12.112

7. Arigela S., Asari V. K. A Locally Tuned Nonlinear Technique for Color Image Enhancement. WSEAS Transactions on Signal Processing. 2008, vol. 4, iss. 8, pp. 514–519.

8. Arnold M., Ghosh A., Ameling S., Lacey G. Automatic segmentation and inpainting of specular highlights for endoscopic imaging. EURASIP J. on Image and Video Processing. 2010, vol. 2010, pp. 1–12. doi: 10.1155/2010/814319

9. Guo D., Cheng Y., Zhuo S., Sim T. Correcting Over-Exposure in Photographs. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition. San Francisco, USA, 13–18 June 2010. IEEE, 2010, pp. 515–521. doi: 10.1109/CVPR.2010.5540170

10. Yoon Y.-J., Byun K.-Y., Lee D.-H., Jung S.-W., Ko S.-J. A New Human Perception-Based Over-Exposure Detection Method for Color Images. Sensors. 2014, vol. 14, iss. 9, pp. 17159–17173. doi: 10.3390/s140917159

11. Arnold M., Ghosh A., Ameling S., Gerard L. Automatic Segmentation and Inpainting of Specular Highlights for Endoscopic Imaging. EURASIP J. on Image and Video Processing. 2010, vol. 2010, pp. 1–12. doi: 10.1155/2010/814319

12. Luo M. R., Cui G., Rigg B. The Development of the CIE 2000 Colour‐Difference Formula: CIEDE2000. Color Research & Application. 2001, vol. 26, iss. 5, pp. 340–350. doi: 10.1002/col.1049

13. Obukhova N. A., Yang X. Automatic Method for Segmentation of Fluorescent Images Obtained in the Near-Infrared Region. J. of the Russian Universities. Radioelectronics. 2022, vol. 25, no. 6, pp. 40–49. doi: 10.32603/1993-8985-2022-25-6-40-49 (In Russ.)

14. Obukhova N. A., Motyko A. A. Automatic Method of Colposcopic Multi-Spectral Images Analysis for Television Systems Diagnostics of Cervical Cancer. J. of the Russian Universities. Radioelectronics. 2015, no. 6, pp. 24–33. (In Russ.)

15. Sinaga K. P., Miin-Shen Yang. Unsupervised K-Means Clustering Algorithm. IEEE Access. 2020, vol. 8, pp. 80716–80727. doi: 10.1109/ACCESS.2020.2988796

16. Syakur M. A., Khotimah B. K., Rochman E. M., Satoto B. D. Integration K-Means Clustering Method and Elbow Method for Identification of the Best Customer Profile Cluster. IOP Conf. Series: Materials Science And Engineering. 2018, vol. 336, art. 012017. doi: 10.1088/1757-899X/336/1/012017