توسعه مدلی ترکیبی از شبکه های EfficientNetو ترنسفورمر BiFormerبرای بخش بندی دقیق و سریع تصاویر کولونوسکوپی

نورائی یگانه, محمد امین; سلطانیان زاده, حمید

توسعه مدلی ترکیبی از شبکه های EfficientNetو ترنسفورمر BiFormerبرای بخش بندی دقیق و سریع تصاویر کولونوسکوپی

مقالات آماده انتشار

نوع مقاله : مقاله پژوهشی

نویسندگان

محمد امین نورائی یگانه

حمید سلطانیان زاده

دانشگاه تهران، دانشکدگان فنی، دانشکده برق و کامپیوتر

چکیده

بخش‌بندی تصاویر پزشکی نقش مهمی در تشخیص دقیق و برنامه‌ریزی درمان دارد و یکی از مسائل اساسی در حوزه بینایی ماشین محسوب می‌شود. دشواری در شناسایی مرزها و تفاوت‌های بافتی به‌ویژه در شناسایی پولیپ‌ها در تصاویر کولونوسکوپی، نیاز به روش‌هایی با دقت بالا را آشکار می‌سازد. در این پژوهش، رویکردی ترکیبی ارائه شده است که در آن بخش رمزگذار مسئول استخراج ویژگی‌های چندمقیاسی و عمیق و بخش رمزگشا متکی بر سازوکار توجه، جهت بازسازی ساختارهای دقیق تصویر به‌کار رفته است. در این راستا، EfficientNetV2 برای استخراج‌ ویژگی‌ها و BiFormer Transformer برای بازنمایی و ترسیم مرزها استفاده شده است. برای ارزیابی کارایی روش پیشنهادی، آزمایش‌ها بر روی مجموعه‌داده‌های معتبر کولونوسکوپی انجام شده است. یافته‌های کمی و کیفی نشان می‌دهد که روش پیشنهادی در شاخص‌های Dice و IoU، نسبت به مدل‌های مطرح و پیشرفته اخیر، بهبود ایجاد کرده است. همچنین با کاهش تعداد پارامترها و بار محاسباتی، نشان‌دهنده توازن مناسب میان دقت و پیچیدگی محاسباتی است. نتایج کیفی هم توانایی مدل در نمایش شفاف و دقیق مرز پولیپ‌ها در تصاویر کولونوسکوپی را نشان می دهند. بنابراین، نتایج به‌دست‌آمده نشان می‌دهد که معماری پیشنهادی از پتانسیل مناسبی برای توسعه سامانه‌های کمکی بخش‌بندی تصاویر کولونوسکوپی برخوردار است.

کلیدواژه‌ها

بخش‌بندی تصاویر پزشکی

بخش‌بندی پولیپ

تصاویر کولونوسکوپی

EfficientNetV2

BiFormer

ترنسفورمرهای بینایی

عنوان مقاله English

Development of a Hybrid EfficientNet and BiFormer Transformer Model for Accurate and Fast Segmentation of Colonoscopy Images

نویسندگان English

Mohammad Amin Nooraei Yeganeh

Hamid Sltanianzadeh

School of Electrical and Computer Engineering, University of Tehran, Tehran, Iran.

چکیده English

Medical image segmentation plays a crucial role in accurate diagnosis and treatment planning and is considered one of the fundamental challenges in the field of computer vision. The difficulty in identifying boundaries and distinguishing tissue characteristics, particularly for polyp detection in colonoscopy images, highlights the need for highly accurate segmentation methods. In this study, a hybrid approach is proposed in which the encoder is responsible for extracting deep, multiscale features, while the decoder relies on an attention mechanism to reconstruct fine image structures. Specifically, EfficientNetV2 is employed for feature extraction, while the BiFormer Transformer is used for representation learning and boundary delineation. To evaluate the performance of the proposed method, experiments were conducted on established colonoscopy image datasets. Quantitative results demonstrate that the proposed method achieves substantial improvements in Dice and IoU scores compared with recent state-of-the-art models. Moreover, by reducing the number of parameters and computational complexity, the proposed approach enables greater stability and practical applicability for real-time processing. Qualitative results further demonstrate the model’s ability to delineate polyp boundaries clearly and accurately in colonoscopy images. Therefore, the proposed approach has the potential to be incorporated into computer-aided clinical diagnostic systems.

کلیدواژه‌ها English

Convolutional Neural Networks (CNNs)

Vision Transformers

Deep Neural Networks

Deep Learning

Colonoscopy Image Segmentation

[1] W. Yao, J. Bai, W. Liao, Y. Chen, M. Liu, and Y. Xie,“From CNN to Transformer: A Review of Medical Image Segmentation Models,” Journal of Imaging Informatics in Medicine, vol. 37, no. 4, pp. 1529–1547, Aug. 2024.

[2] A. Khan, Z. Rauf, A. R. Khan, S. H. Rathore, S. H. Khan, N. Shah, U. Farooq, H. Asif, A. Zahoor, and R. U. Khalil, “A recent survey of vision transformers for medical image segmentation,” IEEE Access, Oct. 2025.

[3] M.Jabarouti Moghaddam and H.Soltanian-Zadeh, “Medical Image Segmentation Using Artificial Neural Networks,”in Artificial Neural Networks – Methodological Advances and Biomedical Applications, K. Suzuki, Ed. Rijeka, Croatia: IntechOpen, 2011, pp. 121–138.

[4] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” in Proc. Int. Conf. Medical Image Computing and Computer-Assisted Intervention (MICCAI), Cham, Switzerland: Springer, 2015, pp. 234–241.

[5] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale,” in Proc. Int. Conf. on Learning Representations (ICLR), 2021.

[6] M. Tan and Q. V. Le, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,” in Proc. Int. Conf. on Machine Learning (ICML), vol. 97, pp. 6105–6114, Long Beach, CA, USA: PMLR, 2019.

[7] B.-D. Dinh, T.-T. Nguyen, T.-T. Tran, and V.-T. Pham, “1M parameters are enough? A lightweight CNN-based model for medical image segmentation,” in Proc. Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), IEEE, 2023, pp. 1279–1284.

[8] Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, and J. Liang, “UNet++: A Nested U-Net Architecture for Medical Image Segmentation,” in Proc. International Workshop on Deep Learning in Medical Image Analysis, Cham, Switzerland: Springer, 2018, pp. 3–11.

[9] S. Cai, Y. Tian, H. Lui, H. Zeng, Y. Wu, and G. Chen, “Dense-UNet: A novel multiphoton in vivo cellular image segmentation model based on a convolutional neural network,” Quantitative Imaging in Medicine and Surgery, vol. 10, no. 6, pp. 1275–1285, Jun. 2020.

[10] L.C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 4, pp. 834–848, Apr. 2017.

[11] H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. Jégou, “Training data-efficient image transformers & distillation through attention,” in Proc. International Conference on Machine Learning (ICML), Jul. 2021, pp. 10347–10357.

[12] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin Transformer: Hierarchical vision transformer using shifted windows,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Oct. 2021, pp. 10012–10022.

[13] H. Cao, Y. Wang, J. Chen, D. Jiang, X. Zhang, Q. Tian, and M. Wang, “Swin-UNet: UNet-like pure transformer for medical image segmentation,” in Proc. Eur. Conf. Comput. Vis. (ECCV), Cham, Switzerland: Springer Nature, Oct. 2022, pp. 205–218.

[14] J. Chen, Y. Lu, Q. Yu, X. Luo, E. Adeli, Y. Wang, L. Lu, A. L. Yuille, and Y. Zhou, “TransUNet: Transformers make strong encoders for medical image segmentation,” arXiv preprint arXiv:2102.04306, Feb. 2021.

[15] L. Lan, P. Cai, L. Jiang, X. Liu, Y. Li, and Y. Zhang, “BRAU-Net++: U-shaped hybrid CNN-transformer network for medical image segmentation,” arXiv preprint arXiv:2401.00722, Jan. 2024.

[16] X. Huang, Z. Deng, D. Li, and X. Yuan, “MISSFormer: An effective medical image segmentation transformer,” arXiv preprint arXiv: 2109.07162, Sep. 2021.

[17] Y. Ji, R. Zhang, H. Wang, Z. Li, L. Wu, S. Zhang, and P. Luo, “Multi-compound transformer for accurate biomedical image segmentation,” in Proc. Int. Conf. Med. Image Comput. Comput.-Assist. Interv. (MICCAI), Cham: Springer International Publishing, Sep. 2021, pp. 326–336.

[18] X. Li, W. Zhu, X. Dong, O. M. Dumitrascu, and Y. Wang, “EViT-UNET: U-Net like efficient vision transformer for medical image segmentation on mobile and edge devices,” in Proc. IEEE 22nd Int. Symp. Biomed. Imaging (ISBI), Apr. 2025, pp. 1–5.

[19] Y. Chen, B. Zou, Z. Guo, Y. Huang, Y. Huang, F. Qin, Q. Li, and C. Wang, “Scunet++: Swin-UNet and CNN bottleneck hybrid architecture with multi-fusion dense skip connection for pulmonary embolism CT image segmentation,” in Proc. IEEE/CVF Winter Conf. Appl. Comput. Vis. (WACV), 2024, pp. 7759–7767.

[20] M. Heidari, A. Kazerouni, M. Soltany, R. Azad, E. K. Aghdam, J. Cohen-Adad, and D. Merhof, “Hiformer: Hierarchical multi-scale representations using transformers for medical image segmentation,” in Proc. IEEE/CVF Winter Conf. Appl. Comput. Vis. (WACV), 2023, pp. 6202–6212.

[21] J. M. J. Valanarasu, P. Oza, I. Hacihaliloglu, and V. M. Patel, “Medical transformer: Gated axial-attention for medical image segmentation,” in Proc. Int. Conf. Med. Image Comput. Comput.-Assist. Intervent. (MICCAI), 2021, pp. 36–46.

[22] D. Bo, W. Wang, D.-P. Fan, J. Li, H. Fu, and L. Shao, “Polyp-PVT: Polyp segmentation with pyramid vision transformers,” CAAI Artificial Intelligence Research, vol. 2, no. 3, pp. 59–0015, 2023.

[23] A. Lin, B. Chen, J. Xu, Z. Zhang, G. Lu, and D. Zhang, “DS-TransUNet: Dual Swin Transformer U-Net for medical image segmentation,” IEEE Transactions on Instrumentation and Measurement, vol. 71, pp. 1–15, May 2022.

[24] M. Tan and Q. Le, “EfficientNetV2: Smaller models and faster training,”Proc. Int. Conf. Machine Learning (ICML), 2021, pp. 10096–10106.

[25] L. Zhu, X. Wang, Z. Ke, W. Zhang, and R. W. H. Lau, “Biformer: Vision transformer with bi-level routing attention,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2023, pp. 10323–10333.

[26] D. Jha, P. H. Smedsrud, M. A. Riegler, P. Halvorsen, T. de Lange, D. Johansen, and H. D. Johansen, “Kvasir-SEG: A Segmented Polyp Dataset,” in Proc. Int. Conf. on Multimedia Modeling (MMM), Cham, Switzerland: Springer, 2019, pp. 451–462.

[27] J. Bernal, F. J. Sánchez, G. Fernández-Esparrach, D. Gil, C. Rodríguez, and F. Vilariño, “WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians,” Computerized Medical Imaging and Graphics, vol. 43, pp. 99–111, Jul. 2015.

[28] H. Tang, Y. Chen, T. Wang, Y. Zhou, L. Zhao, Q. Gao, M. Du, T. Tan, X. Zhang, and T. Tong, “HTC-Net: A hybrid CNN-transformer framework for medical image segmentation,” Biomedical Signal Processing and Control, vol. 88, 105605, Feb. 2024.

[29] A. K. Upadhyay and A. K. Bhandari, “MaS-TransUNet: A multiattention Swin Transformer U-Net for medical image segmentation,” IEEE Transactions on Radiation and Plasma Medical Sciences, vol. 9, no. 5, pp. 613–626, Oct. 2024.

[30] H. Wu, Z. Zhao, and Z. Wang, “META-Unet: Multi-scale efficient transformer attention Unet for fast and high-accuracy polyp segmentation,” IEEE Transactions on Automation Science and Engineering, vol. 21, no. 3, pp. 4117–4128, Jul. 2024.

[31] J. Liu, K. Li, C. Huang, H. Dong, Y. Song, and R. Li, “MixFormer: A mixed CNN–Transformer backbone for medical image segmentation,” IEEE Transactions on Instrumentation and Measurement, vol. 74,pp. 1–12, 2025, doi: 10.1109/TIM.2024.3497060.

[32] Q. Xu, Z. Ma, N. He, and W. Duan, “DCSAU-Net: A deeper and more compact split-attention U-Net for medical image segmentation,” Computers in Biology and Medicine, vol. 154, p. 106626, Mar. 2023, doi: 10.1016/j.compbiomed.2023.106626.

[33] J. Gu, F. Tian, and I. S. Oh, “RAMIS: Increasing robustness and accuracy in medical image segmentation with hybrid CNN-transformer synergy,” Neurocomputing, vol. 618, p. 129009, 2025.

[34] X. Pan, W. Xu, and H. Xu, “DSSINet: Dynamic token selection and spatial interaction framework for medical image segmentation,” Alexandria Engineering Journal, vol. 130, pp. 954–968, 2025.