مجله ماشین بینایی و پردازش تصویر

مجله ماشین بینایی و پردازش تصویر

مروری بر شبکه‌های عصبی حذف نویز تصویر

نوع مقاله : مقاله مروری

نویسندگان
1 دانشجوی دکتری مهندسی کامپیوتر-نرم افزار، دانشکده فنی و مهندسی، گروه مهندسی کامپیوتر، دانشگاه اراک، اراک، ایران.
2 دانشکده فنی و مهندسی، گروه مهندسی کامپیوتر، دانشگاه اراک
چکیده
حذف نویز تصویر یک موضوع فعال در پردازش تصویر سطح پایین است و تاکنون تحقیقات بسیاری در این زمینه انجام شده است. روش‌های حذف نویز تصویر را می‌توان به دو دسته کلی روش‌های مبتنی بر مدل و روش‌های مبتنی بر یادگیری افتراقی تقسیم‌بندی کرد. شبکه‌های عصبی کانولوشن (CNN) عمیق یکی از روش‌های مبتنی بر یادگیری هستند که در سال‌های اخیر نتایج خوبی را در وظایف مختلف بازیابی تصویر (از جمله حذف نویز تصویر) ارائه کردند. در این پژوهش ابتدا روش‌های حذف نویز مبتنی بر مدل به صورت اجمالی معرفی و مزایا و معایب آن‌ها بررسی شده است؛ سپس شبکه‌های عصبی حذف نویز، از نظر معماری شبکه و بخش‌های تشکیل دهنده، مورد بررسی قرار گرفته و ساختارهای مورد استفاده در این شبکه‌ها، که موجب بهبود عملکرد حذف نویز شده است، معرفی شده‌اند. در بخش پایانی این مقاله ساختار کلی مشترک شبکه‌های عصبی حذف نویز مدرن، به همراه عملکرد بخش‌های مختلف این ساختار مورد بررسی قرار گرفته است. این مقاله می‌تواند مرجع مناسبی برای مطالعه ایده‌های مورد استفاده در معماری شبکه‌های عصبی کانولوشن حذف نویز باشد.
کلیدواژه‌ها

[1] Bajaj, K., D.K. Singh, and M.A. Ansari, "Autoencoders Based Deep Learner for Image Denoising". Procedia Computer Science, 2020. 171: p. 1535-1541.
[2] Joshua, O., S. Ibiyemi, and B. Adu, "A Comprehensive Review On Various Types of Noise in Image Processing". International Journal of Scientific and Engineering Research, 2019. 10: p. 388-393.
[3] Boncelet, C., Chapter 7 - Image Noise Models, in The Essential Guide to Image Processing, A. Bovik, Editor. 2009, Academic Press: Boston. p. 143-167.
[4] Goyal, B., et al., "Image denoising review: From classical to state-of-the-art approaches". Information Fusion, 2020. 55: p. 220-244.
[5] Zhang, K., et al., "Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising". IEEE Transactions on Image Processing, 2017. 26(7): p. 3142-3155.
[6] Chatterjee, P. and P. Milanfar, "Is Denoising Dead?". IEEE Transactions on Image Processing, 2010. 19(4): p. 895-911.
[7] Mohd Sagheer, S.V. and S.N. George, "A review on medical image denoising algorithms". Biomedical Signal Processing and Control, 2020. 61: p. 102036.
[8] Tian, J. and L. Chen, "Image despeckling using a non-parametric statistical model of wavelet coefficients". Biomedical Signal Processing and Control, 2011. 6(4): p. 432-437.
[9] Zhang, J., et al., "Wavelet and fast bilateral filter based de-speckling method for medical ultrasound images". Biomedical Signal Processing and Control, 2015. 18: p. 1-10.
[10] Aja-Fernandez, S., C. Alberola-Lopez, and C.F. Westin, "Noise and Signal Estimation in Magnitude MRI and Rician Distributed Images: A LMMSE Approach". IEEE Transactions on Image Processing, 2008. 17(8): p. 1383-1398.
[11] Alnuaimy, A.N.H., et al. "BM3D Denoising Algorithms for Medical Image". in 2024 35th Conference of Open Innovations Association (FRUCT). 2024.
[12] L, M. and M.A. Dorairangasamy, "An Extensive Review of Significant Researches on Medical Image Denoising Techniques". International Journal of Computer Applications, 2013. 64(14): p. 1-12.
[13] Ding, Q., et al., "Statistical Image Reconstruction Using Mixed Poisson-Gaussian Noise Model for X-Ray CT". 2018.
[14] P, P. and V.V. R. "A Comprehensive Analysis of Satellite Image Denoising using Earth Observation Data". in 2023 7th International Conference on Computing Methodologies and Communication (ICCMC). 2023.
[15] Ayanala, B., et al., "Satellite Image Denoising Using Machine Learning". 2024.
[16] Wei, S., et al., "Research on SAR image denoising method based on feature extraction". Electronics Letters, 2024. 60(8): p. e13180.
[17] "RETRACTED ARTICLE: Image denoising and despeckling methods for SAR images to improve image enhancement performance: a survey". Multimedia Tools Appl., 2021. 80(17): p. 26547–26569.
[18] Liu, S., et al., "LG-DBNet: Local and Global Dual-Branch Network for SAR Image Denoising". IEEE Transactions on Geoscience and Remote Sensing, 2024. 62: p. 1-15.
[19] Khasanova, A., A. Makhmutova, and I. Anikin. "Image Denoising for Video Surveillance Cameras Based on Deep Learning Techniques". in 2021 International Conference on Industrial Engineering, Applications and Manufacturing (ICIEAM). 2021.
[20] Zhang, K., W. Zuo, and L. Zhang, "FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising". IEEE Transactions on Image Processing, 2017. PP.
[21] Fan, L., et al., "Brief review of image denoising techniques". Visual Computing for Industry, Biomedicine, and Art, 2019. 2(1): p. 7.
[22] Bled, C. and F. Pitié, Lightweight Video Denoising Using a Classic Bayesian Backbone. 2024.
[23] Maggioni, M., et al. "Efficient Multi-Stage Video Denoising with Recurrent Spatio-Temporal Fusion". in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021.
[24] Yue, H., et al. "Supervised Raw Video Denoising With a Benchmark Dataset on Dynamic Scenes". in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020.
[25] Tassano, M., J. Delon, and T. Veit. "FastDVDnet: Towards Real-Time Deep Video Denoising Without Flow Estimation". in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020.
[26] Vaksman, G., M. Elad, and P. Milanfar. "Patch Craft: Video Denoising by Deep Modeling and Patch Matching". in 2021 IEEE/CVF International Conference on Computer Vision (ICCV). 2021.
[27] Liang, J., et al., "VRT: A Video Restoration Transformer". IEEE Transactions on Image Processing, 2024. 33: p. 2171-2182.
[28] Fu, Z., et al., Temporal As a Plugin: Unsupervised Video Denoising with Pre-trained Image Denoisers, in Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29–October 4, 2024, Proceedings, Part LVI. 2024, Springer-Verlag: Milan, Italy. p. 349–367.
[29] Gonzalez, R.C. and R.E. Woods, Digital Image Processing (3rd Edition). 2006: Prentice-Hall, Inc.
[30] Al-Ameen, Z., S. Al-Ameen, and G. Sulong, "Latest Methods of Image Enhancement and Restoration for Computed Tomography: A Concise Review". Applied Medical Informatics, 2015. 36: p. 1-12.
[31] Benesty, J., J. Chen, and Y. Huang. "Study of the widely linear Wiener filter for noise reduction". in 2010 IEEE International Conference on Acoustics, Speech and Signal Processing. 2010.
[32] Pitas, I. and A.N. Venetsanopoulos. "Nonlinear Digital Filters - Principles and Applications". in The Springer International Series in Engineering and Computer Science. 1990.
[33] Ruikang, Y., et al., "Optimal weighted median filtering under structural constraints". IEEE Transactions on Signal Processing, 1995. 43(3): p. 591-604.
[34] Tomasi, C. and R. Manduchi. "Bilateral filtering for gray and color images". in Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271). 1998.
[35] Xie, Q., et al., "Color and direction-invariant nonlocal self-similarity prior and its application to color image denoising". Science China Information Sciences, 2020. 63(12): p. 222101.
[36] Buades, A., B. Coll, and J.-M. Morel, "Nonlocal Image and Movie Denoising". International Journal of Computer Vision, 2008. 76(2): p. 123-139.
[37] Buades, A., B. Coll, and J.M. Morel. "A non-local algorithm for image denoising". in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05). 2005.
[38] Mairal, J., et al. "Non-local sparse models for image restoration". in 2009 IEEE 12th International Conference on Computer Vision. 2009.
[39] Dabov, K., et al., "Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering". IEEE Transactions on Image Processing, 2007. 16(8): p. 2080-2095.
[40] Mairal, J., M. Elad, and G. Sapiro, "Sparse Representation for Color Image Restoration". IEEE Transactions on Image Processing, 2008. 17(1): p. 53-69.
[41] Elad, M. and M. Aharon, "Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries". IEEE Transactions on Image Processing, 2006. 15(12): p. 3736-3745.
[42] Gu, S., et al. "Weighted Nuclear Norm Minimization with Application to Image Denoising". in 2014 IEEE Conference on Computer Vision and Pattern Recognition. 2014.
[43] Dong, W., et al., "Nonlocally Centralized Sparse Representation for Image Restoration". IEEE Transactions on Image Processing, 2013. 22(4): p. 1620-1630.
[44] Chen, Y. and T. Pock, "Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration". IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017. 39(6): p. 1256-1272.
[45] Perona, P. and J. Malik, "Scale-space and edge detection using anisotropic diffusion". IEEE Transactions on Pattern Analysis and Machine Intelligence, 1990. 12(7): p. 629-639.
[46] Weickert, J., "Anisotropic Diffusion in Image Processing". 1998.
[47] Schmidt, U. and S. Roth. "Shrinkage Fields for Effective Image Restoration". in 2014 IEEE Conference on Computer Vision and Pattern Recognition. 2014.
[48] Roth, S. and M.J. Black. "Fields of Experts: a framework for learning image priors". in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05). 2005.
[49] Krizhevsky, A., I. Sutskever, and G.E. Hinton, "ImageNet classification with deep convolutional neural networks". Commun. ACM, 2017. 60(6): p. 84–90.
[50] Ioffe, S. and C. Szegedy, Batch normalization: accelerating deep network training by reducing internal covariate shift, in Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37. 2015, JMLR.org: Lille, France. p. 448–456.
[51] He, K., et al. "Deep Residual Learning for Image Recognition". in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016.
[52] Burger, H.C., C.J. Schuler, and S. Harmeling. "Image denoising: Can plain neural networks compete with BM3D?". in 2012 IEEE Conference on Computer Vision and Pattern Recognition. 2012.
[53] Jain, V. and H.S. Seung, Natural image denoising with convolutional networks, in Proceedings of the 21st International Conference on Neural Information Processing Systems. 2008, Curran Associates Inc.: Vancouver, British Columbia, Canada. p. 769–776.
[54] Yu, F. and V. Koltun, Multi-Scale Context Aggregation by Dilated Convolutions. 2016.
[55] Tian, C., et al., "Attention-guided CNN for image denoising". Neural Networks, 2020. 124: p. 117-129.
[56] Zhang, K., et al. "Learning Deep CNN Denoiser Prior for Image Restoration". in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017.
[57] Larrue, T., X. Meng, and C. Han. "Denoising Videos with Convolutional Autoencoders A Comparison of Autoencoder Architectures". 2018.
[58] Xie, J., L. Xu, and E. Chen, "Image Denoising and Inpainting with Deep Neural Networks". Advances in Neural Information Processing Systems, 2012. 1.
[59] Wu, W., et al., Image Blind Denoising Using Dual Convolutional Neural Network with Skip Connection. 2023.
[60] Mao, X.-J., C. Shen, and Y.-B. Yang, Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections, in Proceedings of the 30th International Conference on Neural Information Processing Systems. 2016, Curran Associates Inc.: Barcelona, Spain. p. 2810–2818.
[61] Guo, S., et al. "Toward Convolutional Blind Denoising of Real Photographs". in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019.
[62] Ronneberger, O., P. Fischer, and T. Brox. "U-Net: Convolutional Networks for Biomedical Image Segmentation". in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. 2015. Cham: Springer International Publishing.
[63] Liu, P., et al., "Multi-level Wavelet-CNN for Image Restoration". 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2018: p. 886-88609.
[64] Zhang, K., et al., "Practical Blind Image Denoising via Swin-Conv-UNet and Data Synthesis". Machine Intelligence Research, 2023. 20(6): p. 822-836.
[65] Fan, C.-M., T.-J. Liu, and K.-H. Liu, "SUNet: Swin Transformer UNet for Image Denoising". 2022 IEEE International Symposium on Circuits and Systems (ISCAS), 2022: p. 2333-2337.
[66] Cao, H., et al. "Swin-Unet: Unet-Like Pure Transformer for Medical Image Segmentation". in Computer Vision – ECCV 2022 Workshops. 2023. Cham: Springer Nature Switzerland.
[67] Lefkimmiatis, S. "Non-local Color Image Denoising with Convolutional Neural Networks". in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017.
[68] Plötz, T. and S. Roth, Neural nearest neighbors networks, in Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018, Curran Associates Inc.: Montréal, Canada. p. 1095–1106.
[69] Liu, D., et al., Non-local recurrent network for image restoration, in Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018, Curran Associates Inc.: Montréal, Canada. p. 1680–1689.
[70] Qiao, P., et al., Learning Non-local Image Diffusion for Image Denoising, in Proceedings of the 25th ACM international conference on Multimedia. 2017, Association for Computing Machinery: Mountain View, California, USA. p. 1847–1855.
[71] Valsesia, D., G. Fracastoro, and E. Magli, "Deep Graph-Convolutional Image Denoising". IEEE Transactions on Image Processing, 2020. 29: p. 8226-8237.
[72] Li, Y., X. Fu, and Z.J. Zha. "Cross-Patch Graph Convolutional Network for Image Denoising". in 2021 IEEE/CVF International Conference on Computer Vision (ICCV). 2021.
[73] Valsesia, D., G. Fracastoro, and E. Magli. "Image Denoising with Graph-Convolutional Neural Networks". in 2019 IEEE International Conference on Image Processing (ICIP). 2019.
[74] Simonovsky, M. and N. Komodakis, Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs. 2017. 29-38.
[75] He, J., et al. "Conditional Sequential Modulation for Efficient Global Image Retouching". in Computer Vision – ECCV 2020. 2020. Cham: Springer International Publishing.
[76] Chen, X., et al., HDRUNet: Single Image HDR Reconstruction with Denoising and Dequantization. 2021.
[77] Tian, C., Y. Xu, and W. Zuo, "Image denoising using deep CNN with batch renormalization". Neural Networks, 2020. 121: p. 461-473.
[78] Pan, J., et al. "Learning Dual Convolutional Neural Networks for Low-Level Vision". in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018.
[79] Helou, M.E. and S. Süsstrunk, "Blind Universal Bayesian Image Denoising With Gaussian Noise Level Learning". IEEE Transactions on Image Processing, 2020. 29: p. 4885-4897.
[80] Kim, Y., J.W. Soh, and N.I. Cho, "Adaptively Tuning a Convolutional Neural Network by Gate Process for Image Denoising". IEEE Access, 2019. 7: p. 63447-63456.
[81] He, J., C. Dong, and Y. Qiao. "Interactive Multi-dimension Modulation with Dynamic Controllable Residual Learning for Image Restoration". in Computer Vision – ECCV 2020. 2020. Cham: Springer International Publishing.
[82] Li, B., et al. "All-In-One Image Restoration for Unknown Corruption". in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2022.
[83] Xu, Y., et al., "Unified-Width Adaptive Dynamic Network for All-In-One Image Restoration". ArXiv, 2024. abs/2401.13221.
[84] Soh, J.W. and N.I. Cho. "Deep Universal Blind Image Denoising". in 2020 25th International Conference on Pattern Recognition (ICPR). 2021.
[85] Wang, W., et al. "CFSNet: Toward a Controllable Feature Space for Image Restoration". in 2019 IEEE/CVF International Conference on Computer Vision (ICCV). 2019.
[86] Holla, K.S. and B. Lee. "Convolutional Residual Blocks With Edge Guidance for Image Denoising". in 2022 13th International Conference on Information and Communication Technology Convergence (ICTC). 2022.
[87] Tian, C., et al., "Designing and training of a dual CNN for image denoising". Knowledge-Based Systems, 2021. 226: p. 106949.
[88] Yu, K., et al., "Path-Restore: Learning Network Path Selection for Image Restoration". IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022. 44(10): p. 7078-7092.
[89] Yao, C., et al., "Dense Residual Transformer for Image Denoising". Electronics, 2022. 11: p. 418.
[90] Vaswani, A., et al., Attention is all you need, in Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, Curran Associates Inc.: Long Beach, California, USA. p. 6000–6010.
[91] Li, Y., et al. "Attention-Guided Unified Network for Panoptic Segmentation". in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019.
[92] Wu, W., et al., "Dual residual attention network for image denoising". Pattern Recognition, 2024. 149: p. 110291.
[93] Woo, S., et al. "CBAM: Convolutional Block Attention Module". in Computer Vision – ECCV 2018. 2018. Cham: Springer International Publishing.
[94] Hu, J., L. Shen, and G. Sun. "Squeeze-and-Excitation Networks". in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018.
[95] Wu, W., et al., "Dual convolutional neural network with attention for image blind denoising". Multimedia Systems, 2024. 30(5): p. 263.
[96] Zhang, Y., et al., Image Super-Resolution Using Very Deep Residual Channel Attention Networks: 15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, Part VII. 2018. p. 294-310.
[97] Anwar, S. and N. Barnes. "Real Image Denoising With Feature Attention". in 2019 IEEE/CVF International Conference on Computer Vision (ICCV). 2019.
[98] Islam, S., et al., "A comprehensive survey on applications of transformers for deep learning tasks". Expert Systems with Applications, 2024. 241: p. 122666.
[99] Dosovitskiy, A., et al., "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale". ArXiv, 2020. abs/2010.11929.
[100] Touvron, H., et al., Training data-efficient image transformers & distillation through attention, in Proceedings of the 38th International Conference on Machine Learning, M. Marina and Z. Tong, Editors. 2021, PMLR: Proceedings of Machine Learning Research. p. 10347--10357.
[101] Carion, N., et al. "End-to-End Object Detection with Transformers". in Computer Vision – ECCV 2020. 2020. Cham: Springer International Publishing.
[102] Radford, A., et al. "Learning Transferable Visual Models From Natural Language Supervision". in International Conference on Machine Learning. 2021.
[103] Song, M., W. Wang, and Y. Zhao A Dynamic Network with Transformer for Image Denoising. Electronics, 2024. 13,  DOI: 10.3390/electronics13091676.
[104] Chen, Y., et al. "Dynamic Convolution: Attention Over Convolution Kernels". in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020.
[105] Liu, Z., et al. "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows". in 2021 IEEE/CVF International Conference on Computer Vision (ICCV). 2021.
[106] Liang, J., et al. "SwinIR: Image Restoration Using Swin Transformer". in 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). 2021.
[107] Dai, T., et al., Mix-order Attention Networks for Image Restoration, in Proceedings of the 29th ACM International Conference on Multimedia. 2021, Association for Computing Machinery: Virtual Event, China. p. 2880–2888.
[108] Zhang, Y., et al., Residual Non-local Attention Networks for Image Restoration. 2019.
[109] Wang, F., et al. "Residual Attention Network for Image Classification". in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017.
[110] Wang, P., et al. "Understanding Convolution for Semantic Segmentation". in 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). 2018.
[111] Du, Z., et al., "Multi-scale Attention Dilated Residual Image Denoising Network based on Skip Connection". Instrumentation, 2024. 11(3): p. 41-53.
[112] Zhang, Y., et al., KBNet: Kernel Basis Network for Image Restoration. 2023.
[113] Hassija, V., et al., "Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence". Cognitive Computation, 2024. 16(1): p. 45-74.
[114] Hou, G., Y. Yang, and J.H. Xue. "Residual Dilated Network with Attention for Image Blind Denoising". in 2019 IEEE International Conference on Multimedia and Expo (ICME). 2019.
[115] Tian, C., et al., "Enhanced CNN for image denoising". CAAI Transactions on Intelligence Technology, 2019. 4(1): p. 17-23.
[116] Martin, D., et al. "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics". in Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001. 2001.
[117] Deng, J., et al. "ImageNet: A large-scale hierarchical image database". in 2009 IEEE Conference on Computer Vision and Pattern Recognition. 2009.
[118] Ma, K., et al., "Waterloo Exploration Database: New Challenges for Image Quality Assessment Models". IEEE Transactions on Image Processing, 2017. 26(2): p. 1004-1016.
[119] Franzen, R. Kodak lossless true color image suite. 1999; Available from: https://r0k.us/graphics/kodak/.