Brain tumors are a primary factor causing cancer-related deaths globally, and their classification remains a significant research challenge due to the variability in tumor intensity, size, and shape, as well as the similar appearances of different tumor types. Accurate differentiation is further complicated by these factors, making diagnosis difficult even with advanced imaging techniques such as magnetic resonance imaging (MRI). Recent techniques in artificial intelligence (AI), in particular deep learning (DL), have improved the speed and accuracy of medical image analysis, but they still face challenges like overfitting and the need for large annotated datasets. This study addresses these challenges by presenting two approaches for brain tumor classification using MRI images. The first approach involves fine-tuning transfer learning cutting-edge models, including SEResNet, ConvNeXtBase, and ResNet101V2, with global average pooling 2D and dropout layers to minimize overfitting and reduce the need for extensive preprocessing. The second approach leverages the Vision Transformer (ViT), optimized with the AdamW optimizer and extensive data augmentation. Experiments on the BT-Large-4C dataset demonstrate that SEResNet achieves the highest accuracy of 97.96%, surpassing ViT’s 95.4%. These results suggest that fine-tuning and transfer learning models are more effective at addressing the challenges of overfitting and dataset limitations, ultimately outperforming the Vision Transformer and existing state-of-the-art techniques in brain tumor classification.

1. Pichaivel M, Anbumani G, Theivendren P, et al. An overview of brain tumor. In: Brain Tumors. 2022; pp. 1–10. doi: 10.5772/intechopen.100806.

2. Ostrom QT, Patil N, Cioffi G, et al. CBTRUS statistical report: primary brain and other central nervous system tumors diagnosed in the United States in 2013–2017. Neuro-oncology. 2020;22. doi: 10.1093/neuonc/noaa200.

3. Park J, Park YG. Brain tumor rehabilitation: symptoms, complications, and treatment strategy. Brain & Neurorehabilitation. 2022;15(3). doi: 10.12786/bn.2022.15. e25.

4. Nassar SE, Yasser I, Amer HM, et al. A robust MRI-based brain tumor classification via a hybrid deep learning technique. The Journal of Supercomputing. 2024;80(2):2403-2427. https://doi.org/10.1007/s11227-023-05549-w.

5. Abd-Ellah MK, Awad AI, Khalaf AA, et al. A review on brain tumor diagnosis from MRI images: Practical implications, key achievements, and lessons learned. Magnetic Resonance Imaging. 2019; 61:300-318. https://doi.org/10.1016/j.mri.2019.05.028.

6. Asiri AA, Shaf A, Ali T, et al. Exploring the power of deep learning: fine-tuned vision transformer for accurate and efficient brain tumor detection in MRI scans. Diagnostics. 2023;13(12):2094. https://doi.org/10.3390/diagnostics13122094.

7. Rajput IS, Gupta A, Jain V, et al. A transfer learning-based brain tumor classification using magnetic resonance images. Multimedia Tools and Applications. 2024;83(7):20487-20506.

8. Tulbure A-A, Tulbure A-A, Dulf E-H. A review on modern defect detection models using DCNNs—Deep convolutional neural networks. Journal of Advanced Research. 2022; 35:33-48. https://doi.org/10.1016/j.jare.2021.03.015

9. Morid MA, Borjali A, Del Fiol G, et al. A scoping review of transfer learning research on medical image analysis using ImageNet. Computers in Biology and Medicine. 2021; 128:104115.

10. Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929. 2020.

11. Malla PP, Sahu S, Alutaibi AI. Classification of tumor in brain MR images using deep convolutional neural network and global average pooling. Processes. 2023;11(3):679. https://doi.org/10.3390/pr11030679

12. Hossain S, Chakrabarty A, Gadekallu TR, et al. Vision transformers, ensemble model, and transfer learning leveraging explainable AI for brain tumor detection and classification. IEEE Journal of Biomedical and Health Informatics. 2024;28(3):1261–1272. DOI: 10.1109/JBHI.2023.3266614

13. Ghassemi N, Shoeibi A, Rouhani M. Deep neural network with generative adversarial networks pre-training for brain tumor classification based on MR images. Biomedical Signal Processing and Control. 2020; 57:101678. http://doi.org/10.1016/j.bspc.2019.101678.

14. Shaik NS, Cherukuri TK. Multi-level attention network: application to brain tumor classification. Signal, Image and Video Processing. 2022;16(3):817–824. https://doi.org/10.1007/s11760-021-02022-0.

15. Ahmad B, Sun J, You Q, et al. Brain tumor classification using a combination of variational autoencoders and generative adversarial networks. Biomedicines. 2022;10(2):223. https://doi.org/10.3390/biomedicines10020223

16. Munira HA, Islam MS. Hybrid deep learning models for multi-classification of tumour from brain MRI. J Inf Syst Eng Bus Intell. 2022; 8:162–174. Available at: http://e-journal.unair.ac.id/index.php/JISEBI

17. Vankdothu R, Hameed MA, Fatima H. A brain tumor identification and classification using deep learning based on CNN-LSTM method. Computers and Electrical Engineering. 2022; 101:107960. Available at: https://doi.org/10.1016/j.compeleceng.2022.107960

18. Yurdakul M, TAŞDEMİR Ş. Brain Tumor Detection with Ensemble of Convolutional Neural Networks and Vision Transformer. In: Proceedings of the 2023 2nd International Engineering Conference on Electrical, Energy, and Artificial Intelligence (EICEEAI); 2023. IEEE; p. 1–6. DOI: 10.1109/EICEEAI60672.2023.10590129.

19. Divya S, Suresh LP, John A. A deep transfer learning framework for multi-class brain tumor classification using MRI. In: Proceedings of the 2020 2nd International Conference on Advances in Computing, Communication Control and Networking (ICACCCN); 2020. IEEE; p. 283–290. DOI: 10.1109/ICACCCN51052.2020.9362908

20. Pashaei A, Sajedi H, Jazayeri N. Brain tumor classification via convolutional neural network and extreme learning machines. In: 2018 8th International Conference on Computer and Knowledge Engineering (ICCKE); 2018. p. 314–319. IEEE.

21. Salih MS, Abdulazeez AM. A fusion-based deep approach for enhanced brain tumor classification. Journal of Soft Computing and Data Mining. 2024;5(1):183–193. DOI: 10.30880/jscdm.2024.05.01.015

22. Sarada B, Narasimha Reddy K, Babu R, Ramesh Babu BSSV, et al. Brain tumor classification using modified ResNet50V2 deep learning model. International Journal of Computing and Digital Systems. 2024;16(1):1–10.

23. Suryawanshi S, Patil SB. Efficient brain tumor classification with a hybrid CNN-SVM approach in MRI. Journal of Advances in Information Technology. 2024;15(3):340–354. https://doi.org/10.12720/jait.15.3.340-354.

24. Jun W, Liyuan Z. Brain tumor classification based on attention guided deep learning model. International Journal of Computational Intelligence Systems. 2022;15(1):35. https://doi.org/10.1007/s44196-022-00090-9.

25. Kang J, Ullah Z, Gwak J. MRI-based brain tumor classification using an ensemble of deep features and machine learning classifiers. Sensors. 2021;21(6):2222. https://doi.org/10.3390/s21062222.

26. Mahmud MI, Mamun M, Abdelgawad A. A deep analysis of brain tumor detection from MR images using deep learning networks. Algorithms. 2023;16(4):176. https://doi.org/10.3390/a16040176.

27. Bhuvaji S, Kadam A, Bhumkar P, Dedge S, Kanchan S. Brain tumor classification (MRI). Kaggle; 2020. Available at: https://www.kaggle.com/sartajbhuvaji/brain-tumor-classification-mri (Accessed: 2024-07-02).

28. Karim PJ, Mahmood SR, Sah M. Brain tumor classification using fine-tuning based deep transfer learning and support vector machine. International Journal of Computing and Digital Systems. 2023;13(1):83–96. http://dx.doi.org/10.12785/ijcds/130108

29. Chen T, Qin H, Li X, Wan W, Yan W. A non-intrusive load monitoring method based on feature fusion and SE-ResNet. Electronics. 2023;12(8):1909. https://doi.org/10.3390/electronics12081909.

30. He J, Jiang D. A fully automatic model based on SE-ResNet for bone age assessment. IEEE Access. 2021; 9:62460-62466.

31. Vasant Bidwe R, Mishra S, Kamini Bajaj S, Kotecha K. Attention-focused eye gaze analysis to predict autistic traits using transfer learning. International Journal of Computational Intelligence Systems. 2024;17(1):1–33. https://doi.org/10.1007/s44196-024-00491-y

32. Fagbola T, Igwebuike S. Leveraging pretrained models for multimodal medical image interpretation: An exhaustive experimental analysis. In: medRxiv; August 2024. https://doi.org/10.1101/2024.08.09.24311762

33. Dhibar S. ResNet101 and DAE for Enhance Quality and Classification Accuracy in Skin Cancer Imaging. arXiv preprint arXiv:2403.14248. 2024.

34. Rahimzadeh M, Attar A. A modified deep convolutional neural network for detecting COVID-19 and pneumonia from chest X-ray images based on the concatenation of Xception and ResNet50V2. Informatics in Medicine Unlocked. 2020; 19:100360. https://doi.org/10.1016/j.imu.2020.100360

35. Wu H, Xiao B, Codella N, et al. CVT: Introducing convolutions to vision transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision; 2021. p. 22–31.

36. Chen X, Liang C, Huang D, et al. Evolved optimizer for vision. In: First Conference on Automated Machine Learning (Late-Breaking Workshop); 2022.