AccScience Publishing / AIH / Online First / DOI: 10.36922/aih.4155
Submit to AIH
Cite this article
21
Download
361
Views
Journal Browser
Volume | Year
Issue
Search
Forthcoming Issue
Current Issue
View All
News and Announcements
View All
ORIGINAL RESEARCH ARTICLE

Vision transformers for glioma classification using T1 magnetic resonance imaging

W. M. S. P. B. Wickramasinghe1 ,  Maheshi B. Dissanayake1*
Show Less
1 Department of Electrical and Electronics Engineering, Faculty of Engineering, University of Peradeniya, Peradeniya, Sri Lanka
AIH, 4155 https://doi.org/10.36922/aih.4155
Submitted: 5 July 2024 | Accepted: 19 September 2024 | Published: 6 November 2024
(This article belongs to the Special Issue Artificial intelligence for diagnosing brain diseases)
© 2024 by the Author(s). This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution 4.0 International License ( https://creativecommons.org/licenses/by/4.0/ )
Download PDF
Cite
XML
Abstract

Automated image analysis and classification have increasingly advanced in recent decades owing to machine learning and computer vision. In particular, deep learning (DL) architectures have become popular in resource-limited and labor-restricted environments such as the health-care sector. Transformer architecture, a DL method with self-attention mechanism, excels in natural language processing; however, its application in image-based diagnosis in health-care sector remains limited. Herein, the feasibility, bottlenecks, and performance of transformers in magnetic resonance imaging (MRI)-based brain tumor classification were investigated. To this end, a vision transformer (ViT) model was trained and tested using the popular Brain Tumor Segmentation (BraTS) 2015 dataset for glioma classification. Owing to limited data availability, domain adaptation techniques were used to pretrain the ViT model and the BraTS 2015 dataset was used for its fine-tuning. With the model only trained for 100 epochs, the confusion matrix for the two-class problem of tumor and nontumor classification showed an overall classification accuracy of 81.8%. In conclusion, although convolutional neural networks are traditionally used for DL-based medical image classification owing to their attention mechanism and long-range dependency-capturing capability, ViTs can outperform them in MRI-based brain tumor classification.

Keywords
Vision transformers
Medical image analysis
Deep neural networks
Magnetic resonance imaging
Convolutional neural network
Glioma detection
Funding
None.
Conflict of interest
The authors declare that they have no competing interests.
References
  1. Pal A, Chaturvedi A, Garain U, Chandra A, Chatterjee R. Severity Grading of Psoriatic Plaques using Deep CNN Based Multi-task Learning. Mexico: ICPR; 2016. doi: 10.1109/ICPR.2016.7899846

 

  1. Wang G. A perspective on deep imaging. IEEE Access. 2016;4:8914-8924. doi: 10.1109/ACCESS.2016.2624938

 

  1. Ker J, Wang L, Rao J, Lim T. Deep learning applications in medical image analysis. IEEE Access. 2018;6:9375-9389. doi: 10.1109/ACCESS.2017.2788044

 

  1. Kabir Anaraki A, Ayati M, Kazemi F. Magnetic resonance imaging-based brain tumor grades classification and grading via convolutional neural networks and genetic algorithms. Biocybern. Biomed. Eng. 2019;39(1):63-74. doi: 10.1016/j.bbe.2018.10.004

 

  1. Kaldera HNTK, Gunasekara SR, Dissanayake MB. Brain Tumor Classification and Segmentation Using Faster R-CNN. In: Proceedings ASET. United States: IEEE; 2019. doi: 10.1109/ICASET.2019.8714263

 

  1. Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. In: Advances in Neural Information Processing Systems. United States: The MIT Press; 2017. p. 5998-6008. doi: 10.48550/arXiv.1706.03762

 

  1. Menze BH, Jakab A, Bauer S, et al. The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans Med Imaging. 2014;34(10):1993-2024. doi: 10.1109/TMI.2014.2377694

 

  1. Alsaif H, Guesmi R, Alshammari BM, et al. A novel data augmentation-based brain tumor detection using convolutional neural network. Appl Sci. 2022;12(8):3773. doi: 10.3390/app12083773

 

  1. Pan X, Ge C, Lu R, et al. On the Integration of Self-Attention and Convolution. United States: IEEE/CVF; 2022. p. 815-825. doi: 10.1109/CVPR52688.2022.0008

 

  1. Devlin J, Chang MW, Lee K, Toutanova K. Pre-training of Deep Bidirectional Transformers for Language Understanding; 2018. doi: 10.48550/arXiv.1810.04805

 

  1. Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: Transformers for image recognition at scale; 2020. doi: 10.48550/arXiv.2010.11929

 

  1. Parmar N, Vaswani A, Uszkoreit J, et al. Image Transformer. In: JMLR Workshop and Conference Proceedings; 2018. p. 4055-4064. doi: 10.48550/arXiv.1802.05751

 

  1. Zheng S, Lu J, Zhao H, et al. Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers; 2020. doi: 10.48550/arXiv.2012.15840

 

  1. Child R, Gray S, Radford A, Sutskever I. Generating long sequences with sparse transformers; 2019. doi: 10.48550/arXiv.1904.10509

 

  1. Wu H, Xiao B, Codella N, et al. Introducing Convolutions to Vision Transformers. CVF 2021. United States: IEEE. p. 22-31. doi: 10.1109/ICCV48922.2021.00009

 

  1. Carion N, Massa F, Synnaeve G, Usunier N, Kirillov A, Zagoruyko S. End-to-end object detection with transformers. In: European Conference on Computer Vision 2020 Aug 23. Cham: Springer International Publishing; 2020. p. 213-229. doi: 10.1007/978-3-030-58452-8_13

 

  1. Aloraini M, Khan A, Aladhadh S, Habib S, Alsharekh MF, Islam M. Ombining the transformer and convolution for effective brain tumor classification using MRI Images. Appl Sci. 2023;13:3680. doi: 10.3390/app13063680

 

  1. Mehta S, Lu X, Weaver D, Elmore JG, Hajishirzi H, Shapiro L. HATNet: An end-to-end holistic attention network for diagnosis of breast biopsy images; 2007. doi: 10.48550/arXiv.2007.13007

 

  1. Lan YL, Zou S, Qin B, Zhu X. Potential roles of transformers in brain tumor diagnosis and treatment. Brain X. 2023;1:e23. doi: 10.1002/brx2.23

 

  1. Courant R, Edberg M, Dufour N, Kalogeiton V. Transformers and visual transformers. In: Colliot O, editors. Machine Learning for Brain Disorders. Neuromethods. vol. 197. United States: Humana; 2023. doi: 10.1007/978-1-0716-3195-9_6

 

  1. Zunair H, Ben Hamza A. Sharp U-Net: Depthwise convolutional network for biomedical image segmentation. Comput Biol Med. 2021;136:104699. doi: 10.1016/j.compbiomed.2021.104699

 

  1. Dasanayaka C, Dharmasena B, Bandara WR, Dissanayake MB, Jayasinghe R. Segmentation of Mental Foramen in Dental Panoramic Tomography Using Deep Learning. In: 2019 IEEE 14th Conference on Industrial and Information Systems (ICIIS). IEEE; 2019. p. 81-84. doi: 10.1109/ICIIS47346.2019.9063312

 

  1. Wang P, Yang Q, He Z, Yuan Y. Vision transformers in multi-modal brain tumor MRI segmentation: A review. Meta Radiol. 2023;1:100004. doi: 10.1016/j.metrad.2023.100004

 

  1. Marathe A, Kadam V, Chaumal A, Kodilkar S, Joshi A, Sawant S. Performance analysis of memory-efficient vision transformers in brain tumor segmentation. In: Artificial Intelligence-Based Healthcare Systems. Cham: Springer Nature Switzerland; 2023. p. 125-133. doi: 10.1007/978-3-031-41925-6_9

 

  1. Asiri AA, Shaf A, Ali T, et al. Exploring the power of deep learning: Fine-tuned vision transformer for accurate and efficient brain tumor detection in MRI Scans. Diagnostics. 2023;13(12):2094. doi: 10.3390/diagnostics13122094

 

  1. Salama K. Image Classification with Vision Transformer; 2022. Available: https://keras.io/examples/vision/image_ classification_with_vision_transformer [Last accessed on 2022 Oct 10].

 

  1. Mabu S, Atsumo A, Kido S, Kuremoto T, Hirano Y. Investigating the effects of transfer learning on ROI-based classification of chest CT images: A case study on diffuse lung diseases. J Signal Process Syst. 2020;92:307-313. doi: 10.1007/s11265-019-01499-w

 

  1. Kanesamoorthy K, Dissanayake MB. Prediction of treatment failure of tuberculosis using support vector machine with genetic algorithm. Int J Mycobacteriol. 2021;10(3):279-284. doi: 10.4103/ijmy.ijmy_130_21

 

  1. Sun L, Zhang S, Chen H, Luo L. Brain tumor segmentation and survival prediction using multimodal MRI scans with deep learning. Front Neurosci. 2019;13:810. doi: 10.3389/fnins.2019.00810

 

  1. Latif G. DeepTumor: Framework for brain MR image classification, segmentation and tumor detection. Diagnostics (Basel). 2022;12(11):2888. doi: 10.3390/diagnostics12112888

 

  1. El-Melegy MT, El-Magd KMA. A Multiple Classifiers System for Automatic Multimodal Brain Tumor Segmentation. In: Proceedings of the 2019 15th International Computer Engineering Conference (ICENCO), Giza, Egypt. 29-30 December 2019. New York, NY, USA: IEEE; 2019. doi: 10.1109/ICENCO48310.2019.9027389

 

  1. Xue Y, Yang Y, Farhat FG, et al. Brain tumor classification with tumor segmentations and a dual path residual convolutional neural network from MRI and pathology images. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Germany: Springer; 2020. p. 360-367. doi: 10.1007/978-3-030-46643-5_36

 

  1. Amin J, Sharif M, Gul N, Yasmin M, Shad SA. Brain tumor classification based on DWT fusion of MRI sequences using convolutional neural network. Pattern Recognit Lett. 2020;129:115-122. doi: 10.1016/j.patrec.2019.11.016

 

  1. Maram B, Rana P. Brain Tumour Detection on BraTS 2020 using U-Net. In: 2021 9th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), Noida, India; 2021. p. 1-5. doi: 10.1109/ICRITO51393.2021.9596530

 

  1. Ferdous GJ, Sathi KA, Hossain MA, Hoque MM, Dewan MAA. LCDEiT: A linear complexity data-efficient image transformer for MRI brain tumor classification. IEEE Access. 2023;11:20337-20350. doi: 10.1109/ACCESS.2023.3244228
Share
Back to top
Artificial Intelligence in Health, Electronic ISSN: 3029-2387 Print ISSN: 3041-0894, Published by AccScience Publishing
We use cookies on our website to ensure you get the best experience.
Read more about our cookies here
Accept