Comparative Analysis of Vision Transformer and CNN Architectures in Medical Image Classification

DOI

10.2991/978-94-6463-718-2_112How to use a DOI?

Keywords

Medical image classification; Vision Transformers; Convolutional Neural Networks; Deep learning; Comparative analysis

Abstract

Without the classification of images, it is next to impossible to make a diagnosis for a medical condition and also arrive at an appropriate treatment plan. In particular, ViTs have recently been suggested to replace Convolutional Neural Networks (CNNs) in image processing tasks through advances in deep learning. We discuss the similarities and differences in the architectures of ViT and CNN for several modalities i.e. X-ray, MRI, CT-scans etc. In this paper, we implement state-of-the-art models and evaluate their efficiency on benchmark datasets. Vision transformer (ViT) models have shown effectiveness in both elementary and complex image tasks including classifying MRI-based brain tumors at a rate of 92%, 7%, and 89%. 5% for CNNs. However, with the small dataset, the CNNs scored higher with 88 percent accuracy. The lung X-ray pneumonia detection accuracy is 3%, while the artificial intelligence has an accuracy of 85%. The accuracy of lung X-ray pneumonia detection is only 3%, and that of ViTs is only 1%, but the accuracy of chest X-ray pneumonia detection is far more than that. ViTs perform well in some specific medical imaging assignments, especially large datasets with high features; otherwise, in a limited-resource environment, CNNs still dominance and even outperform viTs. This necessitates task-oriented tuning of the ViT architecture to suit a system level compute budget.

Copyright

© 2025 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - R. Rajesh Sharma
AU  - Akey Sungheetha
AU  - Mohit Tiwari
AU  - Irfan Ahmad Pindoo
AU  - V. Ellappan
AU  - G. G. S. Pradeep
PY  - 2025
DA  - 2025/05/23
TI  - Comparative Analysis of Vision Transformer and CNN Architectures in Medical Image Classification
BT  - Proceedings of the International Conference on Sustainability Innovation in Computing and Engineering (ICSICE 2024)
PB  - Atlantis Press
SP  - 1343
EP  - 1355
SN  - 2352-538X
UR  - https://doi.org/10.2991/978-94-6463-718-2_112
DO  - 10.2991/978-94-6463-718-2_112
ID  - Sharma2025
ER  -