Abstract: By the end of December 2019, the novel coronavirus 2019 (COVID-2019), became a world pandemic affecting the respiratory system. Scientists started investigating using Deep Learning and Convolutional Neural Networks (CNNs) to detect COVID-19 using Chest X-rays (CXRs). One of the main difficulties researchers reported in the detection of lung diseases is the fact that radiographic images can tell that the lungs are abnormal, but they might miss specifying the type of pneumonia exactly. Only the expert radiologist can tell the difference based on patches shapes and distribution on the affted lungs. Also CNN’s require big datasets to provide good results. When new pandemics spread, The limited benchmark datasets for COVID- 19 in CXR images, especially during the onset of the pandemic, is the main motivation of this research. In this research, we will introduce the use of Vision Transformers (ViTs). We consider an updated version of ViT called Compact Transformer (CT) which was proposed to reduce the expansive computations of the self-attention mechanism in ViT and to escape the big data paradigm. As a contribution of this study, We propose using a Hybrid Compact Transformer (HCT) in which a pretrained CNN is used in place of the convolutional layers in CT. Hence, with the hybrid model design, we aim to combine the localization power of CNNs, with the generalization power (attention mechanism or distanced-pixel relations) of ViTs. Based on experimental results using different performance metrics, the Hybrid Compact Transformer is shown to be superior over Compact Transformers and Transfer Learning models. Our proposed technique enjoys the benefits of both worlds; a faster training of the model due to TL with CNNs and reduced data requirements due to CT. Combining localized filters of CNNs and the attention mechanism of CT seems to provide a better discrimination between common pneumonia and Covid-19 pneumonia.

Ghadeer Almoeili and Abdenour Bounsiar, “Using Hybrid Compact Transformer for COVID-19 Detection from Chest X-Ray” International Journal of Advanced Computer Science and Applications(IJACSA), 15(11), 2024. http://dx.doi.org/10.14569/IJACSA.2024.01511127

@article{Almoeili2024,
title = {Using Hybrid Compact Transformer for COVID-19 Detection from Chest X-Ray},
journal = {International Journal of Advanced Computer Science and Applications},
doi = {10.14569/IJACSA.2024.01511127},
url = {http://dx.doi.org/10.14569/IJACSA.2024.01511127},
year = {2024},
publisher = {The Science and Information Organization},
volume = {15},
number = {11},
author = {Ghadeer Almoeili and Abdenour Bounsiar}
}