The recent coronavirus disease 2019 (COVID-19) pandemic instigated by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has become a global catastrophe with 257,469,528 registered cases and 5,158,211 deaths registered worldwide as of 23rd November 2021 [ WHO Coronavirus (COVID-19) Dashboard, Accessed on 24th November 2021]. The virus is primarily spread in 3 different ways- inhaling air that has been contaminated with fine droplets and or aerosol particles consisting of the virus, deposition of exhaled droplets that contain the virus onto exposed mucous membranes which may be the mouth, nose or the eyes, and touching mucous membranes after coming in contact with contaminated or soiled surfaces and or objects [ CDC, Accessed on 24th November 2021]. Therefore, it is essential to maintain and/or improve hygienic habits such as washing one’s hands for at least 20 seconds, and sanitizing hands with sanitizers that contain a minimum alcohol content of 60%. ¹ Avoiding extended stays in enclosed spaces with improper ventilation is also crucial to protect oneself [ CDC, Accessed on 24th November 2021]. An individual contracting the disease may depict a wide range of symptoms: 83%-98% of the patients have fever, 59%-78% of the patients exhibit dry cough and 3%-29% of the affected patients may require admission into an intensive care unit (ICU) for the management of a myriad of complications, which include extreme pneumonia, hypoxemic respiratory failure, acute respiratory distress syndrome (ARDS), multiple organ failure etc. ² On the other hand, it is observed that a small percentage of the affected patients presented no symptoms. ² Affected individuals that exhibit symptoms are often diagnosed to have COVID-19 using the reverse transcription polymerase chain reaction test, commonly referred to as the RT-PCR test. Samples are collected from the nasal and or pharyngeal passages or the throat. Sputum is also often used as a sample. Genetic material is isolated from the sample and multiplied. Fluorescent indicators are added to the multiplied genetic material that glow in the presence of the SARS-CoV-2’s genetic material [ Cleveland Clinic, Accessed on 24th November 2021]. Despite being the gold standard of testing, it provides an average accuracy rate of 63% ^{3 – 5} as the detection of the virus’s genetic material depends on when the patient takes the test and the amount of the virus present in the sample collected. A recovered patient may continue to get a positive RT-PCR test result due to small amounts of genetic material being present, even though they have recovered and cannot spread the virus. On the other hand, a recently infected patient may receive a negative test result due to smaller amounts of virus present in the sample [ Cleveland Clinic, Accessed on 24th November 2021].

To overcome these drawbacks, doctors and scientists alike are researching and coming up with various other methods to diagnose COVID-19 that provide higher accuracy rates. One such method is using radiological images and deep learning techniques. ^{6 , 7} Often, the imaging modality used for this purpose is computerized tomography (CT) ^{8 – 11} and/or X-ray. ^{12 – 15} It is found that COVID-19 patients depict similar features on radiographic images such as bilateral, multi-focal ground glass opacities with peripheral or posterior distribution. This is predominantly found in the lower lobes of the lungs in the early stages. In the later stages, these features can be found in areas of pulmonary consolidation. ¹⁶ Unfortunately, these symptoms overlap with the symptoms of bronchopneumonia, hence, making it difficult for the radiologists to tell COVID-19 apart from other diseases that may affect the lungs. ¹⁷ Deep learning techniques are used, in hopes of revealing underlying features, that often go unnoticed which may aid in confirming diagnosis.

There have been several studies that have employed deep learning methods to diagnose COVID-19 from radiological images- a total of 8 studies with 4 using CT and 4 using X-ray as their imaging modalities. They have been presented in a consolidated form in Table 1 and Table 2.

Past works have used either X-ray image or CT images to train and test their models but not both. There are no studies conducted, to the best of our knowledge, that compare the performances of the models when they are trained with CT and X-ray images. This paper presents a comparative study using 3 non-pre-trained models and 5 pre-trained models to find which imaging modality provides better accuracies in detecting COVID-19 using deep learning techniques.

This section elucidates the dataset used, the kind of network employed and the working of this study that took place from January 2021 to July 2021. The general workflow used in the study is shown in Figure 1. Non-pre-trained models were developed for the study and were rejected due to their low validation accuracies. To compare the results and efficacy and to deduce the best of the two imaging modalities, 5 pre-trained models namely VGG16, VGG19, ResNet50, InceptionV3, and Exception were loaded and fine-tuned according to the needs of this study. The pre-trained models that were selected were chosen primarily based on their accuracy (where higher accuracy yields better results), the speed of the model training and predictions (where faster training is preferred). Ideally, we would use a model that is smaller in size with a higher accuracy and lower time requirement but realistically any model that has deeper layers would make it slower to execute and a smaller model would have much lower accuracy. Hence, we have experimented with the available models to find the best suited option to detect COVID-19.

For the purposes of training our networks, the dataset named ‘ Extensive COVID-19 X-Ray and CT Chest Images Dataset’, curated and augmented by Walid El-Shafai and Fathi Abd El-Samie was used. ¹⁸ The dataset consists of COVID-19 and Non- COVID-19 folders for both X-ray and CT images. There were 5500 non-COVID-19 and 4044 COVID-19 cases belonging to the X-ray folder. For CT images, there were 2628 non-COVID-19 and 5427 COVID-19 images, which brings the total number of images to 17,599. Sample images from the Extensive COVID-19 X-Ray and CT Chest Images Dataset is shown in Figure 2. ¹⁸

A convolutional neural network (CNN) is one of the many networks that fall under neural networks. It primarily works on the basis that the input will be an image or is made up of images. Therefore, the model is extensively employed in the field of pattern recognition in images, making it an effective method for this study.

In this study, Keras API (version2.4.3), an open-source software library providing an interface for neural networks, is used to build and train the models. We trained and tested a total of 8 models- 3 models that were built from scratch and 5 pre-trained models. Each model was trained with the CT dataset and the X-ray dataset separately and the obtained results were compared. Description of each of the models are mentioned below. The code is available from GitHub and archived with Zenodo. ²¹

Model 1: Consists of 4 alternate convolution and max pooling layers, 32 filters, 4 × 4 kernels and Relu activation function present in each convolution layer. The model consisted of 5 epochs and has 23,618 total parameters, all of which are trainable.

Model 2: This model consists of a total of 6 convolution layer and 3 max pooling layers along with a dropout layer with varying number of filters (from 32 till 256), 3 × 3 kernels in both layers. This model has 5 epochs and 333,154 total parameters and all of them are trainable.

Model 3: This model was an attempt to fine tune Model 2 by varying the number of filters (from 32 till 128) and adding a dropout layer with a dropout rate of 0.3. This model has 462,338 total parameters and all of them are trainable. For experimentation, this model has 10 epochs.

VGG 16: VGG-16 is one of the many pre-trained CNN models that is considered as an excellent vision model and is one of the extensively used image-recognition architectures [ Deep Convolution Networks VGG16 for Image Recognition in Keras, Medium, Accessed on 24th November 2021]. This model consists of 3 epochs and has a total of 14,766,866 parameters out of which 50,178 are trainable. Figure 3 shows the model architecture that was used for this study.

VGG 19: The difference between VGG-16 and VGG-19 is the number of layers -16 and 19 layers, respectively. The architecture of this network is shown in Figure 3. Compared to VGG16, VGG19 performs slightly better but it requires more memory due to more layers [ Understanding the VGG19 Architecture, Opengenus Foundation, Accessed on 24th November 2021]. ¹⁹ The model has 20,074562 parameters out of which 50,178 are trainable and 3 epochs. Figure 4 shows the model architecture that was used for this study.

ResNet 50: ResNet 50 grants users the provision to train tremendously deep neural networks of about 100 and more layers successfully without facing the problem of vanishing gradients [ Understanding and Coding a ResNet in Keras, Towards Data Science, Accessed on 24th November 2021]. The general architecture of the model is presented below in Figure 5. The model has 23,788,418 total parameters, including 200,706 trainable parameters and used 3 epochs

Inception V3: It is a commonly utilized image recognition model that has been proven to have high accuracies when tested on a large dataset, consisting of asymmetric and symmetric blocks of convolution and max pooling layers, dropouts, and fully connected layers [ Advanced Guide to Inception V3 on Cloud TPU, Google Cloud, Accessed on 24th November 2021]. The model consisted of 21,905,18 parameters out of which 102,402 are trainable with 3 epochs. Figure 6 depicts the model’s architecture.

Xception: This model is derived from the Inception V3 model, with the inception modules replaced by depth wise separable convolutions. It slightly outperforms the Inception V3 due to its efficient use of model parameters. ²⁰ Figure 7 shows the architecture of the model. The model has 21,062,186 total parameters of which 200,706 are trained and 3 epochs.

Once the models were trained, parameters such as number of epochs, training accuracy and loss, validation accuracy and loss, true positive and negative, false positive and negative, prediction accuracy, sensitivity, and training times were recorded.

The open-source dataset, Extensive COVID-19 X-Ray and CT Chest Images Dataset, was downloaded. The dataset contained 5500 non-COVID-19 and 4044 COVID-19 cases belonging to the X-ray folder and there were 2628 non-COVID-19 and 5427 COVID-19 images belonging to the CT folder, which brings the total number of images to 17,599. The images were divided into training, testing and validation sets. For each imaging modality, 20 COVID-19 and 20 non-COVID-19 images were set aside for the testing set. About 1860 images of both COVID-19 and non-COVID-19 were initially kept aside for validation for each of the imaging modalities. Upon experimenting with the number of images in the validation sets, it was found that validation sets consisting of 2500 images of both the cases for each modality resulted in the better validation accuracies. The remaining images were used for training the various models.

Models 1 was trained with the CT training set first. The model was trained with the training set five times, hence 5 epochs. Once trained, the validation set was fed into the model to obtain a validation accuracy of 24.89%. Model 1 was then tweaked in terms of the number of convolution layers, number of max pooling layers, adding dropout layers and changing kernel sizes to obtain Model 2. Model 2 was then trained with the training set with the same number of epochs and then validated. A validation accuracy of 20.74% was obtained. To improve the validation accuracy, certain parameters, such as varying the number of filters and adding a dropout layer with a dropout rate of 0.3, were added and changed in Model 2 to give rise to Model 3. Model 3 was trained with the CT training set with 10 epochs. Increasing the number of epochs increased training time but somehow decreased the validation accuracy to 19.94%.

The above process was repeated with the X-ray training and validation sets. Models 1, 2 and 3 showed validation accuracies of 31.32%, 29.87% and 29.87% respectively.

Due to poor validation accuracies, pre-trained models were used in hoped of obtaining higher validation and testing accuracies. The top layers of the models were dropped from the pre-trained models to make it more suited for the study's objective. Preprocessing layers were also added to the front of certain models.

Once the models are trained, validated, and tested, parameters such as the number of epochs, training accuracy and loss, validation accuracy and loss, true positive and negative, false positive and negative, prediction accuracy, sensitivity, and training times were all recorded and tabulated.

The results obtained during our study have been tabulated and categorized in terms of imaging modalities. Table 3 presents the tabulated performance of the CNN models when CT images are used. Similarly, Table 4 presents the performance of the CNN models when X-ray images are used. The number of epochs, various losses and accuracies are mentioned in both Table 1 and Table 2. The training times are also mentioned.

The 3 non-pretrained, CNN models were first employed to gain more exposure to how the network functions and how we can alter it to increase accuracy rates. From Tables 3 and 4, it can be observed that both Models 1 and 2 have 5 epochs while Model 3 has 10. Among Models 1 and 2, it can be noted that Model 2 has a higher training accuracy [79.52% for CT images and 84.63% for X-ray images] that is due to the presence of more convolutional layers and increased filter size. Model 3 took the most time to train, which is the result of increased number of epochs. However, higher number of epochs which resulted in longer training periods provided higher training accuracies [85.13% for CT images and 85.94% for X-ray images]. Figure 8 is a comparison of the 3 non-pretrained models based on training accuracy. While the CNN models provided high training accuracies, their validation accuracies were not satisfactory. For deep learning techniques to be employed in a clinical setting to detect COVID-19, the validation accuracies must be high. Hence, pre-trained networks were chosen and trained with our dataset. Table 5 and 6 show the performances of the 5 pre-trained models when they are trained and tested using CT images and X-ray images, respectively.

For the pre-trained models, the comparison has mostly been based on validation accuracies. Validation accuracies signify the generalizability of the model to detect COVID-19, in this scenario, from the dataset that was held back from the model while training. Another important parameter used is sensitivity. This gives us the ratio of actual positives that been predicted correctly by the model. The epochs of the training were decided to be set at 3 after experimentation - a higher number of iterations did not result in higher validation accuracies and smaller numbers of iterations result in lower accuracy rates. Figure 9 compares the obtained validation accuracies of the 5 pre-trained models for both the imaging modalities.

The obtained results are compared to the highest obtained results from previous works for each of the imaging modality. These comparisons are done based on the imaging modality and parameters such as validation accuracy, sensitivity and prediction accuracy are taken into consideration (wherever they are available). Figure 10 presents the comparison of our results with previous works’ when X-ray images are used, and Figure 11 presents the comparison when CT images are used.

The following tables summarize and compare the accuracies obtained by previous works and our study for each image modality. They also mention the key points of the methodology in each of the referred works. Table 7 compares and summarizes previous works based on X-ray together with our results that were obtained when the models were trained with X-ray images. Similarly, Table 8 shows the previous works and our work when trained with CT images.

It is seen that our methods have comparatively lower results in terms of accuracy. This can be due to several factors- lower number of epochs, using models with lesser layers etc. While the initial objective of the study was to employ deep learning techniques in COVID-19 detection that work with adequate accuracies, the main aim was to find which imaging modality provided higher accuracies when used to detect COVID-19. An observable trend is that the performances of the model changes with the change in the imaging modality. Hence, comparing CT and X-ray imaging becomes vital as it will help develop an in-depth understanding of how a particular pre-trained network interprets the details provided by the radiological modalities to detect COVID-19.

When trained with CT and X-ray images, VGG19 and Xception have the best validation accuracies, but their associated prediction accuracies are the lowest. Validation accuracies depict the generalizability of the network while prediction accuracies are calculated using the obtained true and false positive and negative values. Oftentimes, a larger difference between these values indicates overfitting. In terms of prediction accuracies, Xception and VGG16 show promising results when trained with CT and X-ray images, respectively. Their accuracies stand at 95% for Xception and 80% for VGG16.

The obtained accuracies can be further improved by increasing the number of epochs during training, however, increasing epochs would also mean increasing the training time. Another possible improvement would be to add on to the already present dataset for better training.

This study was conducted to compare the effectiveness of radiological imaging modalities in the detection of COVID-19, with the help of deep learning techniques. A total of 8 unique models were used- 3 non-pretrained, CNN models and 5 pre-trained models. From the obtained results, it can be noted that Xception provided the highest validation accuracy when X-ray images are used and VGG19 for when CT images are used. When the models are compared in terms of prediction accuracy, VGG16 and Xception provided the highest values when trained with X-ray and CT images respectively. The model, VGG16, showed the most consistent performance for both X-ray and CT images. In terms of validation accuracy, models trained with X-ray images have shown better results than the same models trained with CT images for both non-pretrained and pre-trained models even though training with X-ray images have shown to take much longer time.

•

Source code available from: https://github.com/tejashri-220799/X-Ray-vs-CT-for-detection-of-covid-19