Shibu George Et Al. - 2023
Shibu George Et Al. - 2023
Available at www.sciencedirect.com
ScienceDirect
Gerosh Shibu George a, Pratyush Raj Mishra a, Panav Sinha a, Manas Ranjan Prusty b,*
a
School of Computer Science and Engineering, Vellore Institute of Technology, Chennai, Tamil Nadu, India
b
Centre for Cyber Physical Systems, School of Computer Science and Engineering, Vellore Institute of Technology, Chennai,
Tamil Nadu, India
A R T I C L E I N F O A B S T R A C T
Article history: COVID-19 had caused the whole world to come to a standstill. The current detection meth-
Received 29 July 2022 ods are time consuming as well as costly. Using Chest X-rays (CXRs) is a solution to this
Received in revised form problem, however, manual examination of CXRs is a cumbersome and difficult process
1 November 2022 needing specialization in the domain. Most of existing methods used for this application
Accepted 18 November 2022 involve the usage of pretrained models such as VGG19, ResNet, DenseNet, Xception, and
Available online 24 November 2022 EfficeintNet which were trained on RGB image datasets. X-rays are fundamentally single
channel images, hence using RGB trained model is not appropriate since it increases the
operations by involving three channels instead of one. A way of using pretrained model
Keywords:
for grayscale images is by replicating the one channel image data to three channel which
COVID-19
introduces redundancy and another way is by altering the input layer of pretrained model
Homomorphic Transformation Filter
to take in one channel image data, which comprises the weights in the forward layers that
Deep CNN
were trained on three channel images which weakens the use of pre-trained weights in a
VGG
transfer learning approach. A novel approach for identification of COVID-19 using CXRs,
ANOVA
Contrast Limited Adaptive Histogram Equalization (CLAHE) along with Homomorphic
Transformation Filter which is used to process the pixel data in images and extract features
from the CXRs is suggested in this paper. These processed images are then provided as
input to a VGG inspired deep Convolutional Neural Network (CNN) model which takes
one channel image data as input (grayscale images) to categorize CXRs into three class
labels, namely, No-Findings, COVID-19, and Pneumonia. Evaluation of the suggested model
is done with the help of two publicly available datasets; one to obtain COVID-19 and No-
Finding images and the other to obtain Pneumonia CXRs. The dataset comprises 6750
images in total; 2250 images for each class. Results obtained show that the model has
achieved 96.56% for multi-class classification and 98.06% accuracy for binary classification
* Corresponding author at: Centre for Cyber Physical Systems, School of Computer Science and Engineering, Vellore Institute of
Technology, Chennai, Tamil Nadu 600127, India.
E-mail address: [email protected] (M. Ranjan Prusty).
https://doi.org/10.1016/j.bbe.2022.11.003
0168-8227/Ó 2022 Nalecz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy of Sciences. Published by Elsevier
B.V. All rights reserved.
2 biocybernetics and biomedical engineering 43 (2023) 1–16
using 5-fold stratified cross validation (CV) method. This result is competitive and up to the
mark when compared with the performance shown by existing approaches for COVID-19
classification.
Ó 2022 Nalecz Institute of Biocybernetics and Biomedical Engineering of the Polish Academy
of Sciences. Published by Elsevier B.V. All rights reserved.
model employs no pre-trained models’ weights and provides included a recall of 99.42 %, an accuracy of 86.66 %, and an
promising output with very little training time compared to AUC score of 0.625. The CNN-LSTM model provided the best
other existing models. The proposed model takes CXRs as results with an accuracy of 92.3 %, a recall of 93.68 %, and
input, making it cost effective since CT scans are unreason- an AUC score of 0.90 using Holdout Validation.
able and may not be accessible at an individual level. Then A total of 4 architectures were studied by Ibrahim et al. For
a random selection of these images is provided to the model detecting and diagnosing disorders affecting the human lungs
which works on a single channel. The testing is done in two [16]. Detection was among three classes, namely, Pneumonia,
ways; first, which comprises binary classification where only Lung Cancer, and COVID-19. Out of all the models used for the
two modes, COVID-19 and No-Findings, are taken into consid- study, the CNN + VGG19 model performed best yielding an
eration. The second method involves the classification of accuracy of 98.05 %. An accuracy of 96.09 % was obtained
CXRs across three classes, namely, No-Findings, COVID-19, using GRU + ResNet152V2.
and Pneumonia. The image processing techniques used, viz. A study to detect abnormalities from chest CT scan images
CLAHE and Homomorphic Transformation filter help in of Pneumonia and COVID-19 patients [17] performed by Ni, Q.,
improvising its contrast and plummeting its dynamic range. et al. aimed at comparing various deep learning models for
These image processing techniques incorporated into the the task. Results show that our results are superior compared
proposed model produce a more robust diagnosis for to people who have the expertise in the identification and
COVID-19. The novel aspect of this work is the application detection of lesions. According to Xu’s study, the results
of Homomorphic Transformation Filter as an image process- achieved by their model had a specificity of 67 %, a sensitivity
ing approach on CXRs as well as the designing a VGG inspired of 74 %, and a total accuracy of 73 %. This study revolved
Deep CNN model from scratch that requires less training time around observing inception-migration-learning models and
than models using the same range of datasets. The significant their performance for the task of differentiating COVID-19
contributions made by this paper include the following: from other infections caused by pathogens such as bacteria,
protozoa, viruses, etc.
1. CLAHE + Homomorphic Transformation Filter as image In another study, Ibrahim, et al. analyzed deep neural net-
processing technique on CXRs. work models and their performance using transfer learning
2. A novel VGG inspired deep CNN model consisting of 22 lay- for classification between three classes, viz., COVID-19 Pneu-
ers and the inputs fed to the model have the shape of monia, Non-COVID-19 Viral Pneumonia, and Bacterial Pneu-
64 64 1. monia [18]. For multi-class classification, the model obtained
3. Different hyper parameter tuning methodologies are used 98.19 % sensitivity, 95.78 % specificity, and 94.43 % accuracy.
to examine the potential of the proposed model for the For binary classification among Healthy and Bacterial Pneumo-
task of multi as well as binary-class classification of CXRs. nia classes, the model obtained 91.49 % sensitivity, 100 % speci-
4. Assessment of model’s robustness with the help of 6750 ficity, and 91.43 % accuracy. For binary classification between
images. COVID-19 Pneumonia and Non-COVID-19 Viral Pneumonia, a
testing accuracy of 99.62 % was achieved. Similarly, for classifi-
cation between COVID-19 Pneumonia and Healthy CXRs, a test-
3. Related works ing accuracy of 99.16 % was achieved. Classification across
Bacterial pneumonia, COVID-19 and Healthy yielded a testing
Currently, Deep Learning is adopted in the field of medical accuracy of 94.00 % and 93.42 % for classification among four
imaging. This includes analysis of medical images, radiomics, classes; Healthy, Bacterial Pneumonia, COVID-19, and non-
etc. Deep Learning is mainly used because of its prominent COVID-19 Viral Pneumonia.
and reliable results [12]. Since there is no restriction to the A new transfer learning pipeline consisting of DenseNet-
kind of data that can be used, deep learning is appropriate 121 and the ResNet-50 networks, called DenResCov-19 [19]
to cater to diverse information and data, in order to make pre- was created by Mamalakis, M. et al. This was primarily cre-
dictions [13]. Due to the prolonged pandemic caused by ated for the classification and detection of Pneumonia,
COVID-19, it has become a necessity to come up with a tech- COVID-19, Tuberculosis, or Normal using CXRs. The results
nology that uses deep learning concepts and techniques in achieved by the model for classification of Pneumonia,
order to detect COVID-19 faster and with a higher degree of COVID-19, and Normal included an AUC score of 96.51 %, F1
accuracy since current testing methods are expensive and score of 87.29 %, precision of 85.28 %, and an overall recall
time consuming. CXRs and CT scans can be helpful in achiev- of 89.38 %.
ing this. Since CXRs are more affordable as compared to CT Gouda et al. considered Deep Learning strategies to predict
scans, they are a better alternative. COVID-19. The study proposed two DL approaches based on
Aslan, M. F. et al. worked on binary classification tech- ResNet-50 neural network using chest X-ray (CXR) images.
niques and observed the DenseNet-SVM structure [14] to be COVID-19 Image Data Collection (IDC) and CXR Images (Pneu-
the best one with an accuracy of about 96.29 %. They achieved monia) were used as dataset for the following. The pre-
an average accuracy of 95.21 % by using a total of eight differ- processing was done using augmentation, normalization,
ent SVM based CNN models. Alakus, T. B. et al. validated their enhancement and resizing of the images. To carry out the task,
LSTM deep learning model using a 10-fold cross-validation multiple runs of modified version of Resnet-50 was made done
strategy [15]. An accuracy greater than 84 % was shown by to classify the images. The ResNet-50 feature extraction is done
all the models involved in the study. The obtained results by several convolutional and pooling layers. A fully connected
and soft-max layer does the classification. The weight and bias
4 biocybernetics and biomedical engineering 43 (2023) 1–16
values of convolutional and fully commenced layers are tuned We use an equal number of images for each class to avoid
using the training algorithm. This training algorithm includes the problem of class imbalance. Two chest X-ray image data-
many hyperparameters, which helps to improve the perfor- sets are used to achieve our proposed work. The first public
mance of the ResNet-50 model [20]. In terms of performance, dataset is used to extract COVID-19 and No-Findings images
the values exceed 99.63 % in many metrics including, F1- and the other public database is used to obtain Pneumonia
score, accuracy, recall, precision and AUC [21]. Mahesh Gour images. The former database was created in conjunction with
et al. designed a new stacked CNN model for COVID-19 detec- medical doctors by researchers from the University of Dhaka,
tion. The dataset includes CT images and combination of three Bangladesh, Qatar University in Doha, Qatar, and colleagues
publicly available X-ray images. They firstly used different sub - from Malaysia and Pakistan. It makes use of images from 43
models obtained from VGG-19 and Xception models during the different publications as well as the COVID-19 Database of
training. Then these were together stacked as softmax classi- the Italian Society of Medical and Interventional Radiology
fier. To detect COVID-19 from radiological image data, a stacked (SIRM), the Novel Corona Virus 2019 Dataset created by Joseph
CNN model is proposed, combining the differences between Paul Cohen, Lan Dao’ and Paul Morrison’s repository in
CNN sub-models. The sensitivity for Binary classification and GitHub [27,28]. Kang Zhang, Daniel Kermany, and Michael
Multiclass classification was 98.31 % and 97.62 % respectively Goldbaum’s ‘‘Labeled Optical Coherence Tomography (OCT)
[22]. and Chest X-ray Images for Classification dataset” used
Mahesh Gour et al. developed an automated COVID19 CXR images (anterior-posterior) chosen from retrospective
detection model and was named Uncertainty-Aware Convolu- cohorts of paediatric patients from Guangzhou Women and
tional Neural Network Model (UA-CovNet). The model works Children’s Medical Center, Guangzhou, ranging from the age
on the principles of EfficientNet-B3 to fine tune the X-ray one to five [29]. This dataset was used to obtain CXR images
images and Monte Carlo dropouts for M passes to obtain the for the class of Pneumonia. To maintain consistency through-
posterior predictive distribution. The sensitivity of the Binary out the data used by us, we have resized all of the images to
Classification and Multiclass Classification was 99.30 % and 64 64 pixels for further processing. An image of each class
98.15 % respectively. The G-mean of 99.16 % and 98.02 % was obtained from these datasets is shown in Fig. 1.
seen for both respectively. [23] Yiting Xie et al. believed work-
ing on large medical image dataset is really difficult so they 5. Proposed approach
carried out their work using ImageNet, a pre-trained model.
The pre-trained model can bring in inefficiencies while work- The proposed approach comprises applying pre-processing
ing on a single channel image. To counter this, they intro- augmenting methods [30] to our CXR images, including resiz-
duced Inception V3 model on ImageNet after the images ing our images to a standard size and applying CLAHE. For a
were transformed into grayscales. The performance was not given input image, the algorithm of CLAHE creates non-
found waning, hence concluding that colors do not have crit- overlapping contextual regions (also called sub-images, tiles
ical role to play. It was also seen that that grayscale ImageNet or blocks) and then applies the histogram equalization to
pre-trained models had better performance than the color each contextual region, clips the original histogram to a
one while classifying diseases from CXRs. [24]. specific value and then redistributes the clipped pixels to
In the 1960 s, a technique for image and signal processing each gray level [31]. Then Homomorphic Transformation Fil-
was devised by Thomas Stockham, Ronald W. Schafer, and ter is applied to these processed images. These images are
Alan V. Oppenheim. This technique involved a non-linear randomly provided to the deep CNN model as input. Deep
mapping to a different domain where linear filters are applied convolutional neural networks have proven to yield better
and then mapped back to the original domain [25]. The tech- accuracy when dealing with large volumes of dataset, and
nique, called Homomorphic Transformation Filter can be many researchers tend to use them as de-facto standards
employed to enhance the images. It also increases contrast [32]. A typical architecture of CNN consists of multiple blocks
and homogenizes the brightness throughout the image. It with three kinds of layers: convolution, pooling, and fully con-
can also be used to remove noise from the image. If we take nected layers [33].The architecture of our deep CNN model is
logarithm of the image intensity, we can separate the compo- inspired by VGG model’s architecture. Two schemes are
nents of the image linearly in the frequency domain, which employed in order to test the model’s performance. The first
are combined multiplicatively. Multiplicative noise includes scheme comprises binary classification consisting of two
variations in illumination within the images and can be classes, COVID-19 and No-Findings. The second scheme
reduced by applying filtering techniques in the logarithm involves the classification of CXR images across three classes,
domain. We can also equalize the low-frequency and high- namely, COVID-19, No Findings, and Pneumonia. 2250 images
frequency components of the image to make the illumination have been considered for each class which means scheme 1
more even. This implies that in order to repress low frequen- involves a total of 4500 images and scheme 2 consists of a
cies and intensify high frequencies, high-pass filtering is used total of 6750 images. The block diagram for the suggested
in the log-intensity domain [26]. model is shown in Fig. 2.
The dataset that we used comprises 2250 images for each of Original images taken from both datasets had varied sizes. All
the three classes—COVID-19, Pneumonia, and No-Findings. of the images were converted into a standard size of 64 64
biocybernetics and biomedical engineering 43 (2023) 1–16 5
Fig. 1 – CXR images from each class, i.e, No Findings, COVID-19, and Pneumonia.
[34] and CLAHE was applied to them. By resizing the images, to each neighbouring pixel, which produces a transformation
we can decrease the training time of our model and reduces function. To resize the image the resampling using pixel area
the memory required for the training purpose. Good thing relation known as INTER_AREA function in OpenCV was used.
about having a small size image data is that lot of images Most of recent research studies have used transformation
can be fed into the model for training without exhausting as a key technique for pre-processing of the image data. The
the memory or increasing the training time. It is a good idea behind this is to use artifacts derived from a different
trade-off between the amount of pixel data in one image domain to ease the training process of the model. Typically,
and count of images that can be used for training in a limited the domain is related to either frequency or time. Many differ-
computational environment. CLAHE helps reduce the noise ent transformation techniques have been used to create the
issue by applying a contrast amplification limiting technique state-of-the-art pre-processing method. Homomorphic
Transformation Filter is one such filter belonging to frequency here, HP = high-pass filter, FI = filtered image in frequency
domain that uses Fourier Transformation. This transforma- domain.
tion filter has not been explored for image pre-processing in Then, by using the inverse Fourier transform, frequency
combination with custom CNN for one channel image data domain is returned to spatial domain.
(grayscale image) with respect to detection of COVID19. In this nðx; yÞ ¼ invFðFIðu; vÞÞ ð6Þ
research, the combination of CLAHE and Homomorphic
Transformation is studied to understand its efficacy as pre- Lastly, to obtain the improved image, we apply the expo-
processing techniques. The pre-processing of the image data nential function [35] to remove the log we used earlier.
happens in two stages after the preliminary processing like newImageðx; yÞ ¼ expðnðx; yÞÞ ð7Þ
resizing. First, CLAHE is applied and then Homomorphic
Fig. 3 shows an image from each class after Homomorphic
Transformation.
Transformation Filter is applied to the dataset.
a set of learned kernels. The layer typically consists of a com- Depending upon the classification type, the number of
bination of convolution operation and activation function nodes used in the final output layer is decided. Each of the
[39]. 2D Global Average Pooling is utilized to flatten the output neurons represents a different class. A softmax function is
of previous layers. The mean value of all values over the then used to evaluate this output using the following formula
whole (input width) (input height) matrix for each of the [42],
input channels using a tensor of size (input width) (input ewtv
height) (input channels) is calculated by using the 2D Global JðtÞ ¼ PT ð8Þ
wtv
t¼1 e
Average Pooling block.
This is followed by a dense layer and dropout layer. This is Here wt represents the weight vector of the final layer’s tth
repeated once more successively. A dense layer with softmax neuron which represents the output, and v is the fully con-
as the activation function serves as the final output layer. nected layers’ feature vector before it.
Except the output layer, all convolutional and dense layers The architecture of the proposed GrayVIC model is shown
use ReLU as the activation function. ReLU, which stands for in Fig. 5. The suggested model is trained in four scenarios
rectified linear activation function, is a non-linear or piece- where we use 50 epochs and 100 epochs with and without
wise linear function that, if the input is positive, outputs ReduceLRonPlateau each. When a statistic stops improving,
the input directly; if not, outputs zero. Following the convolu- ReduceLRonPlateau lowers the learning rate. Once learning
tion layers is the 2D Max pooling layer. The maximum value reaches a plateau, models frequently gain by decreasing the
for each input channel over an input window of the size spec- learning rate by a factor of 2–10. This scheduler reads a met-
ified by pool size is used to down sample the input along its rics quantity, and the learning rate is decreased if no progress
spatial dimensions (height and width). Steps are taken along is made after a specified number of ‘‘patience” epochs. We
each dimension to move the window. Max pooling layer esti- have used 0.000001 as our minimum learning rate. This serves
mates the max value of pixel according to filter dimension as the threshold for our learning rate, implying that our learn-
mentioned in the layer definition. Pooling layer carries out ing rate would not reduce further after the minimum learning
dimensionality reduction by down sampling the values of rate is encountered.
neurons into a solitary value. Max pooling operation is per-
formed here to combine the output of previous layer into a 5.4. Performance analysis
single value [40].
Before applying Global Average Pooling, we use a batch The metrics used in this study are recall, precision, F1-score
normalization layer. The model is trained more quickly and and accuracy to compare and analyze the performance of
steadily using batch normalization. This provides some regu- our GrayVIC model with other existing models [43]. All of
larization and helps reduce generalization errors. The drop- these metrics are obtained from the confusion matrix. We
out layer is applied to the suggested model after the batch also use AUC score which is obtained from the ROC curve to
normalization layer and dense layers. The regularization is analyse our model. The likelihood that a random positive
done by the dropout layers which speedup the execution by example will be placed in front of a random negative example
expelling the neurons whose contribution to the yield is not is represented by AUC score. In order to determine the robust-
so high [41]. It also helps us to avoid overfitting the model ness of the model, we employ 5-fold stratified cross validation
by ignoring output of some neurons for the upcoming layer (CV), which biases the large variations on the test data and
[42]. The sum of all inputs is maintained by scaling up non- averages it on each fold [44] and with CV we also used holdout
1
zero inputs by ð1rateÞ. validation [42] where the training and testing data is split
8 biocybernetics and biomedical engineering 43 (2023) 1–16
from the total data in a ratio of 4:1, maximizing the data to multi-class, and two validation schemes – holdout and 5-
shape the model [45]. This implies the training data consists fold cross validation. 4 different combination pairs of the
of 1800 images for each class and testing data consists of number of epochs and ReduceLRonPlateau were been tested
450 images per class. For the validation set, 10 % of the train- as part of hyper parameter tuning, apart from tweaking the
ing data was used to monitor the performance of the model entire model architecture which has been built from scratch.
while training. Stratified 5-fold cross validation will ensure The public dataset used in this research contains an equal
the same class ratio throughout the 5 folds as the ratio in count of images for each class which removes the class imbal-
the original dataset that is equal counts of each class labels, ance problem. The metrics considered for performance eval-
this removes the tension of class imbalance problem during uation for the proposed model is Accuracy (ACC), Recall or
the training. sensitivity (REC), Precision (PRE), F1 score and lastly AUC
We perform multi-class as well as binary classification value.
techniques on our dataset. For binary classification, we clas-
sify images between COVID19 and No Findings. For multi- 6.1. Model performance
class classification, three classes i.e. No Findings, COVID-19,
and Pneumonia are used. The training and validation accuracy and loss versus epoch
plots of the proposed deep CNN model are shown in Fig. 6.
6. Results and discussions In all the scenarios in which the model was trained, the nat-
ure of training accuracy and loss curve were the same. It can
The results of the proposed model are showcased in the fol- be noticed that the model is able to achieve 90 % accuracy
lowing section, with the CXR images transformed using within the first 20–25 epochs of the training phase. The Redu-
Homomorphic Transformation Filter. The model was com- ceLRonPlateau technique helped in training which can been
piled based on two classification schemes – binary and seen evidently in the graph, it is observed that the validation
accuracy/loss fluctuations reduced in the later stages of the teau technique achieves the highest value across all metrics.
training due to lowering of the learning rate by the algorithm The highest cross validation accuracy reached is 0.98 and
which helped in its convergence. The training loss went down recall is 0.97 along with the AUC value of 0.97. The standard
to 0.07 in the best fold of cross validation. The training was deviation between the fold’s testing accuracies was 0.014 with
done in a GPU environment. Total trainable parameter for an average accuracy of 0.96. The average training time taken
the proposed model is 2,684,650. by the proposed model to learn from 3600 images for 100
Table 2 portrays the performance of our proposed model epochs is 5 min and 6 s.
when used on multi-class classification on the basis of differ- Fig. 7 displays the proposed model’s confusion matrix
ent combinations of number of epochs and usage of ReduceL- plots trained with 4 different combinations of hyper parame-
RonPlateau technique during the training phase. It is ters. The results belong to the best fold of the 5-fold cross val-
observed that increasing the count of epochs from 50 to 100 idation of each combination. It has been observed that when
is beneficial and the highest accuracy of 0.97 is achieved in the model is trained using ReduceLRonPlateau technique the
the holdout validation. To highlight the robustness of our number of false predictions reduces for the best cases. The
model and to check that the model is not overfitted, 5-fold ReduceLRonPlateau also reduces the standard deviation to
stratified cross validation results are used. It shows that the an average value of 0.0049. This helps us ensure that our pre-
best result is given when the model is trained for 100 epochs dictions are close to the average value and these predicted
with ReduceLRonPlateau. The highest accuracy achieved in 5- values are spread in a very narrow range. 900 and 1350 images
fold stratified cross validation is 0.97 and recall is 0.95 along have been used for testing for binary and multi-class classifi-
with an AUC value of 0.96. The standard deviation between cation schemes respectively which ensure that enough
the fold’s testing accuracies was 0.0121 with an average accu- instances of each class were used to check the robustness
racy of 0.95. The average time taken by the model to train on of the model towards each class label. In all the scenarios, it
the dataset consisting of 5400 images for 100 epochs is 8 min can be observed that the accuracy is above 0.96.
20 s. Fig. 8 represents ROC curves from two scenarios. The first
Table 3 depicts the results obtained from binary classifica- ROC curve depicts the values for holdout validation using 100
tion using our proposed model based on the same methods epochs without ReduceLRonPlateau. The second ROC curve
used for multi-class. As it can be observed, the binary classi- plots the curves for cross-validation using 100 epochs with
fication’s overall performance exceeds that of multi-class ReduceLRonPlateau. A plot that shows how well a classifica-
classification task. However, this difference is not very large, tion model performs at every level of categorization is called
unlike in existing research literature. This tells us the pro- the Receiver Operating Characteristic curve (ROC curve). As
posed model is effective enough for both kinds of classifica- visible through the curves, we can see our model is capable
tion tasks. The highest accuracy achieved for this task by of differentiating COVID-19 and the rest of the classes. The
the proposed model in holdout validation is 0.98. The 5-fold values obtained for COVID vs Rest for the first and second
stratified cross validation results shows that the model when curve are 0.96 and 0.97 respectively. Since these values are
trained for 100 epochs and employed with ReduceLRonPla- very close to 1, we can confirm that our model is reliable
50 No 0.95 0.94 0.93 0.93 0.95 0.86 0.90 0.85 0.84 0.89
50 Yes 0.96 0.94 0.94 0.94 0.95 0.94 0.93 0.92 0.92 0.94
100 No 0.97 0.95 0.95 0.95 0.96 0.95 0.94 0.93 0.93 0.94
100 Yes 0.94 0.93 0.92 0.92 0.94 0.97 0.95 0.95 0.95 0.96
50 No 0.95 0.93 0.92 0.92 0.93 0.89 0.90 0.87 0.86 0.87
50 Yes 0.96 0.94 0.94 0.94 0.94 0.96 0.96 0.96 0.96 0.96
100 No 0.98 0.94 0.94 0.94 0.94 0.97 0.96 0.96 0.96 0.96
100 Yes 0.97 0.95 0.95 0.95 0.95 0.98 0.97 0.97 0.97 0.97
10 biocybernetics and biomedical engineering 43 (2023) 1–16
Fig. 7 – The best fold of 5-fold Cross Validation’s confusion matrices for all the 4 cases of model training.
biocybernetics and biomedical engineering 43 (2023) 1–16 11
and robust. The highest AUC score achieved by our model is In the first two studies the models described are the
0.98 for multi-class classification scheme. COVID-Caps [49] which is capable of handling small datasets
The proposed model’s computational complexity is esti- and COVIDX-Net [50] which uses seven different architec-
mated as follows. When it comes to Deep Learning models, tures. They achieved accuracies of 95.70 % and 90 % respec-
computational complexity plays a critical role. Computa- tively for the binary classification scheme. When worked on
tional Complexity increases exponentially with the number multi-class classification, with a model ResNet50 combined
if network level grown [46]. Computational complexity is with SVM [51] attained an accuracy of 95.33 %. This paper
often determined with the help of trainable parameters advocated that support vector machine, abbreviated as SVM,
[47] from the model’s architecture. As compared to common is quite reliable when compared to other transfer learning
transfer learning models, our model projected a better com- models. In [52], the authors compare multiple models for
putational result. An approximate total of 2.7 M trainable multi-class classification and among the suggested models,
parameters were required for the proposed model. This is COVID-Net performs best with an accuracy of 93.34 % which
significantly less when compared to 62 M for AlexNet, 25 M were trained using hold-out validation. The authors of [53]
for DenseNet, 23.6 M for InceptionV3, 8 M for CapsNet and use transfer learning to extract important information from
4 M for GoogleNet, since only one channel image data is X-ray images and studied its performance for multi-class
considered. classification. They were able to obtain accuracies of 94.72 %
The model described in the study (as shown in Fig. 5) con- and 85 % respectively. A good accuracy of 96.87 % was noticed
sists of Conv2D-Conv2D-MaxPool pattern layers which are in [54] using 2D curvelet transform-EfficientNet-B0. This
stacked four times. Batch Normalization layer was used to model implemented a blend comprising of chaotic swarm
ease the training of the model and Global Average Pooling algorithm and two dimensional curvelet transformation.
layer was used to condense the output of the convolution pro- In [19], the authors have compared DenResCov-19 and
cess to reduce the number of trainable parameters in the final DenseNet-121 for X-ray images. Although the accuracies of
layers of the model. In the proposed architecture, Rectified these models have not been mentioned, the recall of
Linear Activation Unit (RELU) [48] was utilised as an activation DenResCov-19 is 96.51 % for multi-class classification. The
function after each convolution layer. The introduction of authors of [55] have compared various models for binary clas-
max-pooling layers reduced computational complexity. sification out of which InceptionV3 produces an accuracy of
96.20 %. The authors of [56] have developed a FractalCovNet
6.2. Model comparison with related works architecture for segmentation of chest CT-scan images to
localize the lesion region and have trained it using transfer
The proposed work aims to create a VGG inspired CNN model learning for binary classification achieving an accuracy of
that takes one channel image data as input for detecting and 98 %. The CNN that was proposed in [57] was a VGG16 which
identifying COVID-19 using chest X-rays. All the datasets uti- was optimized with five inception modules, 128 neurons in
lized are treated in accordance with Section 40 s discussion. the two fully connected layers, and a learning rate of 0.0027.
Hence, the datasets considered for comparative analysis The proposed method achieved a sensitivity of 97 % for mul-
would be different from the dataset considered for the model ticlass classification and accuracy it achieved was 88 %.
being proposed. Multi-task ViT [58] was used for the multiclass which had
Most of the papers have used 5-fold cross validation or 10- an accuracy and recall of 85.8 % and 87.43 %. In [22], the
fold cross validation, and some of them have only done a authors stacked CNN to create a model which gave the sensi-
hold-out validation, this needs to be considered while com- tivity score as 97.42 % and accuracy of 97.18. Uncertainty-
paring the results. A precis for the comparative analysis is aware convolution networks were developed in [23] paper
provided in Table 4. The comparison is done based on accu- which performed well for binary class with an accuracy of
racy and recall along with the classification type. 99.36 % and for multiclass it was 97.67 %. Other studies took
12 biocybernetics and biomedical engineering 43 (2023) 1–16
for comparison shows new variation in deep learning models Table 5 shows the ANOVA results of binary classification
like the BRISK VGG-19 [59] with 96.5 % accuracy for multiclass task. From the table it is observed that the p value (0.2541)
whereas for binary type classification there were FAWT model is more than 0.05 which implies that there is no significant
[60], deep and correlation feature model [61]. TL-med model difference in the classification results of the models used
[62] with an accuracy of 93.47 %, 97.8,93.24 % respectively. for comparison with the proposed model. This means that
The Cascade VGGCOV19-Net [63] gave a performance score null hypothesis (H0) is accepted. Tukey HSD test was carried
in terms of accuracy of 97.16 % for multiclass and 99.84 % out which showed that the proposed model’s classification
for binary classification task. result showed a no significant difference in means from the
The Analysis of Variance (ANOVA) test was done to evalu- top five performing model used for the statistical analysis
ate the statistical significance of the results attained from the test. On a quantitative basis, the proposed model gave better
proposed model. This test was used to infer whether there is accuracy score than the three of the top five best performing
a significant difference in the performance of the proposed model from the comparison table.
model along with other related works. The null hypothesis Table 6 shows the ANOVA results of multiclass classifi-
in ANOVA is that there is no difference in means of samples cation task. From the table it is observed that the p value
considered for the test. For this test, metric of the top five per- (0.5335) is greater than 0.05 which implies the classification
forming models from the Table 4 is used to analyse the statis- results of the models used for comparison have no signif-
tical significance of the proposed model, for both binary and icant difference, thus proving that null hypothesis (H0) is
multiclass classification task. Tukey’s honestly significant dif- true. This means that the performance of the proposed
ference test (Tukey’s HSD) was used to test differences among model is at par with the top five performing models used
sample means of proposed models with other related models for the comparison study. Tukey HSD test also pointed
to estimate the significances. Fig. 9 shows the ANOVA test towards the same inference and showed no significant dif-
result graph and Fig. 10 shows the Tukey HSD test result graph ference of the proposed model’s classification result when
for the comparison between the proposed model and the compared with other top performing models used for the
other models. ‘Group 60 in both graphs denotes the proposed statistical test. The proposed model’s performance was
GrayVIC model and other groups denote the models used for as good as other models with a better computational
comparison purpose. efficacy.
biocybernetics and biomedical engineering 43 (2023) 1–16 13
Fig. 9 – ANOVA test result graph: (a) binary classification; (b) multiclass classification.
Fig. 10 – Tukey HSD test result graph: (a) binary classification; (b) multiclass classification.
The suggested model has a shorter training time than the This model can also be used for feature extraction since
other models included in the comparison table since it fully connected layers can be removed at the end. The objec-
includes fewer dense as well as convolutional layers. The tive of this proposed model was to make it work on grayscale
advantages of the suggested model are: images, since it is specifically trained on them, it can be used
as feature extraction model that can obtain useful artefacts
1) The number of parameters for the proposed model is from grayscale images like X-rays in a more effective manner.
around 2.7 million which is less than VGG-16 and Mob- The main focus of the suggested model is to help the health
NetV2 architecture. care sector in reducing the burden on the medical staff by
2) The training time for the model for both classification providing a quick screening system to identify the critical
tasks is approximately 5 to 8 min. CXR images. The proposed model can also be used for CT
3) The ReduceLRonPlateau technique restricts the fluctua- scans but it will not be as feasible as CXR images due to its
tions of validation accuracy during the training of the expensive nature and its availability only in large multina-
model. tional hospitals. In addition to that, it is pointless to conduct
14 biocybernetics and biomedical engineering 43 (2023) 1–16
[3] Bader F, Manla Y, Atallah B, Starling RC. Heart failure and of COVID-19 from radiography images. Biocybern Biomed Eng
COVID-19. Heart Fail Rev 2021;26:1–10. https://doi.org/ 2021;41:1702–18. https://doi.org/10.1016/j.bbe.2021.10.004.
10.1007/s10741-020-10008-2. [21] Gouda W, Almurafeh M, Humayun M, Jhanjhi NZ. Detection
[4] Raza A, Estepa A, Chan V, Jafar MS. Acute Renal Failure in of COVID-19 Based on Chest X-rays Using Deep Learning.
Critically Ill COVID-19 Patients With a Focus on the Role of Healthcare 2022;10:343. https://doi.org/10.3390/
Renal Replacement Therapy: A Review of What We Know So healthcare10020343.
Far. Cureus 2020;12. https://doi.org/10.7759/cureus.8429. [22] Gour M, Jain S. Automated COVID-19 detection from X-ray
[5] Feng G, Zheng KI, Yan Q-Q, Rios RS, Targher G, Byrne CD, et al. and CT images with stacked ensemble convolutional neural
COVID-19 and Liver Dysfunction: Current Insights and network. Biocybern Biomed Eng 2022;42:27–41. https://doi.
Emergent Therapeutic Strategies. J Clin Transl Hepatol 2020;8 org/10.1016/j.bbe.2021.12.001.
(1):1–7. [23] Gour M, Jain S. Uncertainty-aware convolutional neural
[6] Riphagen S, Gomez X, Gonzalez-Martinez C, Wilkinson N, network for COVID-19 X-ray images classification. Comput
Theocharis P. Hyperinflammatory shock in children during Biol Med 2022;140. https://doi.org/10.1016/
COVID-19 pandemic. Lancet 2020;395:1607–8. https://doi.org/ j.compbiomed.2021.105047 105047.
10.1016/S0140-6736(20)31094-1. [24] Xie Y, Richmond D. Pre-training on Grayscale ImageNet
[7] Mokhtari T, Hassani F, Ghaffari N, Ebrahimi B, Yarahmadi A, Improves Medical Image Classification. In: Leal-Taixé L, Roth
Hassanzadeh G. COVID-19 and multiorgan failure: A S, editors. Comput. Vis. – ECCV 2018 Workshop, vol. 11134,
narrative review on potential mechanisms. J Mol Histol Cham: Springer International Publishing; 2019, p. 476–84.
2020;51:613–28. https://doi.org/10.1007/s10735-020-09915-3. https://doi.org/10.1007/978-3-030-11024-6_37.
[8] Yang W, Yan F. Patients with RT-PCR-confirmed COVID-19 and [25] Oppenheim A, Schafer R, Stockham T. Nonlinear filtering of
Normal Chest CT. Radiology 2020;295:E3–E. https://doi.org/ multiplied and convolved signals. IEEE Trans Audio
10.1148/radiol.2020200702. Electroacoustics 1968;16:437–66. https://doi.org/10.1109/
[9] Abdulkareem M, Petersen SE. The Promise of AI in Detection, TAU.1968.1161990.
Diagnosis, and Epidemiology for Combating COVID-19: [26] Madisetti VK, editor. The Digital Signal Processing Handbook:
Beyond the Hype. Front Artif Intell 2021;4. https://doi.org/ Digital Signal Processing Fundamentals. Boca Raton: CRC
10.3389/frai.2021.652669 652669. Press; 2017. https://doi.org/10.1201/9781420046076.
[10] Jain DK, Singh T, Saurabh P, Bisen D, Sahu N, Mishra J, et al. [27] Chowdhury MEH, Rahman T, Khandakar A, Mazhar R,
Deep Learning-Aided Automated Pneumonia Detection and Kadir MA, Mahbub ZB, et al. Can AI help in screening
Classification Using CXR Scans. Comput Intell Neurosci Viral and COVID-19 pneumonia? IEEE Access
2022;2022:e7474304. https://doi.org/10.1155/2022/7474304. 2020;8:132665–76.
[11] Singh T, Saurabh P, Bisen D, Kane L, Pathak M, Sinha GR. Ftl- [28] Rahman T, Khandakar A, Qiblawey Y, Tahir A, Kiranyaz S,
CoV19: A Transfer Learning Approach to Detect COVID-19. Abul Kashem SB, et al. Exploring the effect of image
Comput Intell Neurosci 2022;2022:e1953992. https://doi.org/ enhancement techniques on COVID-19 detection using chest
10.1155/2022/1953992. X-ray images. Comput Biol Med 2021;132:104319.
[12] Suzuki K. Overview of deep learning in medical imaging. [29] Kermany D, Zhang K, Goldbaum M. Labeled Optical
Radiol Phys Technol 2017;10:257–73. https://doi.org/10.1007/ Coherence Tomography (OCT) and Chest X-Ray Images
s12194-017-0406-5. for Classification 2018;2. https://doi.org/10.17632/
[13] Chen X-W, Lin X. Big Data Deep Learning: Challenges and rscbjbr9sj.2.
Perspectives. IEEE Access 2014;2:514–25. https://doi.org/ [30] Joshi RC, Yadav S, Pathak VK, Malhotra HS, Khokhar HVS,
10.1109/ACCESS.2014.2325029. Parihar A, et al. A deep learning-based COVID-19 automatic
[14] Aslan MF, Sabanci K, Durdu A, Unlersen MF. COVID-19 diagnostic framework using chest X-ray images. Biocybern
diagnosis using state-of-the-art CNN architecture features Biomed Eng 2021;41(1):239–54.
and Bayesian Optimization. Comput Biol Med 2022;142. [31] Siracusano G, La Corte A, Gaeta M, Cicero G, Chiappini M,
https://doi.org/10.1016/j.compbiomed.2022.105244 105244. Finocchio G. Pipeline for Advanced Contrast Enhancement
[15] Alakus TB, Turkoglu I. Comparison of deep learning (PACE) of Chest X-ray in Evaluating COVID-19 Patients by
approaches to predict COVID-19 infection. Chaos Solitons Combining Bidimensional Empirical Mode Decomposition
Fractals 2020;140. https://doi.org/10.1016/j.chaos.2020.110120 and Contrast Limited Adaptive Histogram Equalization
110120. (CLAHE). Sustainability 2020;12:8573. https://doi.org/
[16] Ibrahim DM, Elshennawy NM, Sarhan AM. Deep-chest: Multi- 10.3390/su12208573.
classification deep learning model for diagnosing COVID-19, [32] Victor Ikechukwu A, Murali S, Deepu R, Shivamurthy RC.
pneumonia, and lung cancer chest diseases. Comput Biol ResNet-50 vs VGG-19 vs training from scratch: A comparative
Med 2021;132. https://doi.org/10.1016/ analysis of the segmentation and classification of Pneumonia
j.compbiomed.2021.104348 104348. from chest X-ray images. Glob Transit Proc 2021;2:375–81.
[17] Ni Q, Sun ZY, Qi Li, Chen W, Yang Yi, Wang Li, et al. A deep https://doi.org/10.1016/j.gltp.2021.08.027.
learning approach to characterize 2019 coronavirus disease [33] Bashar A, Latif G, Ben Brahim G, Mohammad N, Alghazo J.
(COVID-19) pneumonia in chest CT images. Eur Radiol COVID-19 Pneumonia Detection Using Optimized Deep
2020;30(12):6517–27. Learning Techniques. Diagn Basel Switz 2021;11:1972. https://
[18] Ibrahim AU, Ozsoz M, Serte S, Al-Turjman F, Yakoi PS. doi.org/10.3390/diagnostics11111972.
Pneumonia Classification Using Deep Learning from Chest X- [34] Ozturk T, Talo M, Yildirim EA, Baloglu UB, Yildirim O,
ray Images During COVID-19. Cogn Comput 2021. https://doi. Rajendra AU. Automated detection of COVID-19 cases using
org/10.1007/s12559-020-09787-5. deep neural networks with X-ray images. Comput Biol Med
[19] Mamalakis M, Swift AJ, Vorselaars B, Ray S, Weeks S, Ding W, 2020;121. https://doi.org/10.1016/j.compbiomed.2020.103792
et al. DenResCov-19: A deep transfer learning network for 103792.
robust automatic classification of COVID-19, pneumonia, and [35] Gonzalez RC, Woods RE. Digital image processing. New York,
tuberculosis from X-rays. Comput Med Imaging Graph NY: Pearson; 2018.
2021;94:102008. [36] Simonyan K, Zisserman A. Very Deep Convolutional
[20] Murugan R, Goel T, Mirjalili S, Chakrabartty DK. WOANet: Networks for Large-Scale Image Recognition 2015.
Whale optimized deep neural network for the classification https://doi.org/10.48550/arXiv.1409.1556.
16 biocybernetics and biomedical engineering 43 (2023) 1–16
[37] Namani S, Akkapeddi L, Bantu S. Performance Analysis of [51] Sethy PK, Behera SK. Detection of Coronavirus Disease
VGG-19 Deep Learning Model for COVID-19 Detection, 2022. (COVID-19) Based on Deep Features 2020. https://doi.org/
https://doi.org/10.23919/INDIACom54597.2022.9763177. 10.20944/preprints202003.0300.v1.
[38] Hamwi WA, Almustafa MM. Development and integration of [52] Wang L, Lin ZQ, Wong A. COVID-Net: a tailored deep
VGG and dense transfer-learning systems supported with convolutional neural network design for detection of COVID-
diverse lung images for discovery of the Coronavirus identity. 19 cases from chest X-ray images. Sci Rep 2020;10:19549.
Inform Med Unlocked 2022;32. https://doi.org/10.1016/j. https://doi.org/10.1038/s41598-020-76550-z.
imu.2022.101004 101004. [53] Apostolopoulos ID, Mpesiana TA. Covid-19: automatic
[39] Abraham B, Nair MS. Computer-aided detection of COVID-19 detection from X-ray images utilizing transfer learning with
from X-ray images using multi-CNN and Bayesnet classifier. convolutional neural networks. Phys Eng Sci Med
Biocybern Biomed Eng 2020;40:1436–45. https://doi.org/ 2020;43:635–40. https://doi.org/10.1007/s13246-020-00865-4.
10.1016/j.bbe.2020.08.005. [54] Altan A, Karasu S. Recognition of COVID-19 disease from X-
[40] Perumal V, Narayanan V, Rajasekar SJS. Prediction of COVID ray images by hybrid model consisting of 2D curvelet
Criticality Score with Laboratory, Clinical and CT Images transform, chaotic salp swarm algorithm and deep learning
using Hybrid Regression Models. Comput Methods Programs technique. Chaos Solitons Fractals 2020;140. https://doi.org/
Biomed 2021;209. https://doi.org/10.1016/j.cmpb.2021.106336 10.1016/j.chaos.2020.110071 110071.
106336. [55] Narin A, Kaya C, Pamuk Z. Automatic detection of
[41] Perumal V, Narayanan V, Rajasekar SJS. Detection of COVID- coronavirus disease (COVID-19) using X-ray images and deep
19 using CXR and CT images using Transfer Learning and convolutional neural networks. Pattern Anal Appl
Haralick features. Appl Intell Dordr Neth 2021;51:341–58. 2021;24:1207–20. https://doi.org/10.1007/s10044-021-00984-y.
https://doi.org/10.1007/s10489-020-01831-z. [56] Munusamy H, Muthukumar KJ, Gnanaprakasam S,
[42] Nandini GS, Kumar APS. K C. Dropout technique for image Shanmugakani TR, Sekar A. FractalCovNet architecture for
classification based on extreme learning machine. Glob COVID-19 Chest X-ray image Classification and CT-scan
Transit Proc 2021;2:111–6. https://doi.org/10.1016/j. image Segmentation. Biocybern Biomed Eng 2021;41:1025–38.
gltp.2021.01.015. https://doi.org/10.1016/j.bbe.2021.06.011.
[43] Prusty MR, Jayanthi T, Velusamy K. Weighted-SMOTE: A [57] Lacerda P, Barros B, Albuquerque C, Conci A. Hyperparameter
modification to SMOTE for event classification in sodium Optimization for COVID-19 Pneumonia Diagnosis Based on
cooled fast reactors. Prog Nucl Energy 2017;100:355–64. Chest CT. Sensors 2021;21:2174. https://doi.org/10.3390/
https://doi.org/10.1016/j.pnucene.2017.07.015. s21062174.
[44] Mishra NK, Singh P, Joshi SD. Automated detection of COVID- [58] Park S, Kim G, Oh Y, Seo JB, Lee SM, Kim JH, et al. Multi-task
19 from CT scan using convolutional neural network. vision transformer using low-level chest X-ray feature corpus
Biocybern Biomed Eng 2021;41:572–88. https://doi.org/ for COVID-19 diagnosis and severity quantification. Med
10.1016/j.bbe.2021.04.006. Image Anal 2022;75:102299.
[45] Baltazar LR, Manzanillo MG, Gaudillo J, Viray ED, Domingo M, [59] Bhattacharyya A, Bhaik D, Kumar S, Thakur P, Sharma R,
Tiangco B, et al. Artificial intelligence on COVID-19 Pachori RB. A deep learning based approach for automatic
pneumonia detection using chest xray images. PLoS One detection of COVID-19 cases using chest X-ray images.
2021;16(10):e0257884. Biomed Signal Process Control 2022;71. https://doi.org/
[46] Fang L, Wang X. COVID-RDNet: A novel coronavirus 10.1016/j.bspc.2021.103182 103182.
pneumonia classification model using the mixed dataset by [60] Patel RK, Kashyap M. Automated diagnosis of COVID stages
CT and X-rays images. Biocybern Biomed Eng 2022;42:977–94. from lung CT images using statistical features in 2-
https://doi.org/10.1016/j.bbe.2022.07.009. dimensional flexible analytic wavelet transform. Biocybern
[47] Sarv Ahrabi S, Scarpiniti M, Baccarelli E, Momenzadeh A. Biomed Eng 2022;42:829–41. https://doi.org/10.1016/j.
An Accuracy vs. Complexity Comparison of Deep Learning bbe.2022.06.005.
Architectures for the Detection of COVID-19 Disease. [61] Kumar R, Arora R, Bansal V, Sahayasheela VJ, Buckchash H,
Computation 2021;9:3. https://doi.org/ Imran J, et al. Classification of COVID-19 from chest x-ray
10.3390/computation9010003. images using deep features and correlation coefficient.
[48] Mk MV, Atalla S, Almuraqab N, Moonesar IA. Detection of Multimed Tools Appl 2022;81(19):27631–55.
COVID-19 Using Deep Learning Techniques and [62] Meng J, Tan Z, Yu Y, Wang P, Liu S. TL-med: A Two-stage
Cost Effectiveness Evaluation: A Survey. Front. Artif Intell transfer learning recognition model for medical images of
2022:5. COVID-19. Biocybern Biomed Eng 2022;42:842–55. https://doi.
[49] Afshar P, Heidarian S, Naderkhani F, Oikonomou A, org/10.1016/j.bbe.2022.04.005.
Plataniotis KN, Mohammadi A. COVID-CAPS: A capsule [63] Karacı A. VGGCOV19-NET: automatic detection of COVID-19
network-based framework for identification of COVID-19 cases from X-ray images using modified VGG19 CNN
cases from X-ray images. Pattern Recognit Lett architecture and YOLO algorithm. Neural Comput Appl
2020;138:638–43. https://doi.org/10.1016/j.patrec.2020.09.010. 2022;34:8253–74. https://doi.org/10.1007/s00521-022-06918-x.
[50] Hemdan EE-D, Shouman MA, Karar ME. COVIDX-Net: A [64] Gupta RK, Kunhare N, Pateriya RK, Pathik N. A Deep Neural
Framework of Deep Learning Classifiers to Diagnose COVID- Network for Detecting Coronavirus Disease Using Chest X-
19 in X-Ray Images 2020. https://doi.org/10.48550/ Ray Images: Int J Healthc Inf Syst Inform 2022;17:1–27.
arXiv.2003.11055. https://doi.org/10.4018/IJHISI.20220401.oa1.