Enter your email address
Submit
Write your message
International Journal of Medical Laboratory
Shahid Sadoughi University of Medical Sciences
Fri, Jan 24, 2025
[Archive]
Volume 10, Issue 3 (August 2023)
IJML 2023, 10(3): 247-253 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Gholami A, Yousefi P, Tabibzadeh A. Convolutional Neural Networks: A Simple and Functional Approach for COVID-19 Severity Prediction. IJML 2023; 10 (3) :247-253
URL: http://ijml.ssu.ac.ir/article-1-480-en.html
URL: http://ijml.ssu.ac.ir/article-1-480-en.html
Department of Virology, Faculty of Medicine, Iran University of Medical Sciences, Tehran, Iran
Full-Text [PDF 386 kb]
(159 Downloads)
| Abstract (HTML) (445 Views)
References
Full-Text: (144 Views)
Introduction
Severe acute respiratory syndrome Coronavirus -2 (SARS-CoV-2) is an RNA virus in the Coronaviridae family. The virus started spreading around the world in 2019 leading to a pandemic condition [1]. There are a variety of studies focused on the clinical, radiological,
and laboratory features of this virus [2-4]. Moreover, the therapeutic approaches and vaccines were reviewed extensively [5, 6].
The pandemic of SARS-CoV-2 changed the face of medicine and highlighted the importance of technology, machine learning, and AI (Artificial Intelligence) in medicine [7]. Since the start of the pandemic in 2019, the virus has infected millions of people and evolved into different variants [8]. There are some differences between clinical presentation, vaccine response, and severity of disease between different variants of the virus [9]. One of the important approaches for SARS-CoV-2 detection and estimating the severity of disease is using clinical and laboratory profiles of patients [10], such as using D-dimer, C-reactive protein (CRP), lymphocyte, and neutrophil count [11].
Since 2016, the USA Food and Drug Administration (FDA) approved the use of Artificial intelligence (AI) technology in the healthcare system. Since then, many applications have been using this technology in different fields of medicine [12]. No doubt using AI in healthcare has many limitations
and so many considerations in this field should be considered. For example; two separate academic articles were published and confirmed that many proposed models that used AI were flawed due to methodology, potential bias, or poor reporting, and the authors recommended that none be used in clinical practice in Coronavirus disease 2019 (COVID-19) pandemic [13]. Nevertheless, few applications are deployed in this field [7]. Most of the developed AI-based applications are focused on prediction and forecasting the epidemiological features and diagnostic aid in radiological imaging such as Computed tomography (CT) [7,14-16]. In addition, there are some challenges and gaps in knowledge about the algorithms for predicting COVID-19 severity using AI [17]. Based on previous studies we could mention some challenges, namely regulation, unavailability of large-scale data, noises in data, lack of multi-discipline scientists in computer science and medicine, and data privacy [18]. Thus, in the current study, we tried to evaluate a prediction algorithm for COVID-19 prognosis based on demographic and laboratory information.
Materials and Methods
Patients and data
In this study, we used 100 patients for modeling. All COVID-19 cases were confirmed by polymerase chain reaction (PCR) and demographical, laboratory, and clinical data obtained from previous studies in COVID-19 patients [19]. All ethical concerns were considered based on the Helsinki Declaration ethical principles in medical research and the ethical committee of Iran University of Medical Sciences confirms the collection of data and patients [19]. Patients' data included age, background disease (chronic kidney, lung or cardiovascular disease), Lactate dehydrogenase (LDH), undertaking three times of CRP (two days interval between each test) and disease severity (two categories mild and severe).
Data processing and model
We used demographical, background disease, and laboratory parameters of patients such as age and level of CRP at three different times and past medical history of disease for example lung or heart disease as input data of COVID-19 patients to evaluate disease severity. The outlier exclusion and scaling were performed by the Scikit-Learn (sklearn) library. The sequential neural network from the Keras library by TensorFlow was used for prediction. Three dense layers and 298 trainable parapets were used. The first layers were trained by the ReLu function and the last layer used SoftMax. A model with 200 epochs.
Validation
Model validation is a crucial step in the process of developing and evaluating machine learning models K-fold cross-validation involves dividing the data into k subsets (in our case 5), training the model on k-1 subsets, and evaluating its performance on the remaining subset. This process is repeated k times, with each subset being used as the validation set once.
The validation for the method was performed based on an acceptable F1 score (precision and recall). The clinical validation of prediction by model evaluates by receiver operating characteristic (ROC) curve. The area under curve (AUC) > 0.8 was considered an acceptable predictive result. The ROC analysis performed by the sklearn library is a graphical representation of the performance of a binary classification model at various classification thresholds. It is created by plotting the true positive rate against the false positive rate at various threshold values.
Results
The accuracy is defined as true classified data per all available data in each class. The mean F1 score for our current model was 0.62 (in the range of 0-1). The F1 scores for the severe group and mild group were 0.8 and 0.45, respectively. The charts for model overfitting are available in Fig. 1. The clinical validity was evaluated by the ROC curve. The ROC curve revealed an acceptable AUC (0.085) for both severe and mild categories (Fig. 2). The algorithm evaluation revealed higher performance for prediction of severe cases of COVID-19. Based on model evaluation, by using demographical, laboratory data (especially CRP in different time points) and background disease, our primary and simple suggested model can predict the severity of disease (severe and mild) by acceptable ROC (more than 0.8).
Discussion
Using AI could be helpful in the evaluation of the COVID-19 transmission rate [20], progression, outcome [21], and diagnosis [14]. Most of the conducted studies evaluate radiological features of COVID-19 while there are limited studies for laboratory parameters [7].
and laboratory features of this virus [2-4]. Moreover, the therapeutic approaches and vaccines were reviewed extensively [5, 6].
The pandemic of SARS-CoV-2 changed the face of medicine and highlighted the importance of technology, machine learning, and AI (Artificial Intelligence) in medicine [7]. Since the start of the pandemic in 2019, the virus has infected millions of people and evolved into different variants [8]. There are some differences between clinical presentation, vaccine response, and severity of disease between different variants of the virus [9]. One of the important approaches for SARS-CoV-2 detection and estimating the severity of disease is using clinical and laboratory profiles of patients [10], such as using D-dimer, C-reactive protein (CRP), lymphocyte, and neutrophil count [11].
Since 2016, the USA Food and Drug Administration (FDA) approved the use of Artificial intelligence (AI) technology in the healthcare system. Since then, many applications have been using this technology in different fields of medicine [12]. No doubt using AI in healthcare has many limitations
and so many considerations in this field should be considered. For example; two separate academic articles were published and confirmed that many proposed models that used AI were flawed due to methodology, potential bias, or poor reporting, and the authors recommended that none be used in clinical practice in Coronavirus disease 2019 (COVID-19) pandemic [13]. Nevertheless, few applications are deployed in this field [7]. Most of the developed AI-based applications are focused on prediction and forecasting the epidemiological features and diagnostic aid in radiological imaging such as Computed tomography (CT) [7,14-16]. In addition, there are some challenges and gaps in knowledge about the algorithms for predicting COVID-19 severity using AI [17]. Based on previous studies we could mention some challenges, namely regulation, unavailability of large-scale data, noises in data, lack of multi-discipline scientists in computer science and medicine, and data privacy [18]. Thus, in the current study, we tried to evaluate a prediction algorithm for COVID-19 prognosis based on demographic and laboratory information.
Materials and Methods
Patients and data
In this study, we used 100 patients for modeling. All COVID-19 cases were confirmed by polymerase chain reaction (PCR) and demographical, laboratory, and clinical data obtained from previous studies in COVID-19 patients [19]. All ethical concerns were considered based on the Helsinki Declaration ethical principles in medical research and the ethical committee of Iran University of Medical Sciences confirms the collection of data and patients [19]. Patients' data included age, background disease (chronic kidney, lung or cardiovascular disease), Lactate dehydrogenase (LDH), undertaking three times of CRP (two days interval between each test) and disease severity (two categories mild and severe).
Data processing and model
We used demographical, background disease, and laboratory parameters of patients such as age and level of CRP at three different times and past medical history of disease for example lung or heart disease as input data of COVID-19 patients to evaluate disease severity. The outlier exclusion and scaling were performed by the Scikit-Learn (sklearn) library. The sequential neural network from the Keras library by TensorFlow was used for prediction. Three dense layers and 298 trainable parapets were used. The first layers were trained by the ReLu function and the last layer used SoftMax. A model with 200 epochs.
Validation
Model validation is a crucial step in the process of developing and evaluating machine learning models K-fold cross-validation involves dividing the data into k subsets (in our case 5), training the model on k-1 subsets, and evaluating its performance on the remaining subset. This process is repeated k times, with each subset being used as the validation set once.
The validation for the method was performed based on an acceptable F1 score (precision and recall). The clinical validation of prediction by model evaluates by receiver operating characteristic (ROC) curve. The area under curve (AUC) > 0.8 was considered an acceptable predictive result. The ROC analysis performed by the sklearn library is a graphical representation of the performance of a binary classification model at various classification thresholds. It is created by plotting the true positive rate against the false positive rate at various threshold values.
Results
The accuracy is defined as true classified data per all available data in each class. The mean F1 score for our current model was 0.62 (in the range of 0-1). The F1 scores for the severe group and mild group were 0.8 and 0.45, respectively. The charts for model overfitting are available in Fig. 1. The clinical validity was evaluated by the ROC curve. The ROC curve revealed an acceptable AUC (0.085) for both severe and mild categories (Fig. 2). The algorithm evaluation revealed higher performance for prediction of severe cases of COVID-19. Based on model evaluation, by using demographical, laboratory data (especially CRP in different time points) and background disease, our primary and simple suggested model can predict the severity of disease (severe and mild) by acceptable ROC (more than 0.8).
Discussion
Using AI could be helpful in the evaluation of the COVID-19 transmission rate [20], progression, outcome [21], and diagnosis [14]. Most of the conducted studies evaluate radiological features of COVID-19 while there are limited studies for laboratory parameters [7].
Fig. 1. Model accuracy and model loss charts are important tools for evaluating the performance of machine learning models during training and testing. Ideally, we want both orange and blue lines to decrease and increase in the same direction that we know the model in learning and a confusion matrix is a table that summarizes the performance of a classification model by comparing predicted and actual class labels.
A conducted study by Chimmula et al. [20] suggests the end of the pandemic condition in Canada using Long-Short-Term Memory (LSTM) networks. In a conducted study by Li et al. [21] evaluation of COVID-19 background and imaging features indicates that consolidation volume and age can predict disease progression while age and diabetes were predictors for unfavorable outcome. Another study by Sun et al. [22], revealed high performance for prediction of severe or critical COVID-19 using laboratory and demographic features. The suggested model by Sun and colleagues evaluates 336 COVID-19 cases by Support Vector Machine (SVM) which can predict critical patients by ROC > 0.9. Furthermore, a recent study by Chieregato et al. [23] revealed the high performance of CNN models for COVID-19 Intensive care unit admission based on laboratory and imaging data. In this regard, Jahangirimehr and colleagues [24] suggested the SVM models for COVID-19 severity based on vitamin D, calcium, and zink status. Our current suggested model revealed that, based on demographical, laboratory data, and background disease, could predict the severity of disease by acceptable ROC.
Fig. 2. ROC curve for mild (0) and severe (1) patients based on laboratory and background data. The X-axis represents the true positive rate or sensitivity and the Y-axis represents the false positive rate or 1-specificity
The main difference between the performances of our model and with other mentioned models is due to the number of patients and evaluated variables. Our model used the minimum data to predict severity. It is reasonable to use a big database and adding extra features can improve the model efficacy. AI can help healthcare providers predict the severity of COVID-19 by analyzing patient data such as age, sex, comorbidities, and biomarkers. The use of AI in predicting COVID-19 severity has the potential to improve patient outcomes by enabling earlier identification of patients at high risk for severe disease. This can help healthcare providers prioritize resources and provide more personalized care to patients, but we have challenges ahead of us and that is we need big data for training AI for prediction analysis. In this study, we used 100 patient data and our model is limited so we encourage other researchers to provide more data and more training algorithms. Using bigger data sets can provide a more comprehensive view of the suggested model performance. This limited number of primary data and limited laboratory features for each patient is the most important limitation in our current study. There are a wide range of different variables, that can be effects in disease severity in COVID-19. In addition, it should be noted that this study is just a preliminary study and our model could be improved. Using AI in laboratory medicine provides a view landscape for future research in prediction, detection, and laboratory quality control.
Conclusion
The current study introduces a simple machine learning algorithm as an appropriate tool for determining the disease severity of COVID-19 by acceptable ROC analysis. This study could lead us to use such algorithms more often in laboratory medicine and clinical decision-making. Furthermore, the current study is just a preliminary study and highlights the need for further research to validate and refine the proposed predictive model.
Conflict of Interests
There is no conflict of interest.
Acknowledgment
We are grateful to all of those with whom we have had the pleasure to work during this and other related projects.
Conclusion
The current study introduces a simple machine learning algorithm as an appropriate tool for determining the disease severity of COVID-19 by acceptable ROC analysis. This study could lead us to use such algorithms more often in laboratory medicine and clinical decision-making. Furthermore, the current study is just a preliminary study and highlights the need for further research to validate and refine the proposed predictive model.
Conflict of Interests
There is no conflict of interest.
Acknowledgment
We are grateful to all of those with whom we have had the pleasure to work during this and other related projects.
References
- Khan M, Adil SF, Alkhathlan HZ, Tahir MN, Saif S, Khan M, et al. COVID-19: a global challenge with old history, epidemiology and progress so far. Molecules 2020; 26(1): 39.
- Tabibzade A, Abbasi MA, Karampoor S, Niya MHK, Esghaei M, Mousavi SAJ, et al. Clinical presentations of Iranian confirmed COVID-19 patients at the admission time: A preliminary retrospective study. International Journal of Medical Laboratory. 2021; 8(4): 256-263
- Mehta OP, Bhandari P, Raut A, Kacimi SEO, Huy NT. Coronavirus disease (COVID-19): comprehensive review of clinical presentation. Frontiers in Public Health 2021; 8: 582932.
- Wong CK, Wong JY, Tang EH, Au CH, Wai AK. Clinical presentations, laboratory and radiological findings, and treatments for 11,028 COVID-19 patients: a systematic review and meta-analysis. Scientific Reports 2020; 10(1): 1-16.
- Li YD, Chi WY, Su JH, Ferrall L, Hung CF, Wu TC. Coronavirus vaccine development: from SARS and MERS to COVID-19. Journal of Biomedical Science 2020; 27(1): 1-23.
- Stasi C, Fallani S, Voller F, Silvestri C. Treatment for COVID-19: An overview. European Journal of Pharmacology 2020; 889: 173644.
- Lalmuanawma S, Hussain J, Chhakchhuak L. Applications of machine learning and artificial intelligence for Covid-19 (SARS-CoV-2) pandemic: A review. Chaos, Solitons & Fractals 2020; 139: 110059.
- Telenti A, Hodcroft EB, Robertson DL. The evolution and biology of SARS-CoV-2 variants. Cold Spring Harbor Perspectives in Medicine 2022; 12(5): 41390.
- Lauring AS, Tenforde MW, Chappell JD, Gaglani M, Ginde AA, McNeal T, et al. Clinical severity of, and effectiveness of mRNA vaccines against, covid-19 from omicron, delta, and alpha SARS-CoV-2 variants in the United States: Prospective observational study. BMJ 2022; 9:(376): 69761.
- Borges do Nascimento IJ, von Groote TC, O’Mathúna DP, Abdulazeem HM, Henderson C, Jayarajah U, et al. Clinical, laboratory and radiological characteristics and outcomes of novel coronavirus (SARS-CoV-2) infection in humans: A systematic review and series of meta-analyses. PloS One 2020; 15(9): 239235.
- Singh K, Mittal S, Gollapudi S, Butzmann A, Kumar J, Ohgami RS. A meta-analysis of SARS‐CoV‐2 patients identifies the combinatorial significance of D-dimer, C-reactive protein, lymphocyte, and neutrophil values as a predictor of disease severity. International Journal of Laboratory Hematology 2021; 43(2): 324-28.
- Wu E, Wu K, Daneshjou R, Ouyang D, Ho DE, Zou J. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 2021; 27(4): 582-84.
- Roberts M, Driggs D, Thorpe M, Gilbey J, Yeung M, Ursprung S, et al. Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans. Nature Machine Intelligence 2021; 3(3): 199-217.
- Ardakani AA, Kanafi AR, Acharya UR, Khadem N, Mohammadi A. Application of deep learning technique to manage COVID-19 in routine clinical practice using CT images: Results of 10 convolutional neural networks. Computers in Biology and Medicine 2020; 121: 103795.
- Ozturk T, Talo M, Yildirim EA, Baloglu UB, Yildirim O, Acharya UR. Automated detection of COVID-19 cases using deep neural networks with X-ray images. Computers in Biology and Medicine 2020; 121: 103792.
- Yan L, Zhang HT, Goncalves J, Xiao Y, Wang M, Guo Y, et al. An interpretable mortality prediction model for COVID-19 patients. Nature Machine Intelligence 2020; 2(5): 283-88.
- Liu T, Siegel E, Shen D. Deep learning and medical image analysis for COVID-19 diagnosis and prediction. Annual Review of Biomedical Engineering 2022; 24(1): 179-201.
- Alafif T, Tehame AM, Bajaba S, Barnawi A, Zia S. Machine and deep learning towards COVID-19 diagnosis and treatment: survey, challenges, and future directions. International Journal of Environmental Research and Public Health 2021; 18(3): 1117.
- Tabibzadeh A, Niya MHK, Keyvani H, Karampoor S, Yousefi P, Razizadeh MH, et al. Investigation of the Toll-like receptors expression profile in peripheral blood mononuclear cells in COVID-19 patients in Iran 2022. Immuno-pathologia Persa 2023; 9(2): 32405-2405.
- Chimmula VKR, Zhang L. Time series forecasting of COVID-19 transmission in Canada using LSTM networks. Chaos, Solitons & Fractals 2020; 135: 109864.
- Li Y, Shang K, Bian W, He L, Fan Y, Ren T, et al. Prediction of disease progression in patients with COVID-19 by artificial intelligence assisted lesion quantification. Scientific Reports 2020; 10(1): 22083.
- Sun L, Song F, Shi N, Liu F, Li S, Li P, et al. Combination of four clinical indicators predicts the severe/critical symptom of patients infected COVID-19. Journal of Clinical Virology 2020; 128: 104431.
- Chieregato M, Frangiamore F, Morassi M, Baresi C, Nici S, Bassetti C, et al. A hybrid machine learning/deep learning COVID-19 severity predictive model from CT images and clinical data. Scientific reports 2022; 12(1): 1-15.
- Jahangirimehr A, Shahvali EA, Rezaeijo SM, Khalighi A, Honarmandpour A, Honarmandpour F, et al. Machine learning approach for automated predicting of COVID-19 severity based on clinical and paraclinical characteristics: Serum levels of zinc, calcium, and vitamin D. Clinical Nutrition 2022; 51(2): 404-11.
Type of Study: Research |
Subject:
Virology
Received: 2023/04/21 | Accepted: 2023/05/31 | Published: 2023/10/2
Received: 2023/04/21 | Accepted: 2023/05/31 | Published: 2023/10/2
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
