Nomogram for prediction of fatal outcome in patients with severe COVID-19: a multicenter study

Background To develop an effective model of predicting fatal outcomes in the severe coronavirus disease 2019 (COVID-19) patients. Methods Between February 20, 2020 and April 4, 2020, consecutive confirmed 2541 COVID-19 patients from three designated hospitals were enrolled in this study. All patients received chest computed tomography (CT) and serological examinations at admission. Laboratory tests included routine blood tests, liver function, renal function, coagulation profile, C-reactive protein (CRP), procalcitonin (PCT), interleukin-6 (IL-6), and arterial blood gas. The SaO2 was measured using pulse oxygen saturation in room air at resting status. Independent high-risk factors associated with death were analyzed using Cox proportional hazard model. A prognostic nomogram was constructed to predict the survival of severe COVID-19 patients. Results There were 124 severe patients in the training cohort, and there were 71 and 76 severe patients in the two independent validation cohorts, respectively. Multivariate Cox analysis indicated that age ≥ 70 years (HR = 1.184, 95% CI 1.061–1.321), panting (breathing rate ≥ 30/min) (HR = 3.300, 95% CI 2.509–6.286), lymphocyte count < 1.0 × 109/L (HR = 2.283, 95% CI 1.779–3.267), and interleukin-6 (IL-6) >  10 pg/ml (HR = 3.029, 95% CI 1.567–7.116) were independent high-risk factors associated with fatal outcome. We developed the nomogram for identifying survival of severe COVID-19 patients in the training cohort (AUC = 0.900, 95% CI 0.841–0.960, sensitivity 95.5%, specificity 77.5%); in validation cohort 1 (AUC = 0.811, 95% CI 0.763–0.961, sensitivity 77.3%, specificity 73.5%); in validation cohort 2 (AUC = 0.862, 95% CI 0.698–0.924, sensitivity 92.9%, specificity 64.5%). The calibration curve for probability of death indicated a good consistence between prediction by the nomogram and the actual observation. The prognosis of severe COVID-19 patients with high levels of IL-6 receiving tocilizumab were better than that of those patients without tocilizumab both in the training and validation cohorts, but without difference (P = 0.105 for training cohort, P = 0.133 for validation cohort 1, and P = 0.210 for validation cohort 2). Conclusions This nomogram could help clinicians to identify severe patients who have high risk of death, and to develop more appropriate treatment strategies to reduce the mortality of severe patients. Tocilizumab may improve the prognosis of severe COVID-19 patients with high levels of IL-6. Supplementary Information The online version contains supplementary material available at 10.1186/s40779-021-00315-6.


Background
Previous studies have indicated that in all coronavirus disease 2019 (COVID-19) patients, the incidence of severe cases is approximately 15% [1,2]. The mortality rate of severe COVID-19 patients varies from 8.0 to 61.5% and significantly increases among older patients [3][4][5][6][7][8]. Early medical intervention is very important to reduce the mortality of severe patients. Thus, it is of great importance to screen out severe patients with a high risk of death promptly and accurately at the initial admission [9]. However, this is particularly difficult because of limited medical resources and staff and the large number of patients. Therefore, elucidating the independent risk factors and establishing an accurate model for predicting severe COVID-19 patients at high risk of death is necessary.
A previously established nomogram suggested five prognostic factors for predicting the outcome: Acute Physiology and Chronic Health Evaluation II (APACHE II), creatine kinase (CK), C-reactive protein (CRP), immunoglobulin A (IgA), and the interaction between CK and APACHE II [10]. However, risk factors associated with fatal outcomes in severe patients are unclear. This study aimed to provide a model to help clinicians identify patients with severe COVID-19 at high risk of death, which may be beneficial for decision making regarding treatment strategies.

Study population
Between February 20, 2020 and April 4, 2020, consecutive confirmed COVID-19 patients were assessed to enter into this study from three designated hospitals of COVID-19 in China: the Guanggu Branch of the Women and Children's Hospital of Hubei Province, Tongji Taikang Hospital and Huoshen Mountain Hospital. The diagnosis of COVID-19 was based on the World Health Organization (WHO) interim guidance and guidelines for diagnosis and treatment of novel coronavirus pneumonia (5 th version) released by the National Health Commission of China [11,12]. The presence of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in respiratory specimens was confirmed by a positive result of quantitative real-time reverse transcriptase-polymerase chain reaction (qRT-PCR) assay from nasal or pharyngeal swab specimens. Swab samples were collected and tested for SARS-CoV-2 with the Chinese Center for Disease Control and Prevention (CDC) recommended Kit (BioGerm, Shanghai, China), following WHO guidelines for qRT-PCR [13][14][15]. All samples were tested for SARS-CoV-2 by use of qRT-PCR with the CDC recommended Kit. The test results were confirmed by nested RT-PCR with designed primers. The nested RT-PCR assay was performed according to the previous report [16].
During the study period, a total of 2541 patients were enrolled into this study. Of 2541 cases included in this study, 1056 patients were from the Guanggu Branch of the Women and Children's Hospital of Hubei Province, 726 were from Tongji Taikang Hospital, and 759 were from Huoshen Mountain Hospital. According to the definition of severe COVID-19 described above, there were 124, 71, and 76 severe cases from the three hospitals, respectively. Therefore, 124 patients with severe diseases from the Guanggu Branch of the Women and Children's Hospital of Hubei Province were included in the training cohort, while 71 and 76 severe cases in other two hospitals formed validation cohort. The selection of the study population was shown in Supplemental Fig. 1. The study was approved by the Ethics Committee of all centers. Written informed consent was waived by the Ethics Commission of each hospital for emerging infections.

Data collection
All patients received chest computed tomography (CT) and serological examinations at admission. Laboratory tests included routine blood tests, liver function, renal function, coagulation profile, CRP, procalcitonin (PCT), interleukin-6 (IL-6), and arterial blood gas. The SaO 2 was measured using pulse oxygen saturation in room air at resting status. Comorbidity was defined as having at least one of the followings: Hypertension, diabetes, cardiovascular disease, cerebrovascular disease, chronic lung disease, and malignant tumor for at least 6 months.
All data were collected, re-checked for accuracy independently by at least two researchers.

Statistical analysis
Continuous variables were expressed as mean (SD). Categorical variables were expressed as frequency (percentage). Categorical variables were compared by the χ 2 test or Fisher's exact test. Continuous variables were compared by the Student's t test or Mann-Whitney U test. Survival curves were analyzed using the Kaplan-Meier method. Differences between curves were assessed using the Log-rank test.
Univariate and multivariate Cox proportional regression analysis was used for investigating the independent risk factors of death. The independent risk factors associated with the risk of mortality of patients with severe COVID-19 were used to build the nomogram in the training cohort. The performance and accuracy of the established nomogram were assessed by receiver operating characteristic (ROC) curve and calibration with 1000 bootstrap samples. The area under ROC (AUC) and optimal cut-off values were determined. Decision curve analysis (DCA) based on the net benefit was depicted by the package of Rmda in R. The nomogram was validated in the validation cohorts 1 and 2, respectively. The nomogram was constructed and evaluated using the R software version 3.4.1 package with the Rms and Hmisc. All statistical analysis was performed using R version 3.4.1, a P < 0.05 in two-tailed was the significance threshold.

Patient clinical characteristics
Of 2541 patients included in this study, 271 patients had severe COVID-19 and 2270 were non-severe cases. Supplemental Table 1 indicated the basic characteristics of the study cohorts. Compared with non-severe cases, patients were older (P < 0.001) and more males (P < 0.001) were found in the severe disease cohort. In addition, there is more comorbidity in severe cases (P < 0.001). The proportion of patients with panting was higher in the severe cases (P < 0.001). Higher levels of white blood cell (WBC) count, neutrophil count, CRP, PCT, IL-6, total bilirubin (TBIL), alanine aminotransferase (ALT), aspartate aminotransferase (AST), lactate dehydrogenase (LDH), γ-glutamyl transpeptidase (γ-GT), and creatinine (Cr) were identified in severe cases (P < 0.001), while the levels of lymphocyte count, platelet (PLT), and SaO 2 of patients with severe disease were lower than those in non-severe cases (P < 0.001). There were 186 patients with no manifestations of chest CT in non-severe COVID-19 cohort, and no significant differences of the findings of chest CT between severe and non-severe cases were found (P > 0.05). By the end of April 4, 2020, there were 58 and 39 patients died in the severe and non-severe disease group, respectively (Supplemental Table 1).
Comparison of baseline characteristics between patients in training and validation cohorts can be seen in Table 1. There were significant differences of age, proportion of smokers, incidence of panting, WBC, neutrophil count, CRP and SaO 2 at admission between the three cohorts (P < 0.05). There was no significant difference in the other variables between the three cohorts (P > 0.05). By the end of April 4, 2020, 22 severe COVID-19 patients died in the training group, and 22 and 14 patients died in the validation group 1 and validation group 2, respectively ( Table 1).
The baseline characteristics of patients in the training cohort were shown in Table 2. There were no significant differences in gender, TBIL, ALT, AST, LDH, γ-GT, Cr, PLT, and the proportion of smokers between survivors and non-survivors (P > 0.05). Survivors were significantly younger than the non-survivors in the training cohort [(70.4 ± 12.3) years vs. (81.6 ± 7.3) years, P < 0.05], however, the proportion of patients with multiple comorbidities and panting (breathing rate ≥ 30/min) was significantly higher in non-survivors (P < 0.05). In addition, WBC and neutrophil count, CRP, D-dimer, PCT, and IL-6 were also significantly higher in nonsurvivors (P < 0.05). The lymphocyte count was significantly lower in non-survivors (P < 0.05).
Survival analysis in the patients with the high level of IL-6 Due to the high level of IL-6 correlating with poor outcomes in severe COVID-19 patients, the therapeutic effect of tocilizumab in the patients with high IL-6 was further analyzed. In the training cohort, it was demonstrated that the prognosis of patients receiving tocilizumab was better than that of patients not receiving tocilizumab, but without significance (P = 0.105, Supplemental Fig. 2a). Similar results were also observed in the validation cohort 1 and validation cohort 2, respectively (P = 0.133, P = 0.210, Supplemental Fig. 2b-c).

Construction and validation of the nomogram
Four independent risk factors found to be associated with the risk of mortality of patients in the multivariate analyses were incorporated into the nomogram (Fig. 1). The ROC curve was employed to assess the predictive ability of the established nomogram, and the result demonstrated that the AUC was 0.900 (95% CI 0.841-0.960) in the training cohort, with a sensitivity of 95.5% and specificity of 77.5% (Fig. 2a). Moreover, the calibration curves for nomogram predicted mortality indicated that a good consistency between observed actual outcomes and predicted ones in the training cohort (Fig. 3a).
In the validation cohort 1, the AUC was 0.811 (95% CI 0.763-0.961) for patients with a sensitivity of 77.3% and specificity of 73.5% (Fig. 2b). In the validation cohort 2, the AUC was 0.862 (95% CI 0.698-0.924) for patients with a sensitivity of 92.9% and specificity of 64.5% (Fig.  2c). The calibration curves also showed good agreement between prediction and observation in the risk of mortality in the two validation cohorts (Fig. 3b-c).
Clinical application of the nomogram DCA based on the net benefit and threshold probabilities was performed to assess the clinical applicability of the risk prediction nomogram. The DCA showed that our risk prediction nomogram had a superior net benefit with a wide range of threshold probabilities in the training cohort and validation cohorts (Fig. 4).

Discussion
Our study revealed the clinical characteristics and risk factors for fatal outcomes in confirmed severe COVID-19 patients based on multicenter cohorts. Multivariate Cox analysis in this study indicated that age, lymphopenia, respiratory rate ≥ 30/min, and IL-6 were independent high-risk factors associated with poor prognosis. Older age has been proven to be a risk factor for the virus infection and survival in many previous studies [17][18][19][20]. Elderly patients with severe COVID-19 were more likely to develop fatal outcomes because of the rapid progression of the disease, which reminded us of providing early intervention for elderly severe patients. Similarly, lymphopenia was more common in the nonsurvivors and severe cases according to previous reports, suggesting dysregulation of the immune response in patients with COVID-19 [21][22][23]. Nevertheless, most of these were only descriptive studies. A study clarified that lower lymphocyte count was predictive of COVID-19  progression [24], whereas the impact of lymphocyte count on the survival of severe COVID-19 was unclear. This study demonstrated that a lymphocyte count < 1.0 × 10 9 /L was independently associated with death in severe cases.
Recent studies have found that the cytokine storm is an important factor leading to rapid disease progression and poor prognosis [25,26]. IL-6 is one of the major cytokines involved in cytokine storms [27,28]. A previous univariate analysis showed that the IL-6 level was   associated with worse survival without significance [18].
Our study showed that a high level of IL-6 was a predictor of death in severe COVID-19 patients. Additionally, the survival curve showed that the outcome was better in patients with tocilizumab than in patients without tocilizumab in the training cohort. Nonetheless, no significant difference was found. Similar results were also found in validation cohort 1 and validation cohort 2.
The reason for this result may be attributable to the small sample size. Although the previous study has shown that tocilizumab can reduce mortality, highquality studies are still needed to verify the effectiveness of tocilizumab on the survival of patients with severe COVID-19 [29]. Increasing respiratory rate is an important clinical feature of acute respiratory distress syndrome (ARDS), which is a major cause of death in severe COVID-19 patients [8,17,30]. A previous study showed that a respiratory rate ≥ 24/min was a risk factor for death in the univariate analysis [18], whereas no significance was found after the multivariate regression analysis. Our multivariate regression analysis clarified that a respiratory rate ≥ 30/min was a predictor of death. For patients with increasing respiratory rates, especially those with respiratory rate ≥ 30/min, it is necessary for physicians to be aware of the potential progression of ARDS.
Many previous studies have shown that comorbidity was significantly associated with high mortality rate and disease progression [24,31]. Nevertheless, in this study, the significance of comorbidity was only indicated in the univariate analysis, but not in the multivariate regression analysis, which may be ascribed to different patients enrolled in these studies. All the patients included in this study had severe COVID-19, and the proportion of comorbidities was approximately 70%, which was significantly higher than those in other studies.
Previous studies have shown several prediction models with different parameters [1,24,31,32]. Compared with other studies, the predictive effect of age, panting, and lymphopenia has been described in previous reports, while the main feature of this study is the analysis of the prognostic value of IL-6 in severe COVID-19 patients for the first time. IL-6 plays an important role in the pathophysiological changes of severe cases. However, the prediction value of IL-6 was not shown in other prediction models. In addition, our study also indicated that   tocilizumab, a monoclonal antibody against the IL-6 receptor, may improve the prognosis of severe COVID-19 patients with the high level of IL-6.
Our study has some limitations. First, this is a retrospective study, and there may be potential biases in the selection of patients. Second, the sample size of the study was relatively small, and the results need to be further validated in a larger cohort.

Conclusion
This study firstly developed a nomogram for predicting fatal outcomes in the severe COVID-19 patients. The four predictors included in the model are easy to obtain. The prediction risk of the model indicated a good consistency with the observed one. Hence, this nomogram may be conducive to more effective treatment to reduce the mortality of those severe cases at high risk of death.
Additional file 1: Supplemental Figure 1 Selection of the study population.