Establishment and effectiveness evaluation of a scoring system for exertional heat stroke by retrospective analysis
Military Medical Research volume 7, Article number: 40 (2020)
Heat stroke (HS) is a serious, life-threatening disease. However, there is no scoring system for HS so far. This research is to establish a scoring system that can quantitatively assess the severity of exertional heat stroke (EHS).
Data were collected from a total of 170 exertional heat stroke (EHS) patients between 2005 and 2016 from 52 hospitals in China. Univariate statistical methods and comparison of the area under the receiver operating characteristic (ROC) curve (AUC) were used to screen exertional heat stroke score (EHSS) parameters, including but not limited body temperature (T), Glasgow Coma Scale (GCS) and others. By comparing the sizes of the AUCs of the APACHE II, SOFA and EHSS assessments, the effectiveness of EHSS in evaluating the prognosis of EHS patients was verified.
Through screening with a series of methods, as described above, the present study determined 12 parameters – body temperature (T), GCS, pH, lactate (Lac), platelet count (PLT), prothrombin time (PT), fibrinogen (Fib), troponin I (TnI), aspartate aminotransferase (AST), total bilirubin (TBIL), creatinine (Cr) and acute gastrointestinal injury (AGI) classification – as EHSS parameters. It is a 0–47 point system designed to reflect increasing severity of heat stroke. Low (EHSS< 20) and high scores (EHSS> 35) showed 100% survival and 100% mortality, respectively. We found that AUCEHSS > AUCSOFA > AUCAPACHE II.
A total of 12 parameters – T, GCS, pH, Lac, PLT, PT, Fib, TnI, AST, TBIL, Cr and gastrointestinal AGI classification – are the EHSS parameters with the best effectiveness in evaluating the prognosis of EHS patients. As EHSS score increases, the mortality rate of EHS patients gradually increases.
Heat stroke (HS) is a serious, life-threatening disease characterized by elevated core body temperature that is simultaneously accompanied by central nervous system (CNS) dysfunction . The disease is often accompanied by multiple-organ dysfunction [2, 3]. HS is mainly divided into two types: classical heat stroke (CHS) and exertional heat stroke (EHS). The former more frequently occurs in children and the elderly, who are exposed to high temperatures, while the latter mostly occurs in healthy young populations who are engaged in high-intensity manual labor, including soldiers, athletes, workers and farmers . Compared with CHS, each organ injury of EHS is more serious . EHS has become the third leading cause of death among athletes .
As a criterion for evaluating the severity of disease in intensive care units (ICUs), scoring systems have become important tools to help clinical physicians to make decisions . At present, scoring systems for severe diseases are mainly divided into two major types. One type of scoring system is suitable for various diseases, such as the Acute Physiology and Chronic Health Evaluation II (APACHE II), the Simplified Acute Physiology Score (SAPS II) and the Multiple Organ Dysfunction Score (MODS); the other type of scoring system is for a specific type of organ or disease, such as the Glasgow Coma Scale (GCS) for evaluating the degree of CNS injury and the Ranson scale for evaluating the degree of injury of severe acute pancreatitis [7, 8]. At present, the scoring systems that are used to evaluate the severity of EHS are mainly APACHE II and SOFA [9, 10]. Studies have found that the common causes that affect the prognosis of EHS are disseminated intravascular coagulation (DIC) caused by coagulation dysfunction and rhabdomyolysis induced by thermal injury [11, 12]. The above-mentioned scoring systems ignore the exact roles played by the above two indicators in the scoring system, with the result that none of the existing scoring systems can evaluate the condition of EHS very objectively and comprehensively, including the systemic scores of APACHE II, SOFA, SAPSII and MODS [13,14,15,16]. As early as 12 years ago, Varghese et al.  suggested that a scoring system dedicated to HS should be established to stratify disease severity and prognosis, which is very important for choosing optimal treatment strategies and improving the success rates of clinical treatment. Once an exertional heat stroke score (EHSS) is finalized, critical care can be accessed sooner, and it may be used in the future to stratify patients for heat stroke-specific treatment. However, more than a decade has passed, and an effective HS scoring system has not yet been established. Similarly, two well-known specialists in the HS field, Professor Leon and Bouchama , both noted that APACHE II scoring was not a specific scoring criterion for HS. EHS has a characteristic dispersed outbreak, which makes it very difficult for researchers to perform large sample size-based clinical studies . This limitation may be the underlying reason that prevents the EHS scoring system from being established.
In summary, creating a scoring system that is consistent with the pathological and physiological characteristics of EHS has clinical significance in evaluating disease, judging treatment efficacy and determining prognosis. The main purpose of this study is to screen and identify the parameters that are consistent with the pathological and physiological characteristics of EHS through a retrospective study to assign corresponding values, to create a scoring system for exertional heat stroke – the EHS score (EHSS) – and to confirm its effectiveness at evaluating EHS. The mortality rates of EHS patients corresponding to different EHSS scores were also investigated.
We collected data from a total of 170 EHS patients between 2005 and 2016 from 52 hospitals in China, including the People’s Liberation Army (PLA) General Hospital. Ninety patients were randomly selected as the study subjects for establishing the EHSS, of which 69 patients (76.7%) survived, and 21 (23.3%) died. The remaining 80 patients were selected as the study subjects to verify the effectiveness of the EHSS in evaluating the prognosis of EHS patients. Of these, 62 (77.5%) survived and 18 (22.5%) died. The sources of the patients were widely distributed across various provinces of China except Tibet, and the geographic distribution of the cases was uniform. The study was led by the PLA General Hospital and was approved by the ethics committees of all 52 hospitals.
Patients and study design
In this study, the EHS patient population mainly included soldiers, athletes, workers and farmers. Inclusion criteria were as follows: 1) patients whose age was no less than 18 years; 2) patients with a history of being engaged in high-intensity manual labor; 3) patients whose axillary temperature was higher than 39 °C (studies find that the rectal temperature, which represents the core body temperature of humans, is usually higher than the oral temperature by 0.27 °C–0.38 °C, whereas the oral temperature is 0.55 °C higher than the axillary temperature, which represents body surface temperature) ; and 4) patients with CNS dysfunction, including delirium, coma, disturbances of consciousness and disorientation. The included patients met all of the above four criteria. The exclusion criteria were as follows: 1) patients who were in the hospital or ICU for less than 24 h; and 2) patients with common comorbidities before EHS onset; in our research, they had diabetes, cerebral infarction, pulmonary infection and dementia. Gastrointestinal tract injury was graded by the acute gastrointestinal injury (AGI) classification developed by the European Society of Intensive Care Medicine (ESICM) in 2012 .
Screening of the scoring system parameters for exertional heat stroke
According to our clinical experience, a literature review and the consensus of heat stroke specialists , the present study first screened 42 physiological parameters that could reflect the severity of the pathological changes of various systems in the body, particularly indicators that are characterized by high fever, rhabdomyolysis, coagulation dysfunction and nervous system dysfunction, as shown in Table 1. The worst values of the physiological parameters within 24 h after admission into the hospital or ICU were selected. First, univariate statistical analysis was used to preliminarily screen EHSS parameters. When the differences were statistically significant, their areas under the receiver operating characteristic (ROC) curves (AUCs) were compared. The diagnostic accuracy for diseases is low when the AUC is 0.5–0.7, acceptable when the AUC is 0.7–0.9 and high when the AUC is above 0.9; therefore, the present study selected the parameters with AUCs> 0.7 as the final parameters for EHSS. By combining our clinical experience and a literature review, the final parameters for EHSS were determined. Multivariate logistic regression analysis is a statistical method that requires data from a large sample size; the smaller the sample size is, the less reliable the results will be . At present, there is still no well-accepted effective sample size calculation formula for multivariate logistic regression analysis; the well-accepted sample size by academics is at least 10 times the number of parameters that are included in the multivariate logistic regression analysis. In the present study, a total of 23 parameters needed to be included in the multivariate logistic regression equation; therefore, if logistic regression analysis was to be performed, the number of EHS patients needed to be at least 230 each for the death and survival groups, for a total of 460 cases, to ensure reliable results. However, the EHS patients for this part of the study numbered only 90. Therefore, logistic regression analysis was not suitable for the screening of EHSS parameters. In contrast with multivariate logistic regression, the ROC curve is a statistical method that can be used for data analysis with a small sample size. Therefore, in the present study, we calculated the AUCs of the 23 parameters whose differences in univariate analysis were statistically significant, and we identified the parameters with AUCs greater than 0.7 as the final EHSS parameters.
Establishment of a scoring system for exertional heat stroke
The method of value assignment to parameters of the MODS scoring system was used as a reference. Values were assigned to parameters according to the mortality rates corresponding to different variable ranges of various parameters. When the variable range was given 4 points, its corresponding ICU mortality rate should be greater than 50% . The assignment of values to parameters gave corresponding grade points to physiological variables with the value assignment methods of the APACHE II and MODS scoring systems as the references, and the assignment was divided into five grades according to the abnormalities of the parameters, with various variables having assigned values of 0–4 points [13, 16].
Verification of the evaluation effectiveness of EHSS on the prognosis of EHS patients.
The worst values of the APACHE II, SOFA and EHSS scoring systems within 24 h after admission to the hospital or ICU were calculated for the 80 EHS patients. The AUCs of the three scoring systems were calculated, and the predictive effectiveness of the EHSS on EHS patients was judged by comparing the sizes of the AUCs of the three scoring systems.
Exploring the correlations between different EHSS scores and the prognosis of EHS patients
The EHSS score of each EHS patient was calculated. The scores were grouped into 5-point intervals, and the total mortality rates of EHS patients corresponding to various EHSS score intervals were calculated.
First, the EHS patients were divided into death and survival groups according to prognosis. Measurement datasets with normal distribution are represented by means ± standard deviations (x ± s), while datasets without normal distribution are represented by medians (interquartile ranges). Count data are expressed as percentages. For the comparison of data between two groups, the univariate statistical method was used for measurement data, the two-independent-samples t-test was used for two sets of data with a normal distribution, and the rank sum test for two independent samples was used for data without a normal distribution. The χ2 test was used for count data. SPSS 17.0 was used for statistical analysis of the above types. Several parameters for some patients were not detected within 24 h after admission into the hospital or ICU, such as HCO3−, TnI, D-D, Cl− and Ca2+. Because these parameters had missing data during data collection, the mean imputation method for treating the missing data would, to a certain extent, affect the authenticity of the result; therefore, in the present study, we discarded the missing data and calculated the missing rates of various parameters. For the parameters in which the difference in the univariate analysis was statistically significant, the ROC curve was used, the AUC was calculated to screen EHSS parameters, and the 95% confidence interval was calculated. The AUCEHSS meant the area under the ROC curve of EHSS, AUCAPACHE II meant the area under the ROC curve of APACHE II, and AUCSOFA meant the area under the ROC curve of SOFA. MedCalc 15.8 was used for statistical analysis of the data. In this study, P < 0.05 was considered statistically significant.
Demographic characteristics and baseline clinical data
The demographics and baseline clinical data of the EHS patients for EHSS establishment and EHSS verification are shown in Table 2. The cooling effect and core body temperature dropped to 38.5 °C within 2 h after EHS onset. The APACHE II score of the nonsurvival group was significantly higher than that of the survival group, and the proportion of patients in the nonsurvival group who used vasoactive agents within 24 h after admission to the hospital or ICU was significantly higher than that of the survival group, suggesting that the condition of EHS patients in the death group within 24 h of admission to the hospital or ICU was more serious than that of the patients in the survival group. When comparing the baseline level of EHS patients for establishing EHSS to the baseline level of EHS patients for verifying the evaluation effectiveness of EHSS, there were no significant differences in various categories. Patients in the two categories had good homogeneity (as shown in Table 3).
Results of screening the scoring system parameters for exertional heat stroke
The univariate statistical analysis of identified EHSS parameters that were actually included in the statistical analysis is shown in Table 4. A total of 23 parameters with significant differences between the two groups were included in the EHSS for the next round of screening.
When the parameters with AUCs > 0.7 were taken as EHSS parameters, 14 were identified as EHSS parameters, as shown in Table 5. The AUCs of various parameters are shown in Fig. 1. When reviewing the establishment process of the classical scoring systems of APACHE II, sequential organ failure assessment (SOFA), MODS and SAPS II for critical and severe diseases, their included parameters were the most representative parameters capable of revealing injuries to various organs and systems [13,14,15,16]. When investigating the correlation between early coagulation function indicators and the prognosis of HS patients, previous clinical studies found that within 24 h of a patient’s admission to the hospital, the capability of predicting their prognosis by the three indicators platelet count (PLT), prothrombin time (PT) and activated partial thromboplastin time (APTT) decreased one by one , in contrast to this study. One clinical study on EHS showed that at the time the EHS patients were admitted to the hospital, 77% (24/31) showed increases in aspartate aminotransferase (AST), whereas the proportion of patients with increases in alanine aminotransferase (ALT) was only 39% (12/31) . In the present study, AUCAST > AUCALT, suggesting that during early EHS, the diagnosis of acute liver injury by AST yielded a better result than diagnosis by ALT. Because lactate dehydrogenase (LDH) is widely distributed in various tissues, it has poor specificity in diagnosing myocardial injury. In contrast, troponin I (TnI) is a specific indicator to reflect myocardial injury, and its time to reach the peak value within 24 h is earlier than that of troponin T (TnT) . Therefore, in this study, TnI was chosen as the indicator for myocardial injury. As seen from the statistical results in Table 5, according to AGI classification, in this study, there was a difference in gastrointestinal tract injuries between the two groups of EHS patients within 24 h after being admitted to the hospital or ICU. Through the series of screenings mentioned above, the present study eventually determined that the 12 parameters body temperature (T), GCS, pH, lactate (Lac), PLT, PT, fibrinogen (Fib), TnI, AST, total bilirubin (TBIL), creatinine (Cr) and gastrointestinal tract AGI could be classified as the parameters for EHSS.
Establishment of the scoring system for exertional heat stroke
The mortality rates corresponding to different values of various parameters are shown in Fig. 2. As can be noted from the statistical results in Table 4, according to AGI classification, among EHS patients, the difference in the number of patients with grade I gastrointestinal injuries and the numbers of patients with grades II, III and IV gastrointestinal injuries was statistically significant between the two groups, whereas the difference in the numbers of patients with grades II and III gastrointestinal injuries, the numbers of patients with grades II and IV gastrointestinal injuries and the numbers of patients with grades III and IV gastrointestinal injuries between the two groups were not statistically significant. This result means that the effect of grade I gastrointestinal injuries are different from the effects of grades II, III and IV gastrointestinal injuries on the prognosis of EHS patients, but the difference between grades II, III and IV gastrointestinal injuries seems nonexistent. Therefore, in the present study, AGI grade I was assigned one point, and AGI grades II, III and IV were assigned 3 points. The detailed scoring results are shown in Table 6. The highest score of this scoring system was 47 points. At this point, EHSS establishment was completed.
Evaluation effectiveness of EHSS on the prognosis of EHS patients
The AUC of each parameter for EHSS was calculated using the database to evaluate the effectiveness of EHSS (Fig. 3). APACHE II, SOFA and EHSS scores were calculated according to the abnormality levels of the 80 EHS patients; the scores of the survival and death groups from the two scoring systems are shown in Table 7. There was a significant difference in the two scoring systems between the survival and nonsurvival groups. The AUCEHSS was 0.97 (0.905–0.995), the AUCAPACHE II was 0.885 (0.794–0.945), the AUCSOFA was 0.886 (0.795–0.946), (AUCEHSS> AUCSOFA >AUCAPACHE II; AUCAPACHE II compared with AUCSOFA, P = 0.9725; AUCAPACHE II compared with AUCEHSS, P = 0.0194; AUCSOFA compared with AUCEHSS, P = 0.011, Fig. 4). The optimal cut-off point of the EHSS was 22 points; its corresponding sensitivity was 100%, and its specificity was 90.3%. When the EHSS score was no less than 22 points, the mortality rate of EHS patients was as high as 75%.
Correlation between different EHSS scores and EHS patient prognosis
The mortality rates of EHS patients corresponding to different EHSS score intervals are shown in Table 8. With increasing EHSS score, the mortality rate of EHS patients also increased; however, the mortality rates of the 26–30- and 31–35-point groups showed opposite behaviours. As shown in Table 8, when the EHSS was no larger than 20 points, the mortality rate of EHS patients was 0, and when the EHSS was greater than 35 points, the mortality rate of EHS patients was 100%.
EHS usually occurs in healthy, young people who are engaged in manual labor; even under many interventions, its incidence in athletes and soldiers is still increasing . Therefore, a full understanding of the etiology of EHS and the characteristics of the injuries of various organs, along with a timely and effective evaluation of the severity of the disease in EHS patients, could play an important role in improving the prognosis of patients and reducing the mortality rate, such as taking effective cooling measures or prompting military evacuation from the battlefield.
The scoring system for critically ill patients is an important method to quantitatively evaluate disease severity . The APACHE II scoring system is currently the most widely used and authoritative disease evaluation system for critical illness. This system was designed by the team of Professor Knaus at Washington University in 1985 and is composed of three parts: age, acute physiological score (APS) and chronic health score. The system collects the worst values of various parameters from patients within 24 h after their admission into the ICU to predict the mortality of critically ill patients . The higher the total theoretical score of this scoring system, the more severe the disease it indicates. In general, for common critical illnesses, particularly when the pathological and physiological characteristics of the disease are similar to the inclusion indicators of the APACHE II scoring system, the disease assessment and the prediction of prognosis are often quite accurate . However, studies have continuously confirmed that for some diseases with strong specialist features or some special groups of people, such as pregnant women and nonpregnant women in particular, with characteristic organ injuries or abnormal physiological indicators, all of the above scoring systems have certain flaws. As early as 2006, Professor Stevens et al.  used the APACHE III scoring system to retrospectively analyse obstetric patients admitted to the ICU and documented the patients’ demographic characteristics, obstetric and other disease histories and the 20 physiological variables contained in the APACHE III and reached the conclusion that APACHE III scoring had no correlation with the death of obstetric patients in the ICU. Similarly, Ryan et al.  conducted a meta-analysis on 25 studies regarding the prediction of the mortality rate of puerperae and critically ill puerperae by APACHE II scoring and found that APACHE II often overestimated the mortality rate of this population. Because EHS has unique pathogenic features of coagulation dysfunction and rhabdomyolysis and the above scoring systems do not include indicators for such injuries, the establishment of a disease evaluation system dedicated to EHS is urgently needed to evaluate this disease severity. When reviewing the establishment process of the previous scoring systems (APACHE II, SAPS II and MODS) for critical illness, it became apparent that their establishment was based on large samples [13, 15, 16]. However, for diseases with strong, specialized features, due to the limitations of the primary disease characteristics and case sources, the sample sizes of the above scoring systems, which were established based on relatively large case sample sizes, could not be reached. For example, the Ranson scale was created based on the data of 100 cases of acute severe pancreatitis by professor Ranson in 1974 , and it is still the primary criterion for determining the degree of injury of acute severe pancreatitis. EHS has a sporadic incidence, and it is difficult to obtain large sample sizes of cases. In the present study, a total of 170 cases of EHS were collected in the context of a long-term, multicenter study – one of the largest study sample sizes, either domestic or abroad.
Studies found that the primary thermal cytotoxic effect generated from high heat and the secondary activated systemic inflammatory response were the underlying causes of EHS with the complication of MODS; EHS patients also had manifestations of multiple-organ dysfunction or failure [29, 30]. For any severe disease, its physiological response will be expressed as changes in blood pressure, heart rate, respiratory rate and other basic vital signs . Therefore, in the present study, 42 parameters of body temperature, heart rate, blood pressure, respiratory rate, counts of white blood cells and neutrophils and indicators reflecting injuries to various organs were selected as the screening parameters for the establishment of the EHSS. Because the hospitals in China where the cases were collected were of different levels, during the collection of the cases, we found that some hospitals could not examine some of the parameters listed in Table 1, such as creatine kinase isoenzyme (CK-MB) or brain natriuretic peptide (BNP), which reflect myocardial injury; blood myoglobin (Mb) and urinary myoglobin (uMb), which reflect rhabdomyolysis; fibrinogen degradation products (FDP), which reflect coagulation dysfunction; and C-reactive protein (CRP), interleukin-6 (IL-6) and procalcitonin (PCT), which reflect the inflammatory response. When reviewing the establishment process of the APACHE II, MODS and other authoritative scoring systems for severe diseases [13, 16], without exception, they all noted that the parameters in a scoring system should have the following features: simple, common, easy to obtain and repeatedly detectable. At present, not all hospitals in China can list the eight indicators CK-MB, BNP, Mb, uMb, FDP, CRP, IL-6 and PCT as routine detection indicators. Therefore, the present study ultimately did not list the above eight indicators as the screening parameters for EHSS.
Through univariate analysis and AUC screening, after the initial EHSS parameters were identified, the determination of the final EHSS parameters and the assignment of weights to them were the core issues that needed to be addressed in this study. According to the parameter screening methods of the classical APACHE III, APACHE IV and SAPS II scoring systems for severe diseases, the ultimate determination of the parameters of the scoring system was always obtained from a multivariate logistic regression analysis, and a death risk prediction model was established based on the logistic regression equation. Through statistical analysis, 12 parameters were included in the EHSS to represent the characteristic injuries of EHS patients: high heat, CNS dysfunction and impairments in metabolic, coagulation, cardiac, liver, renal and gastrointestinal functions. These effects are consistent with the characteristics of multiple-organ dysfunction in EHS; therefore, the inclusion of the indicators is appropriate.
The mortality rates corresponding to various assignment intervals of the five parameters T, pH, Lac, AST and TBIL did not show a gradually increasing trend, and the mortality corresponding to the highest score (4 points) assignment interval was instead lower than that of the assignment interval of 3 points (Fig. 2). It is noteworthy that this phenomenon also existed in the establishment process of the MODS scoring system . This behaviour may be due to the small sample size in this study and the uneven distribution of sample size for certain parameters. Existing studies have shown that when the core body temperature exceeded 41.5 °C, the oxidative stress of HS rats was significantly increased, and endotoxin appeared in the portal venous system . When the core body temperature reached 42 °C, HS patients showed systemic endotoxemia . Thus, in the EHSS system established in this study, a body temperature of 41.6 °C–41.9 °C was given 3 points, while a body temperature of no less than 42 °C was given 4 points. For patients with metabolic acidosis, the disease is divided into mild, moderate and severe metabolic acidosis according to the pH value: mild disease had pH values of 7.30–7.36, the moderate disease had pH values of 7.20–7.29, and severe disease had pH values of less than 7.20. In EHSS, we assigned 4 points to pH values of less than 7.20. Numerous studies have shown that Lac values of more than 2 mmol/L are the dividing point of abnormal Lac [33, 34]; when Lac rose every 1.5 mmol/L, the corresponding mortality rate also gradually increased . Studies from Haas et al.  showed that when Lac was more than 10.0 mmol/L, the ICU mortality rate of patients was as high as 78.2%, and the hospitalization mortality rate was as high as 78.5%. In this study, when the Lac value was 8.1–10.0 mmol/L, the mortality rate of EHS patients was as high as 76.9%; even in the presence of errors, when the Lac value was more than 10.0 mmol/L, the mortality rate of EHS patients was also as high as 75%. Thus, the distribution of the Lac interval was reasonable in EHSS, which fully reflected the correlation between the Lac level in EHS patients and the prognosis. At present, there is no commonly accepted standard for using the degrees of increase in aminotransferases (ALT, AST) or bilirubin (TBIL, DBIL) to quantitatively evaluate the severity of acute liver injuries; therefore, combining the correlations between the existing data and the mortality rates of EHS patients, we set different assignment intervals for AST and TBIL.
In addition to the above parameters that were correlated with abnormal ICU mortality rates in the 4-point group, we also found that EHSS did not include parameters that reflected rhabdomyolysis. During the process of establishing a scoring model for rhabdomyolysis, McMahon et al.  found that creatine kinase (CK) would only affect patient prognosis when the level was more than 40,000 U/L. In the present study, the CK levels were 2852.0 (1441.6-5723.0) U/L in the survival group and 8850.0 (2327.7-35,237.0) U/L in the death group (Table 4). Therefore, while rhabdomyolysis may not be a determinant of the prognosis of EHS patients, its worth still needs to be confirmed by further studies with large sample sizes. Similarly, APTT, an indicator that reflects endogenous coagulation function, was also not included in this scoring system. Bouchama et al.  prospectively analysed the blood specimens of 22 HS patients at admission and found that the levels of endothelin, von Willebrand factor (vWF) and intercellular adhesion molecule-1 (ICAM-1) were significantly increased, suggesting that vascular endothelial cells were significantly impaired in patients at the early stage of HS pathogenesis. Injury to vascular endothelial cells mainly activates exogenous coagulation pathways; combined with the results of this study, it can be further speculated that during the early pathogenesis of EHS, exogenous coagulation dysfunction can predict prognosis more accurately. Previous studies showed that troponin I was a good predictor of myocardial injury in HS patients and the severity of HS itself, and when blood TnI was higher than 1.5 ng/ml in HS patients, it indicated the pathogenesis of severe myocardial injury . In the present study, when troponin I was higher than 1.5 ng/ml, the corresponding mortality rate was as high as 75%, consistent with the above results. Because gastrointestinal tract AGI classification is a categorical variable, it could not be screened with AUC. Animal studies confirmed that thermal attack could lead to early necrosis of intestinal epithelial cells  and enhanced intestinal permeability ; patients would mainly show the symptoms of nausea, vomiting, abdominal pain, diarrhea, watery stool and even gastrointestinal bleeding, which seriously affect patient prognosis . Combined with clinical practices, in the present study, AGI classification, which reflects the degree of gastrointestinal tract injury, was included in the EHSS. At this point, after a series of screenings of parameters, the first EHSS had been established.
In the present study, the APACHE II, SOFA and EHSS scores of 80 EHS patients were calculated, and the sizes of their AUCs were compared. The AUCAPACHE II was 0.885 (0.794–0.945) and the AUCSOFA was 0.886 (0.795–0.946), which had moderate accuracy in determining the prognosis of EHS patients. It is noteworthy that the AUCEHSS was 0.97 (0.905–0.995), indicating that its ability to determine the disease severity of EHS patients was significantly superior to that of the APACHE II and SOFA scores, and the diagnostic results were very good. This result preliminarily confirms that the EHSS, which was established in the first part of this study, has extremely high diagnostic value in judging the disease condition of EHS patients. Moreover, when the EHSS score was no less than 22 points, the risk of EHS patient death started to increase. With increasing EHSS score, the mortality rate of EHS patients also increased. However, the mortality rates of the 26–30- and 31–35-point groups showed opposite behaviours. This discrepancy might be due to the small sample size and uneven sample distribution.
This study still has some limitations. First, this study has a small sample size, which may mean that some parameters that affect the prognosis of EHS patients were not included in the EHSS; therefore, the EHSS still needs to be further confirmed with large samples. The evaluation effectiveness of the EHSS and the mortality rates corresponding to different EHSS score intervals still require further validation in large samples. Second, patients who were in the hospital or ICU for less than 24 h were excluded because we could not obtain the worst values of these patients over 24 h. Some were seemingly serious cases, and some may have been mild. Therefore, we did not see merit in scoring those who had short stays. Excluding patients with comorbidities before they suffered from EHS narrowed the population studied and the potential utility of the EHSS. Third, due to the restriction of sample size, this study could not use a multivariate logistic regression analysis method, meaning that the death risk prediction model could not be established. Fourth, the EHSS has not been verified in different populations, which has certain limitations.
A total of 12 parameters – T, GCS, pH, Lac, PLT, PT, Fib, TnI, AST, TBIL, Cr and gastrointestinal AGI classification – have been established as EHSS indicators. These parameters have better evaluation effectiveness for the prognosis of EHS patients; low (EHSS< 20) and high (EHSS> 35) scores showed 100% survival and 100% mortality, respectively. EHSS is the first scoring system dedicated to EHS patients, and this study is a pioneering one in the EHS field. The establishment of such a scoring system indicates that the quantification of EHS severity is truly realized, which will have significance in effectively determining disease severity among EHS patients and improving the treatment success rate in the future. However, the EHSS still requires further verification with large samples.
Availability of data and materials
We agree that the materials described in the manuscript, including all relevant raw data, will be freely available to any scientist wishing to use them for non-commercial purposes, without breaching participant confidentiality.
Acute Gastrointestinal Injury
Acute Kidney Injury
- APACHE II:
Acute Physiology and Chronic Health Evaluation II
Activated part of the Prothrombin Time
Areas Under the Receiver Operating Characteristic Curve
Brain Natriuretic Peptide
Blood Urea Nitrogen
- Ca2+ :
Classical Heat Stroke
Creatine Kinase Isoenzymes
- Cl− :
Disseminated Intravascular Coagulation
Exertional Heat Stroke
Exertional Heat Stroke Score
Fibrin degradation products
Glasgow Coma Scale
Intensive care unit
- K+ :
Mean Arterial Pressure
Multiple-organ dysfunction syndrome
- Na+ :
Receiver Operating Characteristic Curve
- SaO2 :
- SAPS II:
Simplified Acute Physiology Score II
Sequential Organ Failure Assessment
White Blood Cell count
Leon LR, Bouchama A. Heat stroke. Compr Physiol. 2015;5(2):611–47.
Inoue N, Sato A, Ikawa Y, Shimizu M, Okajima M, Taniguchi T, et al. Successful treatment of exertional heat stroke using continuous plasma diafiltration. J Clin Apher. 2016;31(5):490–2.
Fan H, Zhu JH, Ye JH. Cold hemodiafiltration for exertional heat stroke resulting in multiple organ failure. Ther Apher Dial. 2015;19(5):522.
Yeo TP. Heat stroke: a comprehensive review. AACN Clin Issues. 2004;15(2):280–93.
Goforth CW, Kazman JB. Exertional heat stroke in navy and marine personnel: a hot topic. Crit Care Nurse. 2015;35(1):52–9.
Keegan MT, Gajic O, Afessa B. Severity of illness scoring systems in the intensive care unit. Crit Care Med. 2011;39(1):163–9.
Vincent JL, Moreno R. Clinical review: scoring systems in the critically ill. Crit Care. 2010;14(2):207.
Ranson JH, Rifkind KM, Roses DF, Fink SD, Eng K, Localio SA. Objective early identification of severe acute pancreatitis. Am J Gastroenterol. 1974;61(6):443–51.
Hongjun K, Qing S, Yan Z, Liang P, Hui L, Feihu Z. Fluid resuscitation and standard drug treatment strategies in heatstroke Chinese patients. Drug Res (Stuttg). 2015;65(1):18–23.
Pease S, Bouadma L, Kermarrec N, Schortgen F, Regnier B, Wolff M. Early organ dysfunction course, cooling time and outcome in classic heatstroke. Intensive Care Med. 2009;35(8):1454–8.
Knapik JJ, Epstein Y. Exertional heat stroke: pathophysiology, epidemiology, diagnosis, treatment, and prevention. J Spec Oper Med. 2019;19(2):108–16.
King MA, Leon LR, Mustico DL, Haines JM, Clanton TL. Biomarkers of multiorgan injury in a preclinical model of exertional heat stroke. J Appl Physiol (1985). 2015;118(10):1207–20.
Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818–29.
Vincent JL, Moreno R, Takala J, Willatts S, De Mendonca A, Bruining H, et al. The SOFA (Sepsis-related organ failure assessment) score to describe organ dysfunction/failure. On behalf of the working group on Sepsis-related problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996;22(7):707–10.
Le Gall JR, Lemeshow S, Saulnier F. A new simplified acute physiology score (SAPS II) based on a European/north American multicenter study. JAMA. 1993;270(24):2957–63.
Marshall JC, Cook DJ, Christou NV, Bernard GR, Sprung CL, Sibbald WJ. Multiple organ dysfunction score: a reliable descriptor of a complex clinical outcome. Crit Care Med. 1995;23(10):1638–52.
Varghese GM, John G, Thomas K, Abraham OC, Mathai D. Predictors of multi-organ dysfunction in heatstroke. Emerg Med J. 2005;22(3):185–7.
Zeller L, Novack V, Barski L, Jotkowitz A, Almog Y. Exertional heatstroke: clinical characteristics, diagnostic and therapeutic considerations. Eur J Intern Med. 2011;22(3):296–9.
Del Bene VE. Temperature. In: Walker HK, Hall WD, Hurst JW, editors. Clinical methods: the history, physical, and laboratory examinations. Boston: Butterworth Publishers, a division of Reed Publishing; 1990.
Reintam Blaser A, Malbrain ML, Starkopf J, Fruhwald S, Jakob SM, De Waele J, et al. Gastrointestinal function in intensive care patients: terminology, definitions and management. Recommendations of the ESICM working group on abdominal problems. Intensive Care Med. 2012;38(3):384–94.
People’s Liberation Army Professional Committee of Critical Care Medicine Expert consensus on the diagnosis and treatment of heat stroke in China. Mil Med Res. 2020;7:1.
Devika S, Jeyaseelan L, Sebastian G. Analysis of GW.Get(window,"data") in logistic regression in medical research: a newer approach. J Postgrad Med. 2016;62(1):26–31.
Pan ZG, Shao Y, Liu YN, Gu ZT, Zhang XQ, Xu YQ, et al. Relationship between early coagulability parameters at admission and outcome in patients with severe heatstroke. Chin Crit Care Med. 2013;25(12):725–8.
Danese E, Montagnana M. An historical approach to the diagnostic biomarkers of acute coronary syndrome. Ann Transl Med. 2016;4(10):194.
Sekulic AD, Trpkovic SV, Pavlovic AP, Marinkovic OM, Ilic AN. Scoring systems in assessing survival of critically ill ICU patients. Med Sci Monit. 2015;21:2621–9.
Naqvi IH, Mahmood K, Ziaullaha S, Kashif SM, Sharif A. Better prognostic marker in ICU - APACHE II, SOFA or SAP II! Pak J Med Sci. 2016;32(5):1146–51.
Stevens TA, Carroll MA, Promecene PA, Seibel M, Monga M. Utility of acute physiology, age, and chronic health evaluation (APACHE III) score in maternal admissions to the intensive care unit. Am J Obstet Gynecol. 2006;194(5):e13–5.
Ryan HM, Sharma S, Magee LA, Ansermino JM, MacDonell K, Payne BA, et al. The usefulness of the APACHE II score in obstetric critical care: a structured review. J Obstet Gynaecol Can. 2016;38(10):909–18.
Trujillo MH, Bellorin-Font E, Fragachan CF, Perret-Gentil R. Multiple organ failure following near fatal exertional heat stroke. J Intensive Care Med. 2009;24(1):72–8.
Bouchama A, Knochel JP. Heat stroke. N Engl J Med. 2002;346(25):1978–88.
Hall DM, Buettner GR, Oberley LW, Xu L, Matthes RD, Gisolfi CV. Mechanisms of circulatory and intestinal barrier dysfunction during whole body hyperthermia. Am J Physiol Heart Circ Physiol. 2001;280(2):H509–21.
Bouchama A, Parhar RS, El-Yazigi A, Sheth K, Al-Sedairy S. Endotoxemia and release of tumor necrosis factor and interleukin 1 alpha in acute heatstroke. J Appl Physiol (1985). 1991;70(6):2640–4.
Liu V, Morehouse JW, Soule J, Whippy A, Escobar GJ. Fluid volume, lactate values, and mortality in sepsis patients with intermediate lactate values. Ann Am Thorac Soc. 2013;10(5):466–73.
Zhang Z, Chen K, Ni H, Fan H. Predictive value of lactate in unselected critically ill patients: an analysis using fractional polynomials. J Thorac Dis. 2014;6(7):995–1003.
Vincent JL, Quintairos ESA, Couto L Jr, Taccone FS. The value of blood lactate kinetics in critically ill patients: a systematic review. Crit Care. 2016;20(1):257.
Haas SA, Lange T, Saugel B, Petzoldt M, Fuhrmann V, Metschke M, et al. Severe hyperlactatemia, lactate clearance and mortality in unselected critically ill patients. Intensive Care Med. 2016;42(2):202–10.
McMahon GM, Zeng X, Waikar SS. A risk prediction score for kidney failure or mortality in rhabdomyolysis. JAMA Intern Med. 2013;173(19):1821–8.
Bouchama A, Hammami MM, Haq A, Jackson J. al-Sedairy S. evidence for endothelial cell activation/injury in heatstroke. Crit Care Med. 1996;24(7):1173–8.
Hausfater P, Doumenc B, Chopin S, Le Manach Y, Santin A, Dautheville S, et al. Elevation of cardiac troponin I during non-exertional heat-related illnesses in the context of a heatwave. Crit Care. 2010;14(3):R99.
Tang J, Jiang Y, Tang Y, Chen B, Sun X, Su L, et al. Effects of propofol on damage of rat intestinal epithelial cells induced by heat stress and lipopolysaccharides. Braz J Med Biol Res. 2013;46(6):507–12.
Wang YH, Liu TT, Kung WM, Chen CC, Wen YT, Lin IC, et al. Expression of aquaporins in intestine after heat stroke. Int J Clin Exp Pathol. 2015;8(8):8742–53.
The authors thank all research participants throughout the 52 included ICUs for their assistance.
This work was supported by the National Natural Science Foundation of China (81671966); the Beijing Natural Science Foundation (7182155); the Application Research and Achievement Extension of Clinical Characteristics in Chinese Capital Foundation (Z171100001017160); the Cultivation Program for Military Medical Science and Technology Youth-Growth Project (16QNP139); the Clinical Research Support Foundation of Chinese PLA General Hospital (2015FC-ZHCG-1002).
Ethics approval and consent to participate
This study was approved by the Institutional Review Board of Chinese People’s Liberation Army General Hospital, and all methods were in accordance with the Committee’s guidelines. We also obtained ethical approval from the ethics committee of each participating hospital. Since this study did not involve patient treatment, informed consent was not required.
Consent for publication
The authors declare that they have no competing interests.
About this article
Cite this article
Yang, MM., Wang, L., Zhang, Y. et al. Establishment and effectiveness evaluation of a scoring system for exertional heat stroke by retrospective analysis. Military Med Res 7, 40 (2020). https://doi.org/10.1186/s40779-020-00269-1
- Exertional heat stroke
- Scoring system
- Effectiveness of evaluation