Development of a nomogram to predict 30-day mortality of patients with sepsis-associated encephalopathy: a retrospective cohort study
Journal of Intensive Care volume 8, Article number: 45 (2020)
Sepsis-associated encephalopathy (SAE) is related to increased short-term mortality in patients with sepsis. We aim to establish a user-friendly nomogram for individual prediction of 30-day risk of mortality in patients with SAE.
Data were retrospectively retrieved from the Medical Information Mart for Intensive Care (MIMIC III) open source clinical database. SAE was defined by Glasgow Coma Score (GCS) < 15 or delirium at the presence of sepsis. Prediction model with a nomogram was constructed in the training set by logistic regression analysis and then undergone internal validation and sensitivity analysis.
SAE accounted for about 50% in patients with sepsis and was independently associated with the 30-day mortality of sepsis. Variables eligible for the nomogram included patient’s age and clinical parameters on the first day of ICU admission including the GCS score, lactate, bilirubin, red blood cell distribution width (RDW), mean value of respiratory rate and temperature, and the use of vasopressor. Compared with Sequential Organ Failure Assessment (SOFA) and Logistic Organ Dysfunction System (LODS), the nomogram exhibited better discrimination with an area under the receiver operating characteristic curve (AUROC) of 0.763 (95%CI 0.736–0.791, p < 0.001) and 0.753 (95%CI 0.713–0.794, p < 0.001) in the training and validation sets, respectively. The calibration plot revealed an adequate fit of the nomogram for predicting the risk of 30-day mortality in both sets. Regarding to clinical usefulness, the DCA of the nomogram exhibited greater net benefit than SOFA and LODS in both of the training and validation sets. Besides, the nomogram exhibited acceptable discrimination, calibration, and clinical usefulness in sensitivity analysis.
SAE is related to increased 30-day mortality of patients with sepsis. The nomogram presents excellent performance in predicting 30-day risk of mortality in SAE patients, which can be used to evaluate the prognosis of patients with SAE and may be more beneficial once specific treatments towards SAE are developed.
Sepsis-associated encephalopathy (SAE) is the dysfunction of brain that develops during the process of sepsis without evidence of the central nervous system (CNS) infection. It is tightly associated with long-term impairment of behavior, memory, and cognitive function, imposing heavy medical and financial burden on families and society [1,2,3]. More harmfully, patients with SAE tend to have higher short-term mortality than those with sepsis alone. A landmark study conducted by LA et al. demonstrated that encephalopathy is associated with increased hospital mortality from 16% when the Glasgow Coma Score (GCS) is 15 to 63% when GCS is between 3 and 8 . Similar conclusions were drew from another high-quality study performed by Sonneville et al., which showed decreased 30-day survival rate from 67% when GCS is 15 to 32% when GCS is between 3 and 8, and even mild change in consciousness (defined by GCS of 13–14) is an independent risk factor for the 30-day mortality with an hazard rate (HR) of 1.38 after adjusting for confounding factors . Except for these two high quality studies, several other researches also reported that SAE was responsible for increased short-term mortality, prolonged hospitalization time, or overmuch assumption of medical resources [6, 7]. Based on these, identifying SAE patients with high risk of short-term mortality is of great significance for that it may facilitates timely medical intervention and improves the prognosis of such patients. Therefore, the main objective of the present study by a large clinical database is to evaluate the impact of SAE on the 30-day mortality of patients with sepsis and then develop a predictive nomogram to individually predict the probability of 30-day death in SAE patients.
Material and methods
We conducted an observational study by retrieving data from the Medical Information Mart for Intensive Care (MIMIC III) open source clinical database, which contains de-identified health-related data of over forty thousand patients who received treatment in critical care units of the Beth Israel Deaconess Medical Center between June 2001 and October 2012 [8, 9]. The database is continuously updated, and the newest version (MIMIC-III v1.4) was released on 2 September 2016, which enhanced data quality and provided a large amount of additional data. We use the MIMIC-III v1.4 in our study, and all data in it was classified into 26 tables recording various individual information, such as demographic characteristics, treatment measures, nursing notes, and laboratory tests. Besides, it contains prognostic data obtained from the hospital and laboratory health record systems reporting the hospital mortality, or from the Social Security Administration Death Master File recording the out-of-hospital survival data. The MIMIC III database can be freely utilized after successful application and ethical approval from the Institutional Review Boards of both Beth Israel Deaconess Medical Center (Boston, MA, USA) and the Massachusetts Institute of Technology (Cambridge, MA, USA). Since all data are de-identified in this database to remove patients’ information, the requirement for individual patient consent is not indispensable.
Study population and data extraction
PgAdmin (version 4.1, Bedford, USA) was used to run structure query language (SQL) and then to extract data from the MIMIC III database. Six tables were occupied in our study, including DIAGNOSES_ICD, ICUSTAYS, PATIENTS, LABEVENTS, MICROBIOLOGYEVENTS, and PRESCRIPTIONS. We included adult patients (> 17 years of age) with a diagnosis of sepsis according to the Third International Consensus Definitions for Sepsis (Sepsis-3): (1) Patients with infection confirmed by the positive results of microbial cultivation and (2) the Sequential Organ Failure Assessment (SOFA) score ≥ 2 . Excluded were patients (1) with primary brain injury (traumatic brain injury, ischemic stroke, hemorrhagic stroke, epilepsy, or intracranial infection); (2) with pre-existing liver or kidney failure affecting consciousness; (3) with severe burn and trauma; (4) receiving cardiac resuscitation recently; (5) with chronic alcohol or drug abuse; (6 )with severe electrolyte imbalances or blood glucose disturbances, including hyponatremia (< 120 mmol/l), hyperglycemia (> 180 mg/dl), or hypoglycemia (< 54 mg/dl); (7) dying or leaving within 24 h since ICU admission; and (8) without an evaluation of GCS. Eligible patients were included into the final cohort for investigation. For the final cohort, we retrospectively collected the following data: (1) demographic data including age, gender, and ethnicity; (2) 30-day mortality, in which no patient was lost to follow-up during 30 days, and patients live longer than 30 days were recorded as survival; (3) comorbidity as coded and defined in the International Classification of Diseases, Ninth Revision (ICD-9); (4) mean value of vital signs during the first 24 h of ICU stay; (5) the first laboratory data since ICU admission; and (6) site of infection and type of micro-organism. The severity of illness and organ failure was assessed by modified forms of the simplified acute physiology score (SAPSII) and sepsis-related organ failure assessment (SOFA), respectively, on the first day of ICU admission [11, 12]. The modified forms are SAPSIIand SOFA excluding the component of central nervous system. Besides, we created another dataset for sensitivity analysis based on the Martin’s criteria, a widely used approach for identifying sepsis in administrative health data  (Additional file 1: Figure S1).
We defined SAE in the study as sepsis accompanied by GCS ≤ 14 on the first day of ICU admission or delirium according to the ICD-9 code (2930, 2931). The delirium caused by alcohol or drug abuse, dimension, mental disorders, and neurological diseases was excluded from the definition of SAE. GCS was confirmed as an excellent tool for characterizing SAE and distinguishing it from sepsis  and for neurological evaluation of critically ill patients . For sedated or postoperative patients, we adopted GCS measured before sedation or surgery.
Shapiro-Wilk tests were used to assess the distribution of variables. Data were expressed as mean ± standard deviation (SD) for parametric continuous data and as median (interquartile ranges) for non-parametric distribution. Categorical data was expressed as number (percentages). Parametric continuous variables were compared by using unpaired Student’ t test and non-parametric continuous variables by Mann–Whitney U test. The chi-squared test was adopted to assess the differences in categorical variables between groups.
Logistic regression analysis was used to identify risk factors independently associated with the 30-day mortality of sepsis. Specifically, variables related to 30-day death in univariate analysis (p < 0.1) were entered into multivariate logistic regression analysis to calculate estimated odds ratios (OR) and 95% confidence intervals (95%CI), where significant level for independent risk factors was p < 0.05. Collinearity between continuous variables was tested by the variance inflation factor (VIF), and an arithmetic square root of VIF ≤ 2 was considered as non-collinearity. Kaplan–Meier analysis was conducted to visualize the probability of 30-day survival between SAE and non-SAE cohorts, and log-rank test was used to identify between-group difference. Besides, propensity score match (PSM) was conducted between the SAE and non-SAE cohorts, and then the 30-day survival was visualized by Kaplan–Meier analysis and compared by log-rank test.
In the process of nomogram development, patients with SAE were randomly distributed into a training set and a validation set without replacement at a ratio of 7:3. Logistic regression analysis following the steps mentioned above was conducted to identify independent risk factors for the 30-day mortality of SAE. Then, a nomogram in predicting the probability of 30-day death was obtained by the training set according to Occam’s Law of Razor, namely, the best model should be one that can achieve the aim of study with fewer variables . The performance of the nomogram was evaluated and compared with SOFA and Logistic Organ Dysfunction System (LODS)  in both of the training and validation sets by an area under the curve of the receiver operating characteristic (AUROC) and by calibration with bootstrap method with 1000 resampling. Besides, integrated discrimination improvement (IDI) was calculated to compare discrimination slopes and Brier score to evaluate model fitness . DCA analysis was performed to evaluate the net benefit of medical intervention conforming nomogram, SOFA, and LODS at different threshold probabilities in the training and validation sets. Sensitivity analyses were conducted in two cohorts, namely, sensitivity-1 in the cohort diagnosed by Martin’s criteria and sensitivity-2 in a subset (GCS 3-8) of the cohort diagnosed by Sepsis-3.
Statistical analyses were performed using the R software (version 3.6.1, R Foundation for Statistical Computing, Vienna, Austria). Missing values were addressed with multiple imputation in the process of logistic regression and model construction. The imputation technique involves creating multiple copies of the data and replacing missing values with imputed values through a suitable random sample from their predicted distribution. We used the “mice” package of R to obtain 5 imputation datasets. A two-tailed p value of < 0.05 was considered statistically significant. All analyses were reported according to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) guidelines .
Characteristics of participants with sepsis
After screening by the inclusion and exclusion criteria, a total of 4987 patients were included into the final cohort, and SAE was observed in 2474 (49.6%) patients. Characteristics at baseline and upon ICU admission of all participants and participants in SAE and non-SAE groups were exhibited in Tables 1 and 2. Patients with SAE were older than those without it, with a median age of 73 [58, 83] in SAE patients and 68 [55, 79] in non-SAE ones. Patients with SAE were more likely to admit into the medical intensive care unit (MICU) and suffer from hypertension, anemias, and history of neurological diseases, which mainly included Parkinson’s disease, Huntington’s chorea, and leukodystrophy. Besides, patients with SAE had lower level of partial pressure of blood oxygen (PO2), bilirubin, lactate, prothrombin time (PT), and higher level of serium PH value on the first day of ICU admission. The 30-day mortality of SAE and non-SAE cohorts was 527 (21.30%) and 449 (17.87%), respectively, with a statistical difference of p = 0.003. Interestingly, except for CNS, the severity of illness and organ failure was more serious in non-SAE patients than that in SAE ones, with p < 0.001 in both of the modified SAPSIIand SOFA. Besides, patients without SAE exhibited higher frequent use of vasopressor and longer hospital and ICU stay time in our study.
SAE is independently associated with the 30-day mortality of patients with sepsis
After adjusting for baseline characteristics, vital signs, laboratory tests, infection site, and microorganisms, the results of multivariate logistic regression revealed that SAE was an independent risk factor for the 30-day mortality of patients with sepsis [adjusted odd ratio (aOR) = 1.26, 95% CI 1.07–1.49, p = 0.005] (Additional file 2:Table S1). Kaplan–Meier’s survival estimates of patients according to the presence or absence of SAE at ICU admission were presented in Additional file 3: Figure S2. After conducting propensity score match (PSM) between SAE and non-SAE groups according to the differences in baseline characteristics and characteristics at ICU admission (Additional file 4: Figure S3), Kaplan–Meier’s survival analysis was conducted, and results were presented in Additional file 5: Figure S4. Results showed that no matter performing PSM or not, significant differences were observed in 30-day survival between SAE and non-SAE patients, with log-rank p = 0.0018 before PSM and log-rank p < 0.0001 after PSM.
Development of a prediction nomogram in the training set
Characteristics at baseline and upon ICU admission of SAE patients and participants in the training set and validation set were exhibited in Additional file 6: Table S2 and Additional file 7: Table S3. Results revealed that both sets had no statistic difference in all the variables.
The risk factors related to 30-day mortality of SAE identified by multivariable logistic regression were shown in Table 3. The VIF was calculated, and no continuous variables in Table 3 had an arithmetic square root of VIF ≤ 2. Furthermore, the correlation between continuous variables was visualized in Additional file 8: Figure S5, and linear correlation was not observed, indicating that collinearity was not existed in the regression model. Lung infection and catheter-related infection were excluded from model development since these two parameters depended on microbial culture, which is a time-consuming process. The performances of the remaining risk factors in Table 3 were then comprehensively evaluated and finally, according to the Occam’s Law of Razor, a model integrating age, lactate, bilirubin, RDW, mean value of respiratory rate and temperature, and the use of vasopressor was established for its similar discrimination compared with the model including all the risk factors (combined) in both of the training and validation sets (Fig. 1). Based on this model, a nomogram was plotted to predict the probability of 30-day death of SAE (Fig. 2).
Validation of the prediction nomogram
We compared the nomogram with SOFA and LODS for predicting 30-day risk of mortality in SAE patients, and results were shown in Table 4. Results showed that the AUROC of the nomogram was significantly higher than that of SOFA and LODS in both of the training and validation sets, indicating that the predictive nomogram had better discrimination than SOFA and LODS in predicting the 30-day mortality of patients with SAE.
Calibration curves were depicted for both of the training and validation sets, and the bias-corrected line is formed using a bootstrap method. In both sets, the apparent curve and bias-corrected curve slightly deviated from reference line, but a good conformity between observation and prediction is still observed (Fig. 3). The Brier score of the nomogram was 0.139 (95%CI 0.129–0.150) in the training set and 0.144 (95%CI 0.128–0.159) in the validation set, which were higher than that of SOFA and LODS in both sets, indicating that the nomogram had better calibration of prediction than SOFA and LODS. Moreover, the IDI of the nomogram was significantly higher than that of SOFA and LODS in both sets, revealing that the nomogram could increase the prediction probability of SOFA and LODS in the two sets.
Clinical use of the nomogram
With regard to clinical use, the DCA for nomogram was depicted and compared with SOFA and LODS. In the training set, medical intervention guided by the nomogram could add more net benefit than SOFA and LODS when the threshold probability (PT) > 0.1 (Fig. 4a). In the validation set, treatment directed by nomogram could gain more net benefit than SOFA and LODS when PT was between 0.1 and 0.6 (Fig. 4b).
We conducted sensitivity analysis by including patients with SAE diagnosed by the Martin’s criteria (sensitivity-1) or patients with GCS of 3-8 in the dataset extracted by Sepsis-3 (sensitivity-2). The discrimination, calibration, and clinical usefulness of the nomogram were compared with SOFA and LODS by using AUROC, calibration curve, DCA curve, Brier index, and IDI (Additional file 9: Figure S6; Additional file 10: Figure S7; Additional file 11: Table S4). Results showed that the prediction nomogram had better discrimination and calibration as well as more clinical net benefit than SOFA and LODS in both of the sensitivity analyses, which further consolidated the stable and excellent predictive performance of the nomogram.
In this retrospective analysis by the MIMIC III database, we conducted logistic regression to recognize the risk factors related to the 30-day mortality of patients with SAE and predictors including age, lactate, bilirubin, RDW, mean value of respiratory rate and temperature, and the use of vasopressor that were identified and integrated into a best-fit prediction model visualized as a prediction nomogram. To the best of our knowledge, this is the first study to evaluate the independent risk factors associated with the 30-day mortality of SAE and develop a nomogram to predict it.
EW et al. find that 20–50% of patients with sepsis have delirium or impairment of consciousness, indicating that SAE is highly prevalent in the ICU and regraded to be the most common encephalopathy in the surgical intensive care unit . CL et al. found that patients with acutely altered mental status associated with sepsis have higher mortality rates (49%) than septic patients with pre-existing mental status changes (41%) or normal mental status (26%) . Since then, several other studies conducted in different study cohorts further supported their conclusion that SAE is closely related to the increased short-term mortality of patients with sepsis [4, 5, 20]. Based on these, two limitations seem obvious and need to be solved urgently. One is that treatment towards SAE is still challenging by the fact that the widely used guidelines of sepsis listed a variety of evidence-based recommendations for the treatment of sepsis, but none for SAE [21,22,23]. This predicament may reveal another limitation that clinicians do not pay enough attention to SAE, which can be reflected by the fact that they seem to be overly optimistic about prognosis and the therapeutic impact of treatment in patients with sepsis [24,25,26,27,28]. Therefore, it is important for clinicians to comprehensively evaluate the true risks of mortality and objectively estimate the risks/benefits of medical intervention in patients with sepsis to allow clinicians, patients, and their families the ability to carefully evaluate the impact of potential treatment options, help them to make medical decision together, and prevent medical disputes. The prediction nomograms is therefore essential to an improved risk stratification process of sepsis and then useful to a clear statement of the condition by clinicians to the families of patients with SAE. Further studies should focus on the pathogenesis of SAE and the development of specific treatment to it, which may further enhance the clinical usefulness of the nomogram by reducing the mortality of SAE. As assessment with a nomogram may be time consuming and complicated to use in clinical practice, developing a software which can be embedded into the electronic medical system is our next work, which can guide clinicians for the timely treatment of SAE and reduce the mortality of patients without increasing the working time and burden of clinicians.
SOFA and LODS had been demonstrated to be useful tools in predicting the short-term mortality of patients with sepsis [10, 29], but whether they are applicable to SAE is still unclear. Thus, we developed the current predictive model in the datasets extracted by Sepsis-3 and compared its predictive performance with SOFA and LODS in both of the datasets extracted by Sepsis-3 and by the Martin’s criteria. Results showed that SOFA and LODS in discriminating SAE patients under the risk of 30-day death were not so good as the results reported in patients with sepsis in the previous studies, whereas the prediction nomogram could improve the predictive performance of SOFA and LODS and exhibited acceptable discrimination and calibration. Besides, to validate its clinical usefulness, decision curve analysis was employed to account for both the benefits and the costs of medical intervention to SAE patient guided by the nomogram. Results revealed that interventions guided by the current nomogram can add more net benefits than SOFA and LODS.
RDW accounted for the biggest weight in the nomogram, indicating that it is the most important predictor and has the strongest power to predict 30-day mortality of SAE patients. RDW is a measure of the size of circulating erythrocytes and was routinely used in the differential diagnosis of anemia. However, studies have revealed that it is also useful in estimating the short-term mortality of non-hematologic diseases, such as cardiovascular diseases [30, 31], stroke , liver diseases [33, 34], and critical illness . Consistently, our study demonstrated that RDW is an independent risk factor and potent predictor for the 30-day mortality of SAE. The relationship between RDW and the 30-day mortality of SAE was shown in Additional file 12: Figure S8, indicating that the 30-day mortality of SAE was positively associated with the level of RDW. Mechanisms under the relationship between RDW and the 30-day mortality of SAE remain largely unknown, but several studies had revealed that the inflammatory response during sepsis may contribute to the adverse impact of RDW on the prognosis of SAE as RDW is positively associated with inflammatory markers, such as C-reactive protein (CRP) and erythrocyte sedimentation rate (ESR) [36,37,38,39]. Besides, oxidative stress may be another reason to connect RDW with poor 30-day outcome because studies indicated that oxidative stress can increase anisocytosis by disrupting erythropoiesis and altering the circulating half-life of red blood cell, ultimately leading to increased level of RDW [40, 41].
The diagnosis of sepsis remains controversial for that the pathophysiology of sepsis is still not fully understood. Both of Sepsis-3 and the systemic inflammatory response syndrome (SIRS) definitions are currently used in clinical practice to diagnose patients with sepsis [10, 12], but each has its own defects. For Sepsis-3, the parameters needed in SOFA are burdensome to collect and not usually available at the bedside to help with clinical decision-making. qSOFA is simpler, but it was only validated outside the ICU and not suitable to patients which already admitted to an ICU. Besides, Sepsis-3 relies on the clinician’s experience to identify suspected infection, which may not be apparent early on, making it less sensitive than SIRS for diagnosing early sepsis . SIRS definition based on SIRS to diagnose sepsis, but changes in white blood cell count, temperature, and heart rate only reflect inflammation, the more dangerous host response to infection cannot be well indicated by the criteria of SIRS, and studies has demonstrated that SIRS criteria are present in many hospitalized patients, including those who never develop infection and never incur adverse outcomes . Therefore, we established two datasets in this study by extracting data from patients with sepsis diagnosed by the Sepsis-3 criteria (dataset-1) and Martin’s criteria (dataset-2), respectively. Then, we developed the prediction nomogram in dataset-1 and conducted sensitivity analysis in dataset-2. Results showed that the nomogram exhibited stable and acceptable predictive performance in both of the two datasets. However, we are still cautious about the prediction efficiency of the nomogram in the condition of Sepsis-3. In Sepsis-3 [10, 29], infection is identified as patients who had body fluids sampled for culture and received antibiotics, which required the combination of culture and antibiotic start time to occur within a specific time epoch, namely, if the antibiotic was given first, the culture sampling must have been obtained within 24 h. If the culture sampling was first, the antibiotic must have been ordered within 72 h, but it is difficult to extract data like this in our study as culture sampling time is lacking in the MIMIC III database. Consequently, we just include patients with infection confirmed by the positive results of microbial cultivation, which may result in the fact that the dataset of our study is smaller than that extracted by the definition of Sepsis-3. Therefore, further studies based on our own data strictly extracted by Sepsis-3 should be performed to further validate the predictive performance of the nomogram in the condition of Sepsis-3.
Two points should be noted when using the nomogram. First, as vital signs in our study are the mean values of the first 24 h of each ICU patient, and the nomogram is not applicable to patients dying or leaving within 24 h since ICU admission. Second, laboratory tests in the nomogram are the first results since ICU admission; therefore, all the laboratory tests included in the nomogram should be completed within the first 24 h since ICU admission.
This study has some limitations: First, one of the challenges in studying SAE is that without specific diagnostic method, it remains a rule-out definition, which may lead to a high specificity, but relatively low sensitivity for the diagnosis of SAE. Thus, the current nomogram can only be used in SAE diagnosed by exclusion and may require further modification once specific diagnostic methods are developed. Second, the retrospective nature of this observational study determined that unidentified confounding factors may affect the results if adding to the model. Third, studies based on the results of brain MRI have revealed that the impairment of cerebral white matter in patients with critical illness are not only related to sequelae of the central nervous system but also associated with increased mortality , but neuroimaging data was not included in the study, making it impossible to assess the impact of organic lesion of brain on outcome of SAE. Finally, as data in the MIMICIII database is slightly old and we only conducted an internal validation by this database, external validation based on our own data should be performed in the future study to further validate the robustness and performance of the nomogram.
A prediction nomogram based on patients’ age, together with the GCS score, lactate, bilirubin, RDW, mean value of respiratory rate and temperature, and the use of vasopressor on the first day of ICU admission can be conveniently used to serve accurate prognostic prediction in the 30-day mortality of SAE. This may be particularly beneficial in preventing the deterioration of SAE once specific treatments towards encephalopathy are developed and finally improve the prognosis of SAE patients.
Availability of data and materials
The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.
- MIMIC III:
Medical Information Mart for Intensive Care
Glasgow Coma Score
Decision curve analysis
Blood urea nitrogen
Red blood cell distribution width
The simplified acute physiology score
Sequential Organ Failure Assessment
Logistic Organ Dysfunction System
Area under the receiver operating characteristic curve
Central nervous system
International Classification of Diseases, Ninth Revision
Structure query language
Confusion Assessment Method developed for ICU patients
Variance inflation factor
Integrated discrimination improvement
Coronary care unit
Cardiac surgical intensive care unit
Medical intensive care unit
Surgical intensive care unit
Trauma/surgical intensive care unit
Red blood cell distribution width
Propensity score match
Erythrocyte sedimentation rate
Quick Sequential Organ Failure Assessment
Systemic inflammatory response syndrome
Ebersoldt M, Sharshar T, Annane D. Sepsis associated delirium. Intensive Care Med. 2007;33:941–50.
Iwashyna TJ, Ely EW, Smith DM, Langa KM. Long-term cognitive impairment and functional disability among survivors of severe sepsis. JAMA. 2010;304(16):1787–94.
Widmann CN, Heneka MT. Long-term cerebral consequences of sepsis. Lancet Neurol. 2014;13(6):630–6.
Eidelman LA, Putterman D, Putterman C, Sprung CL. The spectrum of septic encephalopathy. Definitions, etiologies, and mortalities. JAMA. 1996;275:470–3.
Sonneville R, de Montmollin E, Poujade J, et al. Potentially modifiable factors contributing to sepsis-associated encephalopathy. Intensive Care Med. 2017;43:1075–84.
Sprung CL, Peduzzi PN, Shatney CH, et al. Impact of encephalopathy on mortality in the sepsis syndrome. Crit Care Med. 1990;18:801–6.
Bleck TP, Smith MC, Pierre-Louis SJC, et al. Neurologic complications of critical medical illnesses. Crit Care Med. 1993;21:98–103.
Johnson A, Pollard T, Mark R. MIMIC-III Clinical Database. Physio Net. 2016. https://doi.org/10.13026/C2XW26.
Johnson AEW, Pollard TJ, Shen L, et al. MIMIC-III, a freely accessible critical care database. 2016;3:160035.
Singer M, Deutschman CS, Seymour CW, et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8).
Le Gall JR, Lemeshow S, Saulnier F. A New Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA. 1993;270(24):2957–63.
Ferreira FL, Bota DP, Bross A, Mélot C, Vincent JL. Serial evaluation of the SOFA score to predict outcome in critically ill patients. JAMA. 2001;286:1754–8.
Martin GS, Mannino DM, Eaton S, Moss M. The epidemiology of sepsis in the United States from 1979 through 2000. N Engl J Med. 2003;348:1546–54.
Sharshar T, Citerio G, Andrews PJ, et al. Neurological examination of critically ill patients: a pragmatic approach. Report of an ESICM expert panel. Intensive Care Med. 2014;40(4):484–95.
Van Den Berg HA. Occam’s razor: from Ockham’s via moderna to modern data science. Sci Prog. 2018;101(3):261–72.
Le Gall JR, Klar J, Lemeshow S, Saulnier F, Alberti C, Artigas A, Teres D. The Logistic organ dysfunction System. a new way to assess organ dysfunction in the intensive care unit. ICU Scoring Group. JAMA. 1996;276(10):802–10.
Pencina MJ, D'Agostino RB, D'Agostino RB, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157–72.
Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD). Ann Intern Med. 2015;162:735–6.
Ely EW, Shintani A, Truman B, et al. Delirium as a predictor of mortality in mechanically ventilated patients in the intensive care unit. JAMA. 2004;291:1753–62.
Polito A, Eischwald F, Maho ALL, et al. Pattern of brain injury in the acute setting of human septic shock. Crit Care. 2013;17:R204.
Peake SL, Delaney A, Bailey M, et al. Goal-directed resuscitation for patients with early septic shock. N Engl J Med. 2014;371(16):1496–506.
Rhodes A, Evans LE, Alhazzani W, et al. Surviving sepsis campaign: international guidelines for management of sepsis and septic shock: 2016. Intensive Care Med. 2017;43(3):304–77.
Nishida O, Ogura H, Egi M, et al. The Japanese clinical practice guidelines for management of sepsis and septic shock 2016 (J-SSCG 2016). J Intensive Care. 2018;6:7.
From the Centers for Disease Control. Increase in national hospital discharge survey rates for septicemia--United States, 1979–1987. JAMA. 1990;263(7):937–8.
Stevenson EK, Rubenstein AR, Radin GT, Wiener RS, Walkey AJ. Two decades of mortality trends among patients with severe sepsis: a comparative meta-analysis. Crit Care Med. 2014;42(3):625–31.
Dombrovskiy VY, Martin AA, Sunderram J, Paz HL. Facing the challenge: decreasing case fatality rates in severe sepsis despite increasing hospitalizations. Crit Care Med. 2005;33(11):2555–62.
Brun-Buisson C, Meshaka P, Pinton P, Vallet B. EPISEPSIS: a reappraisal of the epidemiology and outcome of severe sepsis in French intensive care units. Intensive Care Med. 2004;30(4):580–8.
Engel C, Brunkhorst FM, Bone HG, Brunkhorst R, Gerlach H, Grond S, et al. Epidemiology of sepsis in Germany: results from a national prospective multicenter study. Intensive Care Med. 2007;33(4):606–18.
Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315:762–74.
Turcato G, Serafini V, Dilda A, et al. Red blood cell distribution width independently predicts medium-term mortality and major adverse cardiac events after an acute coronary syndrome. Ann Transl Med. 2016;4:254.
Sangoi MB, Da Silva SH, da Silva JE, et al. Relation between red blood cell distribution width and mortality after acute myocardial infarction. Int J Cardiol. 2011;146(2):278–80.
Ani C, Ovbiagele B. Elevated red blood cell distribution width predicts mortality in persons with known stroke. J Neurol Sci. 2009;277(1-2):103–8.
Hu Z, Sun Y, Wang Q, et al. Red blood cell distribution width is a potential prognostic index for liver disease. Clin Chem Lab Med. 2013;51:1403–8.
Xu WS, Qiu XM, Ou QS, et al. Red blood cell distribution width levels correlate with liver fibrosis and inflammation: a noninvasive serum marker panel to predict the severity of fibrosis and inflammation in patients with hepatitis B. Medicine (Baltimore). 2015;94:e612.
Salciccioli JD, Marshall DC, Pimentel MA, et al. The association between the neutrophil-to-lymphocyte ratio and mortality in critical illness: an observational cohort study. Crit Care. 2015;19:13.
Lippi G, Targher G, Montagnana M, Salvagno GL, Zoppini G, Guidi GC. Relation between red blood cell distribution width and inflammatory biomarkers in a large cohort of unselected outpatients. Arch Pathol Lab Med. 2009;133(4):628–32.
Hu ZD, Sun Y, Guo J, et al. Red blood cell distribution width and neutrophil/lymphocyte ratio are positively correlated with disease activity in primary Sjogren's syndrome. Clin Biochem. 2014;47(18):287–90.
Hu ZD, Chen Y, Zhang L, et al. Red blood cell distribution width is a potential index to assess the disease activity of systemic lupus erythematosus. Clin Chim Acta. 2013;425:202–5.
Vaya A, Sarnago A, Fuster O, Alis R, Romagnoli M. Influence of inflammatory and lipidic parameters on red blood cell distribution width in a healthy population. Clin Hemorheol Microcirc. 2015;59(4):379–85.
Perlstein TS, Weuve J, Pfeffer MA, Beckman JA. Red blood cell distribution width and mortality risk in a community-based prospective cohort. Arch Intern Med. 2009;169:588–94.
Weiss G, Goodnough LT. Anemia of chronic disease. N Engl J Med. 2005;352(10):1011–23.
Levy MM, Fink MP, Marshall JC, Abraham E, Angus D, Cook D, Cohen J, Opal SM, Vincent JL, Ramsay G. 2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference. Crit Care Med. 2003;31(4):1250–6.
Williams JM, Greenslade JH, McKenzie JV, Chu K, Brown AFT, Lipman J. Systemic inflammatory response syndrome, Quick Sequential Organ Function Assessment, and organ dysfunction: insights from a prospective database of ED patients with infection. Chest. 2017;151(3):586–96.
Sedaghat S, Cremers LG, de Groot M, et al. Lower microstructural integrity of brain white matter is related to higher mortality. Neurology. 2016;87(9):927–34.
Ethics approval and consent to participate
The MIMIC III database has received ethical approval from the Institutional Review Boards of both Beth Israel Deaconess Medical Center (Boston, MA, USA) and the Massachusetts Institute of Technology (Cambridge, MA, USA). All data are de-identified in this database to remove patients’ information, and the requirement for individual patient consent is not indispensable.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Flowchart of data extraction. Patients with sepsis were extracted from the MIMIC III database by both of Sepsis-3 and the Martin’s criteria. Then, we excluded patients with comorbidities that may have adverse impact on consciousness, or without a record of GCS, or died within 24 hours since ICU admission. The order of exclusion was consistent with what we performed by the SQL. After exclusion , patients remained in the “ Sepsis-3” cohort (blue) were picked out to make up the final cohort and those with GCS 3-8 were used to develop the nomogram and conducted sensitivity analysis, respectively. Besides, patients with SAE in the “Martin’s criteria” cohort (red) were picked out to conduct sensitivity analysis.
Factors independently associated with 30-day mortality of patients with sepsis in the multivariate logistic analysis.
The Kaplan-Meier’s survival estimated of the 30-day survival probability of SAE and non-SAE patients. Results showed that the 30-day survival of SAE patients were significantly lower than that of non-SAE patients (Log-rank p = 0.0018).
Propensity score match (PSM) between SAE and non-SAE patients. The statistically different variables in baseline characteristics and characteristics at ICU admission of patients with sepsis were exhibited as the red dot (unmatched) and all of them were matched well and similar between the two groups (green dot: matched) after PSM.
The Kaplan-Meier’s survival estimated of the 30-day survival probability of SAE and non-SAE patients after PSM. Results showed that the 30-day survival of SAE patients were significantly lower than that of non-SAE patients (Log-rank p < 0.0001).
Baseline characteristics of patients in the training and validation setsa.
Characteristics at ICU admission in the training and validation setsa
The correlation between continuous variables which were associated with the 30-day mortality of SAE patients in the multivariable logistic regression. The figure exhibited that no linear correlation was existed among the continuous variables, indicating that collinearity was not existed in the regression model.
Sensitivity analysis conducted in patients in the “Martin’s criteria” cohort. In the ROC curve, the AUROC of nomogram was significantly higher than that of SOFA and LODS (A). In the calibration curve, the apparent curve and bias-corrected curve were slightly deviated from reference line, but a good conformity between observation and prediction is observed (B). In the DCA curve, medical intervention guided by the nomogram could add more net benefit than SOFA and LODS when the threshold probability (PT) between 0.1 and 0.65.
Sensitivity analysis conducted in patients with GCS3-8 in the “Sepsis-3” cohort. In the ROC curve, the AUROC of nomogram was higher than that of SOFA and LODS (A). In the calibration curve, the apparent curve and bias-corrected curve were slightly deviated from reference line, but a good conformity between observation and prediction is still observed (B). In the DCA curve, medical intervention guided by the nomogram could add more net benefit than SOFA and LODS when the threshold probability (PT) between 0.1 and 0.6.
Comparison of models in predicting the 30-day mortality of SAEa (Sensitivity analysis).
The relationship between the 30-day mortality of SAE and the level of RDW. (A) The level of RDW in patients with SAE who died or survived within 30-day since ICU admission. (B) The Kaplan-Meier’s survival estimated of the 30-day survival probability of SAE patients who were divided into two groups based on the upper limit of reference interval.
About this article
Cite this article
Yang, Y., Liang, S., Geng, J. et al. Development of a nomogram to predict 30-day mortality of patients with sepsis-associated encephalopathy: a retrospective cohort study. j intensive care 8, 45 (2020). https://doi.org/10.1186/s40560-020-00459-y
- Sepsis-associated encephalopathy
- 30-day mortality