The prognostic performance of qSOFA for community-acquired pneumonia

Abstract

Background

Quick Sepsis-related Organ Failure Assessment (qSOFA) is a new screening system for sepsis. The prognostic performance of qSOFA for patients with suspected infections outside the intensive care unit (ICU) is similar to that of full SOFA; however, its performance for community-acquired pneumonia (CAP) has not yet been evaluated in detail.

The objectives of the present study were to compare the prognostic performance of qSOFA with existing pneumonia severity scores, such as CURB-65 (confusion, blood urea nitrogen > 19 mg/dL, respiratory rate ≥ 30/min, systolic blood pressure < 90 mmHg, or diastolic blood pressure ≤ 60 mmHg, age ≥ 65 years) and the pneumonia severity index (PSI), and examine its usefulness for predicting mortality and ICU admission in patients with CAP of high severity and mortality that requires hospitalization.

Methods

We performed a secondary analysis of data from a prospective observational study of adult patients who were admitted to our hospital between October 2010 and June 2016. We compared the prognostic performance of qSOFA, CURB-65, and PSI for predicting in-hospital mortality and ICU admission using the C statistics.

Results

The median age of the 1045 enrolled patients was 77 (68–83) years, and 71.4% were males. The in-hospital mortality and ICU admission rates of the entire cohort were 6.1 and 7.9%, respectively. All scores were significantly higher in non-survivors and ICU admission patients than in survivors and non-ICU admission patients (p < 0.001). The C statistics of qSOFA for predicting in-hospital mortality was 0.69 (95% CI 0.63–0.75), and no significant differences were observed between CURB-65 (C statistics, 0.75; 95% CI 0.69–0.81) and PSI (C statistics, 0.74; 95% CI 0.69–0.80). The C statistics of qSOFA for predicting ICU admission was 0.76 (95% CI 0.71–0.80), and no significant differences were noted between CURB-65 (C statistics, 0.73; 95% CI 0.67–0.79) and PSI (C statistics, 0.72; 95% CI 0.66–0.78).

Conclusions

Regarding hospitalized CAP, the prognostic performance of qSOFA for in-hospital mortality and ICU admission was not significantly different from those of CURB-65 and PSI. qSOFA only requires a few items and vital signs, and, thus, may be particularly useful for emergency department or non-respiratory specialists.

Background

Sepsis-3, a new definition for sepsis, and its diagnostic criteria were published in 2016 [1]. Quick Sepsis-related Organ Failure Assessment (qSOFA) has been used to screen sepsis outside of the intensive care unit (ICU) [2]. Sepsis is diagnosed if at least two out of the three criteria are positive in patients with suspected infections. qSOFA is a very simple screening tool because it only has three evaluation items, does not require laboratory examinations, and may be conducted at the bedside. Moreover, in the original study, the C statistics for the in-hospital mortality of qSOFA was 0.81 as a good indicator of prognosis, and its usefulness as a prognostic tool has been evaluated [3,4,5]; however, some studies have questioned its usefulness [6,7,8].

Community-acquired pneumonia (CAP) is a common infection, is frequently a causative disease for sepsis, and its mortality rate is between approximately 3 and 9% [1, 2, 9,10,11,12,13]. Therefore, the prognosis of CAP needs to be accurately assessed in order to select an appropriate treatment strategy.

Few studies have examined the use of qSOFA for CAP [3, 10, 12]; however, one of these studies reported a high mortality rate [3], while another was a short report in a letter format [10]. Thus, there is currently no consensus as to whether qSOFA is an effective strategy.

A previous study comprehensively examined existing pneumonia severity scores for CAP and qSOFA [12]; however, relatively mild pneumonia that may be treated internally was incorporated and the patient background used was relatively young. The original clinical significance of severity scores was for pneumonia of high severity and mortality that requires hospitalization; however, it currently remains unclear whether qSOFA is useful as a prognostic tool for CAP that requires hospitalization.

The aims of the present study were to compare the prognostic accuracy of qSOFA with existing pneumonia severity scores (CURB-65 [Confusion, Urea, Respiratory Rate, Blood Pressure, and Age] [11] and the pneumonia severity index [PSI] [14]) and to examine the usefulness of qSOFA for CAP of high severity and mortality that requires hospitalization.

Methods

Study design and settings

We performed a secondary analysis of data from a prospective observational study of patients admitted to Kurashiki Central Hospital (1166-bed community hospital, Okayama, Japan) between October 2010 and June 2016.

Adult patients (age, ≥ 18 years) with pneumonia diagnosed by new infiltrates on chest imaging studies (radiography or computed tomography) and two or more symptoms consistent with pneumonia (including cough, dyspnea, fever, sputum production, breathlessness, and/or pleuritic chest pain) were enrolled at an emergency department or outpatient visit.

Exclusion criteria were as follows: age younger than 18 years, a resident of an extended care facility or nursing home, recently discharged from hospital within 90 days, an elderly or disabled individual receiving nursing care, and those receiving regular endovascular treatments as an outpatient (dialysis, antibiotic therapy, chemotherapy, and immunosuppressant therapy).

The present study was approved by the Kurashiki Central Hospital Ethics Committee. Informed consent was obtained from all patients at the time of admission.

Data collection

Data on demographic characteristics, vital signs, imaging, and laboratory test results of the enrolled patients were recorded upon hospital arrival at an emergency department or outpatient visit. All patients who were intubated immediately after hospital arrival also recorded vital signs before intubation. qSOFA, CURB-65, and PSI were calculated using data obtained at enrollment. All data were collected by a study team consisting of a four board-certificated pulmonologist.

Outcome variables

The primary outcome was in-hospital mortality and the secondary outcome was ICU admission.

Statistical analysis

We assessed the predictive performance of qSOFA, CURB-65, and PSI for the primary and secondary outcomes.

Data were presented as medians with interquartile ranges for continuous variables and as numbers and percentages for categorical variables. Variables with significant differences were tested by a binary logistic analysis together with the qSOFA score and presented with an odds ratio (OR) with a 95% confidence interval (CI). Categorical variables were compared using χ2 statistics. In order to assess the discriminatory power of the qSOFA score for predicting outcomes, we compared the C statistics of the qSOFA score with those of the CURB-65 and PSI scores. The C statistic is a summary measure of discrimination which quantifies the ability of the model to assign a high probability. C statistics are equivalent to the area under the receiver operating characteristic curve. C statistics range from 0.5 to 1.0; a measure of 0.5 indicates that the discrimination is caused by chance alone, and 1.0 indicates perfect discrimination. All p value analyses were two-sided and a p value of less than 0.05 was considered to be significant. All statistical analyses were performed with EZR (Saitama Medical Center, Jichii Medical University, Saitama, Japan), which is a graphical user interface for R (The R Foundation for Statistical Computing, Vienna, Austria). More precisely, it is a modified version of R commander designed to add statistical functions frequently used in biostatistics [15].

Results

Characteristics of the study cohort

A total of 1954 patients were evaluated in the enrollment period, and 909 were excluded, among which three were < 18 years, 133 were residents of an extended care facility or nursing home, 361 were recently discharged from hospital within 90 days, 301 were elderly or disabled and receiving nursing care, and 111 were receiving regular endovascular treatment as outpatients. Some patients fulfilled multiple items.

We ultimately enrolled 1045 patients with pneumonia, and their baseline characteristics are listed in Table 1. The median age of these patients was 77 (68–83) years, and 71.4% were males. The main comorbidities were respiratory diseases, such as chronic obstructive pulmonary disease (COPD) (25.5%) and bronchial asthma (13.8%), diabetes (19.5%), chronic heart failure (30.9%), and cerebrovascular diseases (14.3%). The median of qSOFA was 1 point (0–1), the median of CURB-65 was 2 points (1–2), and the median score of PSI was 97 (81–120) and its median class was class IV (III-IV).

Table 1 Baseline characteristics of the study cohort

The median duration of the hospital stay was 11 (8–18) days, the overall in-hospital mortality rate was 6.1%, the 28-day mortality rate was 4.9%, and the ICU admission rate was 7.9%.

Comparison between non-survivors and survivors

The median age (80 (74–85) vs 76 (68–83) years: p = 0.004) was significantly higher among non-survivors than survivors. Respiratory rates (25 (20–30) / min. vs 22 (19–26) / min.: p < 0.001) and mental confusion (37.5 vs 11.2%: p < 0.001) were significantly higher among non-survivors than survivors; however, no significant differences were noted in systolic blood pressure between the two groups (p = 0.21).

qSOFA, CURB-65, and PSI scores were significantly higher among non-survivors than survivors (p < 0.001) (Additional file 1: Figure S1-a).

Comparison between ICU and non-ICU admissions

Respiratory rates (26 (23–30) / min. vs 22 (19–25) / min.: p < 0.001) and mental confusion (38.6 vs 10.6%: p < 0.001) were significantly higher among ICU than non-ICU admissions. Systolic blood pressure was significantly lower among ICU than non-ICU admissions (120 vs 128 mmHg: p = 0.032).

qSOFA, CURB-65, and PSI scores were significantly higher among ICU than non-ICU admissions (p < 0.001) (Additional file 1: Figure S1-b).

In-hospital mortality and ICU admission rates according to each score

Among patients with pneumonia, 13.9% (n = 145) were screened with sepsis based on a qSOFA score ≥ 2 points.

In-hospital mortality rates were 2.1, 5.9, 17.3, and 16.7% for each of the qSOFA points. The mortality rate of a qSOFA score ≥ 2 points was significantly higher than that of a qSOFA score < 2 points (17.2 vs 4.3%, p < 0.001) (Fig. 1a).

Fig. 1

In-hospital mortality and ICU admission rates. a (upper) Stratified by qSOFA scores. (above) qSOFA scores less than 2 or greater than or equal to 2. b (upper) Stratified by CURB65 scores. (above) CURB-65 scores less than 2 or greater than or equal to 2. c (upper) Stratified by PSI scores. (above) PSI risk class less than IV or greater than or equal to IV. Abbreviations: CURB-65 Confusion, Urea, Respiratory Rate, Blood Pressure and Age, ICU intensive care unit, PSI Pneumonia Severity Index, qSOFA quick Sequential (Sepsis-related) Organ Failure Assessment

ICU admission rates became higher as qSOFA scores increased. The ICU admission rate of a qSOFA score ≥ 2 points was significantly higher than that of a qSOFA score < 2 points (23.4 vs 5.4%: p < 0.001) (Fig. 1a).

Hospitalization was recommended for 60.8% (n = 635) and 59.9% (n = 626) of patients based on the CURB-65 score (≥ 2) and PSI class (≥ IV), respectively.

The hospital mortality and ICU admission rates of CURB-65 and PSI also became higher as qSOFA scores increased. The hospital mortality and ICU admission rates for patients with a CURB-65 score ≥ 2 points and PSI class ≥ IV were significantly higher than those with the lower cut-off values (p < 0.001) (Fig. 1b, c).

Score performance

The C statistics of qSOFA for predicting hospital mortality was 0.69 (95% CI, 0.63–0.75), and no significant differences were observed between CURB-65 (C statistics, 0.75; 95% CI 0.69–0.81; p = 0.11) and PSI (C statistics, 0.74; 95% CI, 0.69–0.80; p = 0.18) (Fig. 2a).

Fig. 2

(a) C statistics for predicting hospital mortality. (b) C statistics for predicting ICU admission. Abbreviations: CURB-65 confusion, urea, respiratory rate, blood pressure, and age, ICU intensive care unit, PSI pneumonia severity index, qSOFA quick Sequential (Sepsis-related) Organ Failure Assessment

The C statistics of qSOFA for predicting ICU admission was 0.76 (95% CI 0.71–0.80), and no significant differences were noted between CURB-65 (C statistics, 0.73; 95% CI 0.67–0.79; p = 0.41) and PSI (C statistics, 0.72; 95% CI 0.66–0.78; p = 0.21) (Fig. 2b).

Sensitivity, specificity, and the likelihood ratio

A qSOFA score ≥ 2 points presented moderate sensitivity at 39% and high specificity at 88% for predicting hospital mortality. A CURB-65 score ≥ 2 points and PSI class ≥ IV presented high sensitivity, moderate specificity, and a high negative likelihood (Table 2).

Table 2 Performance of qSOFA for predicting hospital mortality

The prediction of ICU admission was similar, with a qSOFA score ≥ 2 points presenting moderate sensitivity and high specificity, and a CURB-65 score ≥ 2 points and PSI class ≥ IV presenting high sensitivity and moderate specificity (Table 3).

Table 3 Performance of qSOFA for predicting ICU admission

Discussion

The present study is the first to compare the prognostic performance of qSOFA with existing pneumonia severity scores (CURB-65 and PSI) for CAP of high severity and mortality that requires hospitalization. Regarding hospitalized CAP, the prognostic performance of qSOFA for in-hospital mortality did not significantly differ from the existing pneumonia severity scores (CURB-65 and PSI). Furthermore, no significant difference was observed in the prediction performance for ICU admission. PSI is the most famous severity classification of CAP [14], and it has been clearly shown to correlate with mortality. However, this index involves 20 evaluation items and its calculation is complex; therefore, its practical use in a busy clinical setting is limited. On the other hand, CURB-65 has been described as a convenient classification with only five items (confusion, urea, respiratory rate, blood pressure, and age) with high practicality and excellent prognostic accuracy [11]. However, “Urea > 7 mmol/L” requires a blood test. qSOFA has fewer evaluation items than CURB-65 and PSI, and because it involves tests that may be conducted at the bedside, it is regarded as a simpler prognostic tool than CURB-65 and PSI.

The prognostic performance of disease-specific severity scores is excellent because the severity classification was created using a database for each disease. CURB-65 and PSI were also created from a pneumonia database [11, 14]. Therefore, it was of interest that qSOFA, which was created for general infectious diseases, did not significantly differ from pneumonia-specific severity classifications, such as CURB-65 and PSI. The use of separate severity scores for each disease is considered to be burdensome. The present results demonstrated that qSOFA, which may be used for other diseases, may be substituted for existing severity scores for CAP and may be particularly useful for non-respiratory specialists.

Three previous studies were conducted on qSOFA for CAP: 1641 patients in China (28-day mortality rate: 33%, C statistics for predicting 28-day mortality: 0.655), 9327 patients in Germany (30-day mortality rate: 3.0%, C statistics for predicting 30-day mortality: 0.70), and 6874 patients in Spain (in-hospital mortality: 6.4%, C statistics for predicting in-hospital mortality: 0.649) [3, 10, 12]. Although mortality rates markedly varied among these studies, the C statistics of qSOFA for predicting mortality ranged between 0.655 and 0.70, with no significant differences being observed between studies. While our results revealed a 28-day mortality rate of 4.9% and in-hospital mortality rate of 6.1%, the C statistics of 0.69 for in-hospital mortality was in accordance with previous studies [3, 10, 12]. Therefore, qSOFA may be used for CAP irrespective of mortality, regional differences, and patient background differences.

The present study on patients with CAP showed that the C statistics of qSOFA for predicting in-hospital mortality was 0.69, which was lower than that of 0.81 in the original study [2]. Three previous studies on patients with CAP also showed C statistics for predicting mortality of between 0.655 and 0.70 [3, 10, 12]. The respiratory rate, which was one of the three qSOFA criteria evaluated in the present study, scored positive in at least 50% of the 1045 patients analyzed (Fig. 3). Since qSOFA only has three criteria, having a positive score for one of the criteria in more than 50% of patients may have decreased its reliability, thereby reducing the C statistics. The reason for the reduced C statistics may be the low cut-off value for the respiratory rate. Therefore, we changed the cut-off value for the respiratory rate in qSOFA to ≥ 30/min, which was the same as that in CURB-65 or PSI. However, no significant difference was observed, even though it changed from ≥ 22/min to ≥ 30/min (C statistics: qSOFA 0.69, qSOFA (RR ≥ 30/min) 0.71; p = 0.28) (Additional file 2: Figure S2).

Fig. 3

Distribution of each criterion of qSOFA. Abbreviations: qSOFA quick Sequential (sepsis-related) Organ Failure Assessment

Previous studies comparing qSOFA with CURB-65 and PSI were reviewed by the Spanish study [12] described above, in which qSOFA had a lower prognostic performance than CURB-65 and PSI. CURB-65 and PSI included age as an item of severity. The severity score in elderly patients was slightly high when measured by PSI because a patient’s age largely contributes to its scoring system. Since the median age of our study population was more than 10 years older than that of the Spanish study, patients in our study may have scored higher in PSI, thereby reducing the C statistics. Besides CURB-65 and PSI, many severity classifications include age as an evaluation item. Previous studies reported that it was not necessary to include age (older than 65 years) as one of the three items in qSOFA [3, 5]. In the present study, the addition of age to qSOFA did not affect the results obtained (qSOFA vs qSOFA + age ≥ 65: p = 0.10), although C statistics slightly increased to 0.71 (Additional file 3: Figure S3); therefore, the addition of age was unnecessary based on convenience.

In the present study, 13.9% of pneumonia cases were diagnosed with sepsis with a qSOFA score ≥2 points, and the mortality rate of a qSOFA score ≥ 2 points was 17.2%. This severity was close to “life-threatening organ dysfunction caused by a dysregulated host response to infection, in-hospital mortality rate 10%” defined as sepsis-3. As the score of qSOFA became higher, in-hospital mortality and ICU admission rates gradually increased, the in-hospital mortality rate for patients with a qSOFA score ≥ 2 points was fourfold higher than those with lower cut-off values, and the ICU admission rate was fivefold higher. Previous studies that investigated qSOFA in CAP showed high specificity, but moderate sensitivity, similar to the present study. On the other hand, CURB-65 and PSI showed high sensitivity and moderate specificity. C statistics were no significantly different between these predictive models; qSOFA might be used to screen severe pneumonia, because of its high specificity.

Limitations

The limitations of the present study were as follows. First, it was a single-center secondary analysis of data from a prospective observational study. The prospective observational study of qSOFA was only one study targeting all infectious diseases in the emergency department [4], and there has not yet been a prospective observational study on qSOFA for pneumonia. Therefore, multicenter prospective studies are needed in order to validate our results. Furthermore, although the C statistics of qSOFA for predicting mortality was similar to the three previous studies [3, 10, 12], there are still only four studies on pneumonia, including the present study; therefore, the evaluation of qSOFA for pneumonia is currently insufficient. In addition, it is unclear whether qSOFA may be used reliably for patients in different regions or with varying backgrounds. Second, it was that the present study only considered hospitalized CAP. Pneumonia in patients with a poor performance status and hospital-acquired pneumonia also need to be considered. Third, the present study have the potential for the information bias given our retrospective study design. We attempted to reduce the information bias by using a prospectively collected data. Forth, the decision to enter the ICU is decided based on preset ICU admission criteria (such as, respiratory failure requiring mechanical ventilation, circulatory failure requiring vasopressor, confusion, etc.). However, potential selection bias existed, e.g., the decision of the physician might be biased by the score. Finally, if the period from emergency department visit to ICU admission was long, the meaning of the score at entry could be diminished. But in our study, the length from emergency department visit to ICU admission was all within 7 days except for two cases.

Conclusion

Regarding hospitalized CAP, the prognostic performance of qSOFA for in-hospital mortality and ICU admission was not significantly different to those of CURB-65 and PSI. Since qSOFA only requires a few items and vital signs, it may be particularly useful for emergency department or non-respiratory specialists.