Accuracy of Critical Care Pain Observation Tool and Behavioral Pain Scale to assess pain in critically ill conscious and unconscious patients: prospective, observational study
© The Author(s). 2016
Received: 15 March 2016
Accepted: 28 October 2016
Published: 7 November 2016
Critically ill patients admitted to intensive care unit (ICU) may suffer from different painful stimuli, but the assessment of pain is difficult because most of them are almost sedated and unable to self-report. Thus, it is important to optimize evaluation of pain in these patients. The main aim of this study was to compare two commonly used scales for pain evaluation: Critical Care Pain Observation Tool (CPOT) and Behavioral Pain Scale (BPS), in both conscious and unconscious patients. Secondary aims were (1) to identifying the most relevant parameters to determine pain scales changes during nursing procedures, (2) to compare both pain scales with visual analog scale (VAS), and (3) to identify the best combination of scales for evaluation of pain in patients unable to communicate.
In this observational study, 101 patients were evaluated for a total of 303 consecutive observations during 3 days after ICU admission. Measurements with both scales were obtained 1 min before, during, and 20 min after nursing procedures in both conscious (n.41) and unconscious (n.60) patients; furthermore, VAS was recorded when possible in conscious patients only. We calculated criterion and discriminant validity to both scales (Wilcoxon, Spearman rank correlation coefficients). The accuracy of individual scales was evaluated. The sensitivity and the specificity of CPOT and BPS scores were assessed. Kappa coefficients with the quadratic weight were used to reflect agreement between the two scales, and we calculated the effect size to identify the strength of a phenomenon.
CPOT and BPS showed a good criterion and discriminant validity (p < 0.0001). BPS was found to be more specific (91.7 %) than CPOT (70.8 %), but less sensitive (BPS 62.7 %, CPOT 76.5 %). COPT and BPS scores were significantly correlated with VAS (p < 0.0001). The combination of BPS and CPOT resulted in better sensitivity 80.4 %. Facial expression was the main parameter to determine pain scales changes effect size = 1.4.
In critically ill mechanically ventilated patients, both CPOT and BPS can be used for assessment of pain intensity with different sensitivity and specificity. The combination of both BPS and CPOT might result in improved accuracy to detect pain compared to scales alone.
KeywordsPain Critical Care Pain Observation Tool Behavioral Pain Scale Critical ill patients Intensive care unit Pain management
Pain management in critically ill patients is a complex process, but relevant to the clinical management. Pain is highly underestimated although it seems to be the patients’ worst memory in intensive care unit (ICU) [1, 2] even after 5 years from ICU discharge . The perception of pain in ICU patients is mainly associated with respiratory therapy, positioning of nasogastric tube, venous and arterial catheters, and lack of mobilization . However, patients are usually unable to self-report their pain due to sedative drugs and intubation, likely leading to its underestimation [4, 5]. Pain with agitation and delirium has been reported to negatively affect outcome of mechanically ventilated patients . Thus, it is required to have valid and reliable methods to assess pain in unconscious patients to optimize treatment . There is no one standard approach to evaluate pain in ICU, and the proposed tools actually used have several advantages and disadvantages. In conscious patients, self-report, i.e., visual analog scale (VAS) is the gold standard for pain assessment . In unconscious patient, new methods have been developed to assess pain by using behavioral scales [9–11]. In unconscious patients, two scales have been proposed to assess pain in ICU patients: Behavioral Pain Scale (BPS)  and Critical Care Pain Observation Tool (CPOT) . However, the potential superiority of each of them for assessment of pain in mechanically ventilated patients is not well established [14, 15]. The main difference between CPOT and BPS is the evaluation of body movements and muscle tension. We hypothesized that CPOT is more sensitive and accurate to assess pain compared to BPS in critically ill patients being specifically focused on muscular tension.
The aims of the present study were the following: (1) to compare CPOT and BPS separately, in conscious and unconscious critically ill mechanically ventilated patients; (2) to identify the most relevant parameters to determine pain scales changes during nursing procedures; (3) to compare both pain scales with VAS; and (4) to identify the best combination of scales for evaluation of pain in patients unable to communicate.
This was a prospective, mono-centric study registered at ClinicalTrials.gov (ID NCT01669486). The study was conducted in the ICU at “Ospedale di Circolo Fondazione Macchi Varese”. The study was revising by the local Ethical Committee (protocol n.0003412), and informed consent was obtained from the relatives or patients according to local regulations.
The staff is made up of 12 doctors and 28 nurses for 12 beds in a general intensive care unit. The patients-nurse ratio is 2:1 by day and 3:1 by night. The medical staff is also completed by five physicians in training. The most active phase of nursing care was made every morning and every afternoon. The nurses performed maneuvres for taking care of hygiene, therapy administration, and continuous monitoring of vital parameters. They are graduated in nursing after 3 years at university school and 2 years of master in intensive care. The medical staff is provided by physician graduated in medicine and surgery and specialized in anesthesia and intensive care after 5 years in training. The medical staff during the study was responsible for the assessment of pain through the acquisition of the behavioral scale scores. Assessments have been carried out by medical staff trained to identify the presence of pain with two scales. These observations were not blinded. In the morning, nurses provide patients’ passive turning, cleaning, and repositioning; they perform airway suctioning, medications, and catheter management. In the afternoon, nurses provide only patient cleaning and repositioning. To get standardized measurements before, during, and after the maximum level of pain stimuli, we analyzed the behavioral scale scores during the nursing care performed in the morning .
The patients were evaluated with the Glasgow Coma Scale (GCS) and Sedation Agitation Scale (SAS). The conscious patients were identified with a GCS >10 and SAS = 4 (avoiding too agitated and too sedated patients to determine VAS).
Behavioral Pain Scale, Critical Care Pain Observation Tool, Behavioral Pain Scale and Critical Care Pain Observation Tool combination
BPS and CPOT combination score
3 or 4
5 or 6
Upper Limb Movement
Absence of movements
0 or 1
0 or 1
3 or 4
Fully bent with finger flexion
3 or 4
0, 1, or 2
Very tense, or rigid
0, 1, or 2
5, 6, 7, or 8
Coughing but tolerating for the most of time
Coughing but tolerating
Unable to control ventilation
3 or 4
5 or 6
The statistical calculations were performed with in MedCalc for Windows, version 220.127.116.11. First, we calculate the sample size. The sample size calculation determined the number of the patients to enroll, and it was calculated a “priori” and based on 0.05 type I error and 0.20 type II error with a difference expected of 10 % order of magnitude in the area under the curve (AUC) area. We evaluated the validity to both scales, as criterion validity and discriminant validity. Discriminant validity refers to the ability of an instrument to measure the presence or the absence of the variable. In this study, the discriminant was correlated with the mean scores of both scales before and during the nursing care, 20 min after and during the procedure, and before and 20 min after the procedure, in conscious and unconscious patients. This correlation was calculated with Wilcoxon coefficient, which is a non-parametric test to determinate whether two samples come from the same statistical population, in the presence of ordinal values and continuous distribution. Box plot is a way to describe groups of numerical data through their quartiles. Criterion validity refers to the ability of an instrument to accurately measure the phenomenon of interest, in this case, the measurement of pain. It was evaluated by correlating the observed CPOT and BPS scores to the “gold standard” of pain measurement, i.e., VAS, when possible in conscious patients, and using the Spearman rank correlation coefficient (rs). Additionally, the sensitivity and the specificity of CPOT and BPS scores with ROC curve were assessed, only in conscious patients. The best discriminating between real and false positive is obtained by a curve passing in the upper left corner of at x/Y graphic (x = 1 − specificity, y = sensibility). In this case, the true positive corresponds to 100 % and the false positive correspond to 0 %. The AUC measures the ability of the scales to discriminate between patients who did or did not feel pain. The accuracy of individual scales was evaluated using calculation of sensitivity multiplied by the prevalence of positive to the sum of the observations (VAS ≥3, presence of pain) added to the specificity multiplied by the prevalence of negatives (VAS <3, absence of pain). Kappa coefficients with the quadratic weight were used to reflect agreement between the two scales , in conscious and unconscious patients. The Cohen’s Kappa is statistical coefficient that represents the degree of accuracy and reliability in a statistical classification; it is a concordance index calculated according to the ratio between the agreements in excess of the maximum obtainable . The effect size is a quantitative measure of the strength of a phenomenon, for example, the correlation between two variables. We used this test to identify the most important subscale both BPS and CPOT, in conscious and unconscious patients. Cohen classified effect size as small (0.2–0.5), medium (0.5–0.8), large (0.8–1.3), and very large (>1.3) .
The level of significance accepted was a P value <0.05.
Clinical characteristics of the patients at the time of enrollment
65 ± 16.7
APACHE II (mean ± SD)
15.7 ± 7.1
SAPS II (mean ± SD)
43.5 ± 13.4
SAS (mean ± SD)
3.0 ± 1.1
(−) n: 101; (+) n: 0
ICU discharge (n)
The medical patients were affected to respiratory failure, from pulmonary edema and pneumonia. The surgical ones instead included patients undergoing abdominal, vascular, and thoracic surgery and multiple trauma. In addition to usual devices like central lines, arterial line, gastric tube, tracheal tube, and urinary catheter, these patients presented additionally surgical incisions, drainages, and open abdomen treatment.
We calculated discriminant validity for BPS and CPOT in overall, conscious and unconscious patients. BPS showed a statistically significant difference during nursing care (overall Z = −12.3, p < 0.0001; conscious Z = −6.93, p < 0.0001; unconscious Z = −10.68, p < 0.0001) and during and after nursing care (overall Z = −12.6, p < 0.0001; conscious Z = −6.78, p < 0.0001; unconscious Z = −11.15, p < 0.0001). We observed similar results with CPOT (during: overall Z = −12.09, p < 0.0001; conscious Z = −6.48, p < 0.0001; unconscious Z = −10.62, p < 0.0001; during and after, overall Z = −12.81, p < 0.0001; conscious Z = −6.64 p < 0.0001; unconscious Z = −11.36, p < 0.0001).
We compared the two scales in three different moments, with the Cohen’s Kappa, before k = 0.69, during = 0.64, after = 0.66; a k-value larger than 0.6 showed a good correlation.
Among different individual parameters for CPOT and BPS, in overall, conscious and unconscious patients, facial expression was the most important one for pain detection with effect size 1.4 while the other parameters presented an effect size more less (BPS scale Upper Limb Movement = 0.84, Ventilator Compliance = 0.99; CPOT scale Muscle Tone = 0.71, Body Movement = 0.60 and Ventilator Compliance = 1.09).
The criterion validity of BPS and CPOT scale showed a strong correlation with VAS, including all measurements (BPS rs = 0.56; p < 0.0001 CPOT rs = 0.48; p < 0.0001).
The table shows results of CPOT and BPS sensitivity and specificity
BPS and CPOT combination score
In the present study, we evaluated two different pain scales in unconscious and conscious critically ill mechanically ventilated patients during nursing care. We found that (1) CPOT and BPS separately increased during nursing care in both unconscious and conscious patients and were significantly correlated; (2) facial expression showed greater changes for pain assessment; (3) in conscious patients, during nursing care, BPS showed higher specificity, and lower sensitivity compared to CPOT; and (4) the combination of both BPS and CPOT resulted in improved accuracy to detect pain compared to individual pain scales. Thus, against our expectations, our data suggest that CPOT was actually equivalent to BPS in sensitivity and accuracy for pain evaluation because none scale have better sensitivity and specificity to each other (BPS sensitivity 62.8 % and specificity 91.7 %, CPOT sensitivity 76.5 %, and specificity 70.8 %).
Our data suggest that using both BPS and CPOT during nursing care or other painful intervention might improve the evaluation of pain. To our knowledge, this is the first study suggesting that the combination of BPS and CPOT may be considered as a valuable tool for pain assessment in mechanically ventilated critically ill patients [15, 23, 24].
In the present study, we evaluated patients requiring mechanical ventilation and admitted to a general ICU. Thus, our results may be more easily applicable to a mixed population of critically ill patients. Few previous studies simultaneously recorded BPS and CPOT as well as VAS [15, 25, 26]. We compared both BPS and CPOT with VAS in conscious patients during nursing care, but our findings can be applied also to unconscious patients, being the pain assessment and the level of analgesia similar in both groups. These scales are normally used to evaluate pain in unconscious patients. Based on our own data, we demonstrated that BPS and CPOT might provide information about pain in unconscious patients, as shown by the correlation between VAS and BPS and CPOT. First, we showed a correlation between VAS and BPS and CPOT in conscious patients. Secondly, we obtained similar finding in unconscious patients.
Our results are in line with those reported in previous studies showing that both BPS and CPOT increase during nursing care and return back to baseline in a short period of time . The diagnostic performance of both CPOT and BPS worsened after nursing care, suggesting that these scores might be affected by clinical maneuvres. Procedures like passive mobilization, i.e., turning and repositioning, and suctioning have been shown to increase pain. Conversely, active mobilization, i.e., rehabilitation, might be associated with less pain. In the present study, nursing procedures included turning and repositioning, but not routine suctioning and rehabilitation. Rehabilitation was not considered in the present study, since it might not induce pain due to active movements, better controlled by conscious patients only. Both BPS and CPOT have been evaluated in both unconscious and conscious patients. Pain assessment was similar in unconscious and conscious patients, suggesting that nursing care is painful independent of the level of sedation and analgesia . Different individual items are included in BPS and CPOT. Muscular tone movement of arms and legs are included in CPOT but not BPS. Facial expression and ventilator compliance are recorded in both scales, although using different individual scores. We found that facial expression was the most important parameter related to pain assessment, in agreement with previous literature [15, 24]. It is important to note that facial expression is also easier to be scored at bedside. Furthermore, BPS and CPOT showed a good criterion and discriminant validity as previously reported [24–27], but BPS showed higher specificity but lower sensitivity compared to CPOT and so we cannot have considered CPOT superior to BPS, contrary to our hypothesis.
Thus, we hypothesized that of both BPS and CPOT, scales might result in improved accuracy to detect pain compared to each scale alone. The accuracy was evaluated by summing the scores of both scales for each individual observation. We found that during nursing care, the combination of BPS and CPOT resulted in better sensitivity. On the other hand, its specificity was higher than CPOT but lower than BPS. However, it was not possible to assess the best type of combination between the two scales, due to the higher prevalence of true positive cases and the limited sample size. In our study we did not find patients with delirium, and this may have been due to the optimization of the level of sedation.
Our study has limitations to be addressed. First, we evaluated only nursing care and no other possible painful maneuvers like suctioning. Second, the pain scales used are subjective to the operator and not objective. However, only one trained assessor evaluated pain for each patient. Third, we did not found any case of delirium in our analyzed patients, and this may be due to the level of sedative drugs applied, and/or to the feasibility of the CAM-ICU application in our population of critically ills patients.
Fourth, a relatively small group of patients was analyzed. Further studies are required to confirm our results in a larger population of patients.
Both CPOT and BPS scales are applicable to detect pain in conscious and unconscious critically ill mechanically ventilated patients, but with different sensitivity and specificity. The association of both scales might improve the efficiency for pain assessment. The level of consciousness does not affect the perception of pain during nursing care.
Acute Physiology and Chronic Health Evaluation
Area under the curve
Behavioral Pain Scale
Critical Care Pain Observation Tool
Intensive care unit
- SAPS II:
Simplified Acute Physiology Score
Sedation Agitation Scale
Visual analog scale
- Z :
The author would like to specially thank all medical and nursing staff in the department of critical care unit for their collaboration and assistance with data collection. An important thank to patients who participated to the study.
We had no external funding sources; each author has funded the research with their own funds.
Availability of data and materials
The dataset supporting the conclusions of this article is available in data base file in additional materials repository.
SP and PP conceived the study, participated in its design as well as coordination, and drafted the manuscript. EC participated as a local investigator, participated in the design of the study, and helped to draft the manuscript. ES participated as a local investigator and helped to draft the manuscript. NR participated in the design of the study and performed the statistical analysis. All authors read and approved the final manuscript.
-Severgnini Paolo, MD is the coordinator of steering committee. Azienda Ospedaliera Fondazione Macchi ASST Sette Laghi, Intensive Care Unit–Department of Biotechnologies and Sciences of Life, University of Insubria, Varese, Italy
-Pelosi Paolo, MD, FERS is a member of steering committee. IRCCS AOU San Martino-IST, Department of Surgical Sciences and Integrated Diagnostics, University of Genoa, Genoa, Italy
-Elena Contino, MD is a member of steering committee. Azienda Ospedaliera Fondazione Macchi ASST Sette Laghi, Intensive Care Unit–Department of Biotechnologies and Sciences of Life, University of Insubria, Varese, Italy
-Elisa Serafinelli, MD is a member of steering committee. Azienda Ospedaliera Fondazione Macchi ASST Sette Laghi, Intensive Care Unit–Department of Biotechnologies and Sciences of Life, University of Insubria, Varese, Italy
-Novario Raffaele, PH is a member of steering committee, participated in the design of the study and performed the statistical analysis. Azienda Ospedaliera Fondazione Macchi ASST Sette Laghi, Medical Physics Unit–Department of Biotechnologies and Sciences of Life, University of Insubria, Varese, Italy
-Chiaranda Maurizio, MD, PhD is a member of steering committee. Azienda Ospedaliera Fondazione Macchi ASST Sette Laghi, Anesthesia and Intensive Care Unit “A”– Department of Biotechnologies and Sciences of Life, University of Insubria, Varese, Italy
The authors declare that they have no competing interests.
Consent for publication
Informed consent was obtained from the relatives or patients according to local regulations.
Ethics approval and consent to participate
The study was approved by the local Ethical Committee protocol n.0003412 with a deliberative act n.1514 on December 30, 2008, and informed consent was obtained from the relatives or patients according to local regulations, for any patient enrolled.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Granja C, Lopes A, Moreira S, et al. Patients’ recollections of experiences in the intensive care unit may affect their quality of life. Crit Care. 2005;9(2):R96–109.View ArticlePubMedPubMed CentralGoogle Scholar
- Gelinas C. Management of pain in cardiac surgery ICU patients: have we improved over time? Intensive Crit Care Nurs. 2007;23(5):298–303.View ArticlePubMedGoogle Scholar
- Zetterlund P, Plos K, Bergbom I, et al. Memories from intensive care unit persist for several years—a longitudinal prospective multi-centre study. Intensive Crit Care Nurs. 2012;28(3):159–67.View ArticlePubMedGoogle Scholar
- Russell S. An exploratory study of patients’ perceptions, memories and experiences of an intensive care unit. J Adv Nurs. 1999;29(4):783–91.View ArticlePubMedGoogle Scholar
- Cullen L, Greiner J, Titler MG. Pain management in the culture of critical care. Crit Care Nurs Clin North Am. 2001;13(2):151–66.PubMedGoogle Scholar
- Dale CR, Kannas DA, Fan VS, et al. Improved analgesia, sedation, and delirium protocol associated with decreased duration of delirium and mechanical ventilation. Ann Am Thorac Soc. 2014;11(3):367–74.View ArticlePubMedPubMed CentralGoogle Scholar
- Lindenbaum L, Milia DJ. Pain management in the ICU. Surg Clin North Am. 2012;92(6):1621–36.View ArticlePubMedGoogle Scholar
- Chanques G, Viel E, Constantin JM, et al. The measurement of pain in intensive care unit: comparison of 5 self-report intensity scales. Pain. 2010;151(3):711–21.View ArticlePubMedGoogle Scholar
- Puntillo KA, Morris AB, Thompson CL, et al. Pain behaviors observed during six common procedures: results from Thunder Project II. Crit Care Med. 2004;32(2):421–7.View ArticlePubMedGoogle Scholar
- Mateo OM, Krenzischek DA. A pilot study to assess the relationship between behavioral manifestations and self-report of pain in postanesthesia care unit patients. J Post Anesth Nurs. 1992;7(1):15–21.PubMedGoogle Scholar
- Li D, Puntillo K, Miaskowski C. A review of objective pain measures for use with critical care adult patients unable to self-report. J Pain. 2008;9(1):2–10.View ArticlePubMedGoogle Scholar
- Payen JF, Bru O, Bosson JL, et al. Assessing pain in critically ill sedated patients by using a behavioral pain scale. Crit Care Med. 2001;29(12):2258–63.View ArticlePubMedGoogle Scholar
- Gelinas C, Fortier M, Viens C, et al. Pain assessment and management in critically ill intubated patients: a retrospective study. Am J Crit Care. 2004;13(2):126–35.PubMedGoogle Scholar
- Liu Y, Li L, Herr K. Evaluation of two observational pain assessment tools in Chinese critically ill patients. Pain Med. 2015;16(8):1622–8.View ArticlePubMedGoogle Scholar
- Ahlers SJ, van Gulik L, van der Veen AM, et al. Comparison of different pain scoring systems in critically ill patients in a general ICU. Crit Care. 2008;12(1):R15.View ArticlePubMedPubMed CentralGoogle Scholar
- Gelinas C, Fillion L, Puntillo KA, et al. Validation of the critical-care pain observation tool in adult patients. Am J Crit Care. 2006;15(4):420–7.PubMedGoogle Scholar
- Aissaoui Y, Zeggwagh AA, Zekraoui A, et al. Validation of a behavioral pain scale in critically ill, sedated, and mechanically ventilated patients. Anesth Analg. 2005;101(5):1470–6.View ArticlePubMedGoogle Scholar
- Jeitziner MM, Schwendimann R, Hamers JP, et al. Assessment of pain in sedated and mechanically ventilated patients: an observational study. Acta Anaesthesiol Scand. 2012;56(5):645–54.View ArticlePubMedGoogle Scholar
- Riker RR, Picard JT, Fraser GL. Prospective evaluation of the Sedation-Agitation Scale for adult critically ill patients. Crit Care Med. 1999;27(7):1325–9.View ArticlePubMedGoogle Scholar
- Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74.View ArticlePubMedGoogle Scholar
- Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther. 2005;85(3):257–68.PubMedGoogle Scholar
- Sullivan GM, Feinn R. Using effect size-or why the P value is not enough. J Grad Med Educ. 2012;4(3):279–82.View ArticlePubMedPubMed CentralGoogle Scholar
- Barr J, Fraser GL, Puntillo K, et al. Clinical practice guidelines for the management of pain, agitation, and delirium in adult patients in the intensive care unit. Crit Care Med. 2013;41(1):263–306.View ArticlePubMedGoogle Scholar
- Gelinas C, Puntillo KA, Joffe AM, et al. A validated approach to evaluating psychometric properties of pain assessment tools for use in nonverbal critically ill adults. Semin Respir Crit Care Med. 2013;34(2):153–68.View ArticlePubMedGoogle Scholar
- Pudas-Tahka SM, Axelin A, Aantaa R, et al. Pain assessment tools for unconscious or sedated intensive care patients: a systematic review. J Adv Nurs. 2009;65(5):946–56.View ArticlePubMedGoogle Scholar
- Puntillo K, Pasero C, Li D, et al. Evaluation of pain in ICU patients. Chest. 2009;135(4):1069–74.View ArticlePubMedGoogle Scholar
- Rijkenberg S, Stilma W, Endeman H, et al. Pain measurement in mechanically ventilated critically ill patients: Behavioral Pain Scale versus Critical-Care Pain Observation Tool. J Crit Care. 2015;30(1):167–72.View ArticlePubMedGoogle Scholar