- Open Access
Disagreement in cardiac output measurements between fourth-generation FloTrac and critical care ultrasonography in patients with circulatory shock: a prospective observational study
Journal of Intensive Carevolume 7, Article number: 21 (2019)
Cardiac output measurements may inform diagnosis and provide guidance of therapeutic interventions in patients with hemodynamic instability. The FloTrac™ algorithm uses uncalibrated arterial pressure waveform analysis to estimate cardiac output. Recently, a new version of the algorithm has been developed. The aim was to assess the agreement between FloTrac™ and routinely performed cardiac output measurements obtained by critical care ultrasonography in patients with circulatory shock.
A prospective observational study was performed in a tertiary hospital from June 2016 to January 2017. Adult critically ill patients with circulatory shock were eligible for inclusion. Cardiac output was measured simultaneously using FloTrac™ with a fourth-generation algorithm (COAP) and critical care ultrasonography (COCCUS). The strength of linear correlation of both methods was determined by the Pearson coefficient. Bland-Altman plot and four-quadrant plot were used to track agreement and trending ability.
Eighty-nine paired cardiac output measurements were performed in 17 patients during their first 24 h of admittance. COAP and COCCUS had strong positive linear correlation (r2 = 0.60, p < 0.001). Bias of COAP and COCCUS was 0.2 L min−1 (95% CI − 0.2 to 0.6) with limits of agreement of − 3.6 L min−1 (95% CI − 4.3 to − 2.9) to 4.0 L min−1 (95% CI 3.3 to 4.7). The percentage error was 65.6% (95% CI 53.2 to 77.3). Concordance rate was 64.4%.
In critically ill patients with circulatory shock, there was disagreement and clinically unacceptable trending ability between values of cardiac output obtained by uncalibrated arterial pressure waveform analysis and critical care ultrasonography.
Clinicaltrials.gov, NCT02912624, registered on September 23, 2016
Critically ill patients with circulatory shock have increased risks of multi-organ failure, long-term morbidity, and mortality . Advanced hemodynamic monitoring in these patients may inform diagnosis and simultaneously guide management by providing insight into cardiac function, cardiac preload, and afterload . Several methods for measuring cardiac output (CO) exist, varying from invasive (e.g. thermodilution by pulmonary artery catheter (PAC)) to minimally invasive (e.g. pulse contour analysis by FloTrac™ (Edwards Lifesciences, Irvine, USA)) or even non-invasive (e.g. transthoracic Doppler ultrasound by critical care ultrasonography (CCUS)). These methods all have their own merits, disadvantages and requirements .
One type of pulse contour analysis is the uncalibrated arterial pressure waveform analysis method to estimate CO (APCO). Reliability of APCO is questioned in patients with hemodynamic instability, and this occurs frequently in patients admitted to the ICU . Therefore, CO measurements obtained by APCO should be interpreted with caution in critically ill patients with circulatory shock [5, 6].
The FloTrac™ system using the APCO method calculates CO based on the principle that aortic pulse pressure is proportional to stroke volume (SV) and inversely related to aortic compliance using a proprietary algorithm. FloTrac™ has been widely studied in more than 70 validation studies as of yet, mostly showing adequate performance in normo- and hypodynamic conditions, but not in patients with large changes in vascular tone which typically occur in patients with circulatory shock . However, these studies vary by the statistical methods and versions of the algorithm used. Recently, the fourth-generation algorithm was developed to improve performance.
Evaluation of the trending ability rather than the agreement of absolute values of CO monitoring devices is increasingly considered in validation studies for assessment of potential clinical usefulness . In addition to one single CO measurement for diagnosing circulatory shock, repeated measurements of CO informing the trending ability could be informative for monitoring and guidance of supportive treatments of patients with circulatory shock.
The aim of our study was to compare both agreements and trending ability for APCO measurements of CO (COAP) with CO routinely measured by CCUS (COCCUS) in critically ill patients with circulatory shock. CCUS was chosen as the reference standard since it is the preferred method for diagnosis, but not for monitoring, of circulatory shock in critically ill patients and is widely available [2, 9]. Importantly, it should be noted that CCUS is not a gold standard reference technique for method comparison studies aiming to evaluate the validity of CO monitors .
This study was a substudy of the Simple Intensive Care Studies-I (SICS-I), which was a single-centre, prospective, observational cohort study in which all consecutive acutely admitted adult patients expected to stay beyond 24 h were included (NCT02912624) [16, 17]. The STROBE guidelines for reporting observational studies were used (Additional file 1) . The checklist for CO monitor method comparison studies was used . The local institutional review board (Medisch Ethische Toetsingscommissie, University Medical Center Groningen) approved the study (M15.168207 and M16.193856). Written informed consent was obtained from all patients.
In this substudy, all consecutive acutely admitted adult patients with suspected circulatory shock and expected to stay beyond 48 h were included from June 2016 to January 2017. The circulatory shock was defined as the requirement of any dose of vasopressor to maintain a mean arterial pressure (MAP) of 60 mmHg or if the MAP remained below 70 mmHg despite fluid resuscitation (defined by at least 1000 mL of crystalloids). In addition, at least one other sign of organ or tissue hypoperfusion had to be present: altered state of mind (Alert-Voice-Pain-Unresponsive scale) , mottled skin (Mottling score ≥ 1 ), decreased urine output (≤ 0.3 mL kg−1 h−1) or increased serum lactate level (≥ 2 mmol L−1). Exclusion criteria were inability to obtain sufficient quality CCUS images; no arterial line; atrial fibrillation; and aortic valve or mitral valve diseases known to impair the arterial waveform. We included this group of patients because CO measurements are indicated to identify the type of shock, select necessary therapeutic interventions and evaluate patient’s response to therapy .
The primary objective was to evaluate COAP measurements in terms of the agreement and trending ability against COCCUS as reference technique in patients with circulatory shock.
Definitions and bias
Patient characteristics including clinical, hemodynamic and laboratory variables as well as Acute Physiology And Chronic Health Evaluation (APACHE) IV and Simplified Acute Physiology Score (SAPS) II values were recorded [14, 15]. Measurements were performed following protocolized definitions and procedures [16, 17].
In short, COCCUS was measured by transthoracic echocardiography using the Vivid-S6 system (General Electric, Horton, Norway) with cardiac probe M3S or M4S, and with default cardiac imaging setting. The parasternal long axis was used to measure the left ventricular outflow tract diameter. In the apical five-chamber view, a pulse wave Doppler signal in the left ventricular outflow tract was used to measure the velocity time integral. COCCUS was calculated using an established formula . CCUS was performed after ICU admission within 6 h and repeated once every 24 h after admission provided there was no interference with clinical care. Researchers were trained in performing CCUS by experienced cardiologist-intensivists.
The FloTrac™ sensor was connected to an indwelling radial artery catheter and an EV1000™ monitor (version 4.00; Edwards Lifesciences, Irvine, USA), which continuously displayed COAP values. The value of COAP displayed on the EV1000™ monitor was registered simultaneously (i.e. ‘beat-to-beat’) with each COCCUS measurement.
All measurements, including CCUS findings, were kept blind for the caregivers. Quality of CCUS images and COCCUS measurements were validated by an independent specialized core laboratory (Groningen Image Core Lab) blinded for the COAP measurements
No formal sample size calculation was performed due to lack of data on COAP variation in patients with circulatory shock. Therefore, this study has an exploratory nature.
Data were presented as means with standard deviations or medians with interquartile ranges depending on distributions. Normality of data was checked using the Shapiro-Wilk test. Dichotomous and categorical data were presented in proportions.
Correlations were assessed by scatter plot, and the strength of linear correlation was determined by calculating a Pearson (r) coefficient. Bland-Altman analyses of repeated measurements in each patient were performed and means (bias) and SD of the differences, 95% limits of agreement (LOA) (=mean difference ± 1.96 × SD of the difference) as well as the percentage error of COAP versus COCCUS were calculated [19, 20]. To evaluate the trending ability of COAP versus COCCUS a four-quadrant plot was used and the concordance rate was calculated using an exclusion zone of 0.5 L min−1 . For statistical analysis, we used STATA version 15.0 (StataCorp, College Station, USA).
During the study period, 184 patients were diagnosed with circulatory shock, but only 24 patients appeared eligible for this study. One hundred patients who had circulatory shock were not included as they were expected to stay for less than 48 h, and 60 patients with circulatory shock were not included because CCUS was not possible or image quality was insufficient to perform measurements. Six patients had to be excluded because study procedures interfered with clinical care, leaving 18 patients to be included. One patient was excluded afterwards for invalid COAP measurements due to improper use of a FloTrac™ sensor. Thus, 17 patients were included in the final analyses (Fig. 1).
Bias, precision and correlation
The characteristics of the 17 included patients are shown in Table 1 (and Additional file 2: Table S1). The mean COAP and COCCUS for 89 paired measurements were 5.9 ± 1.9 L min−1 and 5.7 ± 2.0 L min−1, respectively (p = 0.24). A significant correlation was observed for all CO measurements (r2 = 0.60, p < 0.001) (Fig. 2). Bias was 0.2 L min−1 (95% CI − 0.2 to 0.6), with LOA of − 3.6 L min−1 (95% CI − 4.3 to − 2.9) to 4.0 L min−1 (95% CI 3.3 to 4.7) (Fig. 3). Plotting a regression line in the Bland-Altman plot gave no arguments for proportional bias (line not shown). The overall percentage error was 65.6% (95% CI 53.2 to 77.3). Individual cardiac output measurements for each patient are provided in Additional file 3: Table S2.
For assessment of trending ability 72 paired measurements were analysed. Trending of measurements was evaluated using a four-quadrant plot (Fig. 4). Forty-five paired measurements showed a clinically relevant change, which was defined as larger than 0.5 L min−1. The concordance rate was 64.4%.
In this prospective observational study, agreement and trending ability of COAP was compared with COCCUS in critically ill patients with circulatory shock. COAP showed a low bias of 0.2 L min−1 but a large percentage error of 65.6% when compared with COCCUS, indicating disagreement . Trending ability was poor with a concordance rate of 64.4%. The new FloTrac™ algorithm should not be used for diagnosis or guidance of treatment in critically ill patients with circulatory shock.
There are no data on the reliability of CO measurements with the fourth-generation FloTrac™ software algorithm in critically ill patients with shock as of yet. The main concern with the previous version(s) of the APCO algorithm was the lack of reliability in tracking CO changes after hemodynamic interventions or in patients with sepsis [7, 22].
The low bias and the high percentage error of CO measurements are in accordance with results from another study, which tested the fourth-generation algorithm for tracking CO measurements after administration of phenylephrine to increase vasomotor tone in patients prior to cardiac surgery (bias − 0.7 L min−1; percentage error 55.4%) . Concordance rate for trending ability was 87% which was higher than in our study. In that study, the chosen reference technique for measuring CO was thermodilution.
In a more recent study in patients undergoing cardiac surgery, the new FloTrac™ algorithm also showed lack of agreement and trending ability (bias − 0.4 L min−1; percentage error 37.1%; concordance rate 76%) . The reference technique was thermodilution and in that study, bias was influenced by systemic vascular resistance.
Another study tested the fourth-generation FloTrac™ algorithm in patients undergoing abdominal aortic aneurysm surgery and also found a low bias and high percentage error (bias 0.4 L min−1; percentage error 46.7%) of CO measurements . The concordance rate for trending ability was 26.9% before and after aortic clamping and 47.3% before and after first unclamping of the iliac artery. The reference technique chosen in this study was transoesophageal echocardiography.
Advanced hemodynamic monitoring techniques are currently used to identify the type of shock, to guide choices of interventions and to evaluate the response to therapy. Less invasive hemodynamic monitoring techniques such as APCO are currently not recommended for use in patients with shock, especially when receiving vasopressors [2, 26]. Our findings support this statement.
Implications and generalizability
Even though CO monitoring is considered a cornerstone in diagnosing and managing circulatory shock, the sequential evaluation of the hemodynamic state during shock is only a level 1 recommendation based on low quality of evidence .
The abovementioned studies validating the new fourth-generation FloTrac™ algorithm were performed in different target populations and contained different reference techniques, which limit comparability. There is a concern about the interchangeability of COCCUS and CO measurements by thermodilution, and tracking ability of the two methods has only been scarcely assessed and needs evaluation by larger studies .
Considerations and limitations
There are several considerations and limitations when interpreting the results of our study. First, since only parallel and no serial CO measurements were performed for each time point, the precision of individual measurements could not be assessed. While only few studies determined the precision of the CCUS and FloTrac™ technologies, it is a given that both methods have some degree of variation which influences precision of agreement . This might influence—and possibly overestimate—the observed bias and precision to an unknown extent, since the precision of the CCUS as reference method was not incorporated.
Second, a stepwise approach and checklist for the complete presentation of CO method comparison research have been published . This checklist includes a design study phase where it is encouraged that criteria for acceptable bias and LOA or percentage error are defined, and a sample size calculation should be performed prior to the conduct of method comparison studies. In our study, we defined clinically acceptable limits based on available literature, but we did not specify a sample size in advance. The current study could serve as a pilot for a further validation study.
Third, during the study period, we included only 17 patients. Patients with circulatory shock were eligible only if they were expected to stay for longer than 48 h and if it was possible to perform CCUS. We chose this definition to ensure that a complete picture of shock treatment could be presented which allowed for the best comparison between the two methods. Last, CCUS was used as a reference technique in our study despite pulmonary or transpulmonary thermodilution being the gold standard for CO method comparison studies . Therefore, we cannot prove direct superiority of either method. In order to do this, a comparison with a thermodilution method will have to be performed. We chose CCUS as reference because it is currently the first-line evaluation modality in patients with circulatory shock and also because it is widely available and used in the ICU for diagnostic purposes [2, 29]. However, images required to make COCCUS measurements are unobtainable in up to 20% of patients .
FloTrac™ measurements of CO are still not recommended in critically ill patients [5, 6], and further clinical studies comparing minimally invasive techniques for CO estimation with a reference technique are needed for further validation of these techniques and also for extending applicability to other types of patients, who were initially not the target population.
In critically ill patients with circulatory shock, there was disagreement and clinically unacceptable trending ability between values of cardiac output obtained by uncalibrated arterial pressure waveform analysis and critical care ultrasonography.
Acute Physiology And Chronic Health Evaluation
Arterial Pressure waveform analysis method to estimate Cardiac Output
Critical care ultrasonography
- COAP :
Cardiac output measured using fourth-generation FloTrac™ algorithm
- COCCUS :
Cardiac output measured using critical care ultrasonography
Intensive care unit
Limits of agreement
Mean arterial pressure
Pulmonary artery catheter
Simple Acute Physiology Score
Simple Intensive Care Studies
STrengthening the Reporting of OBservational studies in Epidemiology
Sakr Y, Reinhart K, Vincent J-L, Sprung CL, Moreno R, Ranieri VM, et al. Does dopamine administration in shock influence outcome? Results of the sepsis occurrence in acutely ill patients (SOAP) study. Crit Care Med. 2006;34:589–97.
Cecconi M, De Backer D, Antonelli M, Beale R, Bakker J, Hofer C, et al. Consensus on circulatory shock and hemodynamic monitoring. Task force of the European Society of Intensive Care Medicine. Intensive Care Med. 2014;40:1795–815.
Clement RP, Vos JJ, Scheeren TWL. Minimally invasive cardiac output technologies in the ICU: putting it all together. Curr Opin Crit Care. 2017;23:302–9.
Saugel B, Wagner JY, Scheeren TWL. Cardiac output monitoring: less invasiveness, less accuracy? J Clin Monit Comput. 2016;30(6):753–5.
De Backer D, Bakker J, Cecconi M, Hajjar L, Liu DW, Lobo S, et al. Alternatives to the Swan–Ganz catheter. Intensive Care Med. 2018;44:730–41.
Saugel B, Vincent J-L. Cardiac output monitoring: how to choose the optimal method for the individual patient. Curr Opin Crit Care. 2018;24(3):165–72.
Slagt C, Malagon I, Groeneveld ABJ. Systematic review of uncalibrated arterial pressure waveform analysis to determine cardiac output and stroke volume variation. Br J Anaesth. 2014;112(4):626–37.
Critchley LA, Lee A, Ho AMH. A critical review of the ability of continuous cardiac output monitors to measure trends in cardiac output. Anesth Analg. 2010;111:1180–92.
Koster G, Van Der Horst ICC. Critical care ultrasonography in circulatory shock. Curr Opin Crit Care. 2017;23:326–33.
Montenij LJ, Buhre WF, Jansen JR, Kruitwagen CL, De Waal EE. Methodology of method comparison studies evaluating the validity of cardiac output monitors: a stepwise approach and checklist. Br J Anaesth. 2016;116:750–8.
von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007;370:1453–7.
American College of Surgeons Committee on Trauma. Advanced life support course for physicians. Chicago: American College of Surgeons; 1993.
Ait-Oufella H, Lemoinne S, Boelle PY, Galbois A, Baudel JL, Lemant J, et al. Mottling score predicts survival in septic shock. Intensive Care Med. 2011;37:801–7.
Zimmerman JE, Kramer AA, McNair DS, Malila FM. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients. Crit Care Med. 2006;34:1297–310.
Gall JR, Lemeshow S, Saulnier F. A new simplified acute physiology score (SAPS II) based on a European/north American multicenter study. JAMA. 1993;270:2957–63.
Hiemstra B, Eck RJ, Koster G, Wetterslev J, Perner A, Pettilä V, et al. Clinical examination, critical care ultrasonography and outcomes in the critically ill: cohort profile of the simple intensive care studies-I. BMJ Open. 2017;7:8–11.
Hiemstra B, Koster G, Wiersema R, Hummel YM, van der Harst P, Snieder H, et al. The diagnostic accuracy of clinical examination for estimating cardiac index in critically ill patients: the Simple Intensive Care Studies-I. Intensive Care Medicine 2019;45(2):190–200.
Lewis JF, Kuo LC, Nelson JG, Limacher MC, Quinones MA. Pulsed Doppler echocardiographic determination of stroke volume and cardiac output: clinical validation of two new methods using the apical window. Circulation. 1984;70:425–31.
Bland JM, Altman DG. Agreement between methods of measurement with multiple observations per individual. J Biopharm Stat. 2007;17:571–82.
Critchley LAH, Critchley JAJH. A meta-analysis of studies using bias and precision statistics to compare cardiac output measurement techniques. J Clin Monit Comput. 1999;15:85–91.
Saugel B, Grothe O, Wagner JY. Tracking changes in cardiac output: statistical considerations on the 4-quadrant plot and the polar plot methodology. Anesth Analg. 2015;121:514–24.
Monnet X, Vaquer S, Anguel N, Jozwiak M, Cipriani F, Richard C, et al. Comparison of pulse contour analysis by Pulsioflex and Vigileo to measure and track changes of cardiac output in critically ill patients. Br J Anaesth. 2015;114:235–43.
Suehiro K, Tanaka K, Mikawa M, Uchihara Y, Matsuyama T, Matsuura T, Funao T, Yamada T, Mori T, Nishikawa K. Improved performance of the fourth-generation FloTrac/Vigileo system for tracking cardiac output changes. J Cardiothorac Vasc Anesth. 2015;29(3):656–62.
Kusaka Y, Ohchi F, Minami T. Evaluation of the fourth-generation FloTrac/Vigileo system in comparison with the intermittent bolus Thermodilution method in patients undergoing cardiac surgery. J Cardiothorac Vasc Anesth. 2018;000:4–11.
Maeda T, Hattori K, Sumiyoshi M, Kanazawa H, Ohnishi Y. Accuracy and trending ability of the fourth-generation FloTrac/Vigileo system™ in patients undergoing abdominal aortic aneurysm surgery. J Anesth. 2018;32(3):387–93.
Teboul JL, Saugel B, Cecconi M, De Backer D, Hofer CK, Monnet X, et al. Less invasive hemodynamic monitoring in critically ill patients. Intensive Care Med. 2016;42:1350–9.
Wetterslev M, Møller-Sørensen H, Johansen RR, Perner A. Systematic review of cardiac output measurements by echocardiography vs. thermodilution: the techniques are not interchangeable. Intensive Care Med. 2016;42:1223–33.
Hapfelmeier A, Cecconi M, Saugel B. Cardiac output method comparison studies: the relation of the precision of agreement and the precision of method. J Clin Monit Comput. 2016;30(2):149–55.
Saugel B, Reese PC, Wagner JY, Buerke M, Huber W, Kluge S, et al. Advanced hemodynamic monitoring in intensive care medicine. Med Klin Intensivmed Notfmed. 2017;113:192–201.
Jensen MB, Sloth E, Larsen KM, Schmidt MB. Transthoracic echocardiography for cardiopulmonary monitoring in intensive care. Eur J Anaesthesiol. 2004;21:700–7.
The SICS Study Group members include the following: project leaders: Geert Koster; Frederik Keus; Iwan CC van der Horst. Research coordinator: Willem Dieperink. Researchers who conducted patient inclusions: Roos Bleijendaal; Yasmin F. Cawale; Ramon P. Clement; Devon Dijkhuizen; Ruben J Eck; Bart Hiemstra; Anja Haker; Casper D.H. Hilbink; Thomas Kaufmann; Martiene Klasen, Manon Klaver; Laura J. Schokking; Victor W. Sikkens; Madelon Vos; Justin Woerlee; Renske Wiersema.
No external funding was obtained for this study. All FloTrac™ sensors were paid for with departmental funding.
Availability of data and materials
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
The local institutional review board (Medisch Ethische Toetsingscommissie, University Medical Center Groningen) approved the study (M15.168207 and M16.193856). Written informed consent was obtained from all patients.
Consent for publication
TWLS received honoraria from Edwards Lifesciences (Irvine, California, USA) for consulting and for giving lectures. The other authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
STROBE Statement-Checklist of items that should be included in reports of cohort studies. Checklist of items that should be included in reports of cohort studies according to the STROBE statement. (DOCX 42 kb)
Table S1. Detailed patient characteristics. Table showing extended patient characteristics to further describe the study population. (DOCX 15 kb)
Table S2. Cardiac output measurements with critical care ultrasonography and FloTrac. Overview of each cardiac output measurement performed with critical care ultrasonography and FloTrac. (DOCX 24 kb)