Skip to main content

Letter the editor: serious methodological concerns about a recently published meta-analysis on oxygen therapy

Abstract

In a recent paper, Chen et al. report the findings of a systematic review with meta-analysis concerning conservative versus conventional oxygen therapy for critically ill patients. We wish to commend the authors for their interest in the matter. However, the authors appear to misquote findings, fail to report results for all specified analyses, do not identify all relevant trials, have post hoc changed the eligibility criteria, and have seemingly switched directions of effects in analyses of secondary outcomes. These issues have led to incorrect conclusions concerning the effects of targeted oxygen therapy in critically ill patients.

To the editor,

We have with interest read the systematic review with meta-analysis concerning the effects of conservative versus conventional oxygen therapy for critically ill patients by Chen et al. [1]. However, we have several concerns relating to the methodology and findings. None of the analyses and figures presented in this letter have been published elsewhere. They were specifically constructed for the purpose of this letter.

In the paper by Chen et al. [1], the mortality rates are erroneously quoted from several trials in the meta-analysis of mortality at longest follow-up. In the paper by Schjørring et al. [2], a mortality of 514/1447 and 529/1441 in the higher and lower group is incorrectly quoted. The correct mortality was 613/1447 and 618/1441, respectively [2]. Mortality in the liberal group in the study by Barrot et al. was 31/102 [3], not 39/102 as stated. Twenty-eight-day mortality for Asfar et al. is quoted despite 90-day mortality is reported in the trial paper [4]. The ICU-mortality in the modified intention-to-treat population for Girardis et al. is quoted although hospital mortality for the intention-to-treat cohort is reported in the trial paper [5]. A revised meta-analysis is presented in Fig. 1. Chen et al. reported the RR as 1.01 (95% CI 0.94–1.09), so there is a slight difference in the 95% CI [1].

Fig. 1
figure 1

Meta-analysis of mortality at the longest follow-up. M-H, Fixed denotes Mantel–Haenszel (M-H) fixed-effect model, CI confidence interval

The literature search is insufficient as the authors fail to identify four relevant papers focusing on: ICU-patients with acute exacerbation of chronic obstructive pulmonary disease [6]; oxygen therapy after cardiac-arrest [7]; normobaric oxygen in stroke patients [8]; and hyperoxaemia in stroke patients [9]. The first paper should have been identified and included in the meta-analysis, whilst the latter three should have been identified and excluded as per their stated exclusion criteria [1]. In their PRISMA-diagram, the authors state that six trials were excluded after full-text review and present the trials along with reasons for exclusions (Additional file 3: Table S1). In the main text and in this table only five trials are quoted. Moreover, the eligibility criteria have been changed post hoc, without justification, now excluding trials with patients at risk of ischaemia or hypoxic encephalopathy. No such criteria are mentioned in the protocol [10].

The authors’ choice of subgroup analysis based on baseline ratios of partial pressure of oxygen to fraction of inspired oxygen (PaO2/FiO2) as according to mild, moderate, and severe acute respiratory distress syndrome (> 200 mmHg, 100–200 mmHg, and < 100 mmHg, respectively) is problematic, as the results from this analysis, specified in the statistical analysis section, are not presented, except for the results from the sensitivity analysis of trials excluding patients with a PaO2/FiO2 ratio < 100 mmHg (in the abstract). In the main text and their Fig. 2, the authors pool three trials all excluding patients with baseline PaO2/FiO2 ratios < 100 mmHg [4, 11] or < 150 mmHg [5]. This selection is inappropriate, as the approximate mean ratios in Mackle et al. were 252 mmHg [12], and in Panwar et al. 247 mmHg [13]. Though both trials did not restrict inclusion based on PaO2/FiO2 ratios, most patients included in these two trials clearly satisfy the criteria for inclusion in the subgroup analysis above. As no baseline PaO2/FiO2 ratios were presented by Girardis et al. [5], no knowledge of severity of respiratory failure can be ascertained. Therefore, this study should be excluded from the subgroup analysis. In the HOT-ICU trial [2], inclusion was not restricted by PaO2/FiO2 ratio, and the median baseline PaO2/FiO2 ratios were approximately 118 mmHg in both groups. However, a substantial proportion of patients had a ratio ≥ 150 mmHg. We acknowledge that cohort-level-based separations may seem to provide easy new knowledge when performing a meta-analysis, but with such heterogenous groups of included patients in each trial, the only reliable answer to risks according to baseline degree of respiratory failure would come from individual-based-separations and access to all trials’ datasets. Below is provided a revised meta-analysis on mortality at longest follow-up stratified on the specified separation of trials (Fig. 2). This clearly changes the conclusion of the subgroup analysis, as the subgroup of trials with reported baseline PaO2/FiO2 ratios > 200 mmHg now produces a statistically non-significant result (and non-significant test for subgroup differences), contrary to the results presented by the authors.

Fig. 2
figure 2

Meta-analysis of mortality at the longest follow-up, separating trials as according to reported baseline PaO2/FiO2 ratios. M-H, Fixed denotes Mantel–Haenszel fixed-effect model, CI confidence interval

Lastly, it appears that the two compared groups have been switched when reporting serious adverse events, despite correct findings are provided in the supplement (Additional file 6) [1]. If inversed, the results are in line with the meta-analysis provided below (Figs. 3, 4, 5). Conclusions based on these analyses now point in the opposite direction as to what was reported by authors, though still statistically insignificant.

Fig. 3
figure 3

Meta-analysis of mesenteric ischaemia at longest follow-up. M-H, Fixed denotes Mantel–Haenszel fixed-effect model, CI confidence interval. Chen et al. reported the RR for mesenteric ischaemia as 1.15 (95% CI 0.73–1.19)

Fig. 4
figure 4

Meta-analysis of pneumonia at longest follow-up. M-H, Fixed denotes Mantel–Haenszel fixed-effect model, CI confidence interval. Chen et al. reported the RR for pneumonia as 0.92 (95% CI 0.72–1.18)

Fig. 5
figure 5

Meta-analysis of stroke at longest follow-up. M-H, Fixed denotes Mantel–Haenszel fixed-effect model, CI confidence interval. Chen et al. reported the RR for stroke as 0.93 (95% CI 0.53–1.63)

Meta-analyses of high-quality trials are considered the highest level of evidence. Thus, the methodology applied needs to be of similar high quality. If not, inappropriate conclusions may be drawn, potentially misguiding clinical practice. In their review and meta-analysis, Chen et al. fail in several crucial domains, thereby presenting incorrect results and conclusions.

Availability of data and materials

Not applicable.

Abbreviations

CI:

Confidence interval

FiO2 :

Fraction of inspired oxygen

HOT-ICU trial:

Handling Oxygenation Targets in the Intensive Care Unit trial

M-H, Fixed:

Mantel–Haenszel fixed-effect model

mmHg:

Millimetres of mercury

PaO2 :

Partial pressure of arterial oxygen

References

  1. 1.

    Chen XL, Zhang BL, Meng C, et al. Conservative oxygen therapy for critically ill patients: a meta-analysis of randomized controlled trials. J Intensive Care. 2021;9:47. https://doi.org/10.1186/s40560-021-00563-7.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  2. 2.

    Schjørring OL, Klitgaard TL, Perner A, Wetterslev J, Lange T, Siegemund M, et al. Lower or higher oxygenation targets for acute hypoxemic respiratory failure. N Engl J Med. 2021;384:1301–11. https://doi.org/10.1056/NEJMoa2032510.

    Article  PubMed  Google Scholar 

  3. 3.

    Barrot L, Asfar P, Mauny F, Winiszewski H, Montini F, Badie J, et al. LOCO2 investigators and REVA research network liberal or conservative oxygen therapy for acute respiratory distress syndrome. N Engl J Med. 2020;382(11):999–1008. https://doi.org/10.1056/NEJMoa1916431.

    CAS  Article  PubMed  Google Scholar 

  4. 4.

    Asfar P, Schortgen F, Boisramé-Helms J, Charpentier J, Guérot E, Megarbane B, et al. Hyperoxia and hypertonic saline in patients with septic shock (HYPERS2S): a two-by-two factorial, multicentre, randomised, clinical trial. Lancet Respir Med. 2017;5:180–90. https://doi.org/10.1016/S2213-2600(17)30046-2.

    CAS  Article  PubMed  Google Scholar 

  5. 5.

    Girardis M, Busani S, Damiani E, Donati A, Rinaldi L, Marudi A, et al. Effect of conservative vs conventional oxygen therapy on mortality among patients in an intensive care unit the oxygen-ICU randomized clinical trial. JAMA J Am Med Assoc. 2016;316:1583–9. https://doi.org/10.1001/jama.2016.11993.

    CAS  Article  Google Scholar 

  6. 6.

    Gomersall CD, Joynt GM, Sa FFA, Care C, Freebairn RC, Lai CKW, et al. Oxygen therapy for hypercapnic patients with chronic obstructive pulmonary disease and acute respiratory failure: a randomised, controlled pilot study. Crit Care Med. 2002;30:113–6. https://doi.org/10.1097/00003246-200201000-00018.

    Article  PubMed  Google Scholar 

  7. 7.

    Jakkula P, Care I, Reinikainen M, Karelia N, Hospital C, Care I, et al. Targeting two different levels of both arterial carbon dioxide and arterial oxygen after cardiac arrest and resuscitation: a randomised pilot trial. Intensive Care Med Springer, Berlin Heidelberg. 2018;44:2112–21. https://doi.org/10.1007/s00134-018-5453-9.

    CAS  Article  Google Scholar 

  8. 8.

    Mazdeh M, Taher A, Torabian S, Seifirad S. Effects of normobaric hyperoxia in severe acute stroke: a randomized controlled clinical trial study. Acta Med Iran. 2015;53(11):676–80 (PMID: 26786987).

    PubMed  Google Scholar 

  9. 9.

    Taher A, Pilehvari Z, Poorolajal J, Aghajanloo M, Mazdeh M, Taher A, et al. Effects of normobaric hyperoxia in severe acute stroke: a randomized controlled clinical trial study. Trauma Mon. 2016;21(1): e26772. https://doi.org/10.5812/traumamon.26772.

    Article  PubMed  PubMed Central  Google Scholar 

  10. 10.

    Huang H, Chen X, Zhang B, Lin H. Conservative versus conventional oxygen therapy for critically ill patients: a Meta-analysis of randomized controlled trials. Inplasy protocol 202070044. 2020 [cited 2021 Aug 22].

  11. 11.

    Yang X, Shang Y, Yuan S. Low versus high pulse oxygen saturation directed oxygen therapy in critically ill patients: a randomized controlled pilot study. J Thorac Dis. 2019;11:4234–40.

    Article  Google Scholar 

  12. 12.

    The ICU-ROX Investigators and the Australian and New Zealand Intensive Care Society Clinical Trials Group. Conservative oxygen therapy during mechanical ventilation in the ICU. N Engl J Med. 2020;382:989–98. https://doi.org/10.1056/NEJMoa1903297.

    Article  Google Scholar 

  13. 13.

    Panwar R, Hardie M, Bellomo R, Barrot L, Eastwood GM, Young PJ, et al. Conservative versus liberal oxygenation targets for mechanically ventilated patients: a pilot multicenter randomized controlled trial. Am J Respir Crit Care Med. 2016;193(1):43–51. https://doi.org/10.1164/rccm.201505-1019OC.

    CAS  Article  PubMed  Google Scholar 

  14. 14.

    Barbateskovic M, Schjørring OL, Russo Krauss S, Jakobsen JC, Meyhoff CS, Dahl RM, et al. Higher versus lower fraction of inspired oxygen or targets of arterial oxygenation for adults admitted to the intensive care unit. Cochrane Database Syst Rev. 2019. https://doi.org/10.1002/14651858.CD012631.pub2.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

None.

Funding

None.

Author information

Affiliations

Authors

Contributions

TLK wrote the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Thomas Lass Klitgaard.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

TLK and OLS are coordinating investigators of the Handling Oxygenation Targets in the Intensive Care Unit (HOT-ICU) trial. FMN and TLK are coordinating investigators of the Handling Oxygenation Targets in COVID-19 (HOT-COVID) trial. BSR is the sponsor and primary investigator of both the HOT-ICU and HOT-COVID trials. All authors are part of a working group currently updating the latest Cochrane review concerned with higher or lower oxygen levels in critically ill patients [14].

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Klitgaard, T.L., Schjørring, O.L., Nielsen, F.M. et al. Letter the editor: serious methodological concerns about a recently published meta-analysis on oxygen therapy. j intensive care 9, 72 (2021). https://doi.org/10.1186/s40560-021-00573-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40560-021-00573-5

Keywords

  • Oxygen
  • Critical care
  • Systematic review
  • Meta-analysis