Next Article in Journal
Effects of Cocoa Polyphenols and Dark Chocolate on Obese Adults: A Scoping Review
Next Article in Special Issue
Prevalence and Impact of Vitamin D Deficiency in Critically Ill Cancer Patients Admitted to the Intensive Care Unit
Previous Article in Journal
Association of the Modified Mediterranean Diet Score (mMDS) with Anthropometric and Biochemical Indices in US Career Firefighters
Previous Article in Special Issue
Oral Nutrition during and after Critical Illness: SPICES for Quality of Care!
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Trial Design in Critical Care Nutrition: The Past, Present and Future

by
Lee-anne S. Chapple
1,2,3,*,
Emma J. Ridley
4,5 and
Marianne J. Chapman
1,2,3
1
Intensive Care Unit, Royal Adelaide Hospital, Adelaide, SA 5000, Australia
2
Discipline of Acute Care Medicine, The University of Adelaide, Adelaide, SA 5005, Australia
3
Centre of Research Excellence in Translating Nutritional Science to Good Health, The University of Adelaide, Adelaide, SA 5005, Australia
4
Australian and New Zealand Intensive Care Research Centre, School of Public Health and Preventative Medicine, Monash University, Melbourne, VIC 3004, Australia
5
Nutrition Department, Alfred Health, Melbourne, VIC 3004, Australia
*
Author to whom correspondence should be addressed.
Submission received: 29 October 2020 / Revised: 19 November 2020 / Accepted: 26 November 2020 / Published: 30 November 2020
(This article belongs to the Special Issue Nutrition and Metabolism in Critical Care)

Abstract

:
The specialty of nutrition in critical care is relatively modern, and accordingly, trial design has progressed over recent decades. In the past, small observational and physiological studies evolved to become small single-centre comparative trials, but these had significant limitations by today’s standards. Power calculations were often not undertaken, outcomes were not specified a priori, and blinding and randomisation were not always rigorous. These trials have been superseded by larger, more carefully designed and conducted multi-centre trials. Progress in trial conduct has been facilitated by a greater understanding of statistical concepts and methodological design. In addition, larger numbers of potential study participants and increased access to funding support trials able to detect smaller differences in outcomes. This narrative review outlines why critical care nutrition research is unique and includes a historical critique of trial design to provide readers with an understanding of how and why things have changed. This review focuses on study methodology, population group, intervention, and outcomes, with a discussion as to how these factors have evolved, and concludes with an insight into what we believe trial design may look like in the future. This will provide perspective on the translation of the critical care nutrition literature into clinical practice.

1. Challenges of Critical Care Nutrition Research

Artificial nutrition is routinely provided to the majority of mechanically ventilated adult patients admitted to the intensive care unit (ICU). Despite this, we have an incomplete understanding about the optimal route, timing, and dose to provide. While there are published international guidelines from Canada, the United States, and Europe [1,2,3], these are largely based on expert opinion or observational data. For example, the American Society of Parenteral and Enteral Nutrition 2016 Critical Care Guidelines provide 95 individual recommendations, of which just three are based on high or moderate-to-high quality evidence (i.e., a high-quality randomised controlled trial (RCT)) and 55 are based on “expert opinion” alone [1]. The lack of evidence is likely due, in part, to the numerous challenges that prevent the conduct of high-quality trials.
First, the critical care population is heterogeneous [4]. However, the metabolic response to injury is not heterogenous [5,6], with individual patient differences in nutrient digestion, absorption, and utilisation variably influenced by a number of factors, including body composition, age, illness/injury severity, co-morbidities, administered medications, and medical interventions such as renal replacement therapy and extracorporeal membrane oxygenation. Despite this, large critical care studies regularly recruit a heterogeneous population with limited ability for adjustment or consideration of the effect that individual patient characteristics have on the interpretation of results. This may occur, in part, for convenience, as it allows for the recruitment of large participant numbers. In addition, it may reflect that clinicians often apply feeding in a universal manner across the patient population; how different patient populations may respond to therapies is unknown.
Second, critical care nutrition trials are difficult, and sometimes impossible, to conduct blindly, particularly those relating to the route (enteral versus parenteral) or mode (intermittent versus continuous) of delivery. While many attempts have been made, very few RCTs have managed to achieve rigorous blinding processes that include investigators, clinical staff, participants, and outcome assessors [7]. Another major challenge of nutrition research in any field is the inability to control individual aspects of nutrition without inadvertently affecting others. For example, adjusting one macronutrient means affecting either calorie delivery or another macronutrient; therefore, it can be difficult to ascertain what is driving the final result.
Patient-centred outcomes that are likely to be influenced by nutrition are long term, potentially subjective, and; hence, difficult to quantify. Pharmaceutical trials exploring blood pressure medication can quantitatively and accurately ascertain the effect on blood pressure (a conveniently and accurately measured known surrogate for survival); however, nutrition outcomes are less tangible, with no identifiable surrogate. Mortality or calorie delivery are most commonly used [8]. An outcome measure likely to be related to nutrition is functional recovery, but the best way to quantify this objectively has yet to be identified. Furthermore, the benefit of nutrition, particularly on outcomes such as survival, is likely to be small and, therefore, difficult to detect [9]. Correspondingly, while core outcome sets exist for other areas of critical care, they do not exist in the field of nutrition. How these aspects have been managed in trial design and evolved over time is discussed in this review (summarised in Table 1), with the timeframes defined as past (pre-2010), present (2010–2020), and future (2020 and beyond).

2. Study Design in the Past

2.1. Methodology

Early critical care nutrition research was largely observational, initially measuring and documenting patients’ clinical progress and later quantifying aspects of physiology and pathophysiology, such as the measurement of energy expenditure and urinary nitrogen loss. Observational research remains important as it helps us identify, characterise, and quantify clinical problems. Early critical care nutrition studies taught us that the catabolic process of critical illness results in a body mass loss and muscle breakdown, and that this is associated with an excess of nitrogen loss in the urine, indicating protein breakdown [10]. In addition, critically ill patients were shown to have normal or above normal energy expenditure [11], which increased with greater injury severity [12].
In the more recent past (prior to 10 years ago), small single-centre interventional trials were undertaken. These were somewhat limited, given our understanding today, but nevertheless provided preliminary data on which to base subsequent research. In these trials, study methodology was not publicly described prior to analysis or publication, randomisation was not always undertaken and was not necessarily performed optimally, blinding and allocation concealment were not always maintained, outcomes were poorly defined and often not identified prior to data analysis, and statistical analyses were undeveloped compared to current approaches. These limitations were partly due to a lack of understanding of study methodology, as well as an inability to perform large clinical trials due to lack of resources and limited access to appropriately sized patient groups.

2.2. Population

Early critical care nutrition trials tended to include specific populations studied at specialist centres or by specialist groups. These included patients with burns [13], traumatic brain injury [14,15,16], or acute respiratory distress syndrome [17]. Convenience samples were frequently used, which likely led to biased results.

2.3. Interventions

Prior to the year 2000, interventional trials concentrated on methods of feeding: such as timing (e.g., early versus later initiation [18]); and route of delivery (e.g., gastric versus jejunal [19,20] and enteral nutrition (EN) versus parenteral nutrition (PN) [21,22]). Studies on macronutrients included protein- versus glucose-based PN [23,24,25,26,27], medium- versus long-chain triglycerides [28,29,30,31], and branched chain amino acids [32,33,34]. Immunonutrition was very popular as it was seen to have the potential to blunt the inflammatory response [17,35,36,37,38]. Many initiatives were driven by industry, rather than investigator-initiated [37].
Interventions in the next decade (2000–2010) continued with some of the same themes, but also started to look at different processes for delivering feeding, including the effects of feeding according to evidence-based algorithms [39,40] and continuous versus bolus feeding [41]. Immunonutrition continued to be popular, but some investigators started to look at different lipids, as well as at other nutritional additives such as probiotics [42,43].

2.4. Outcomes

Physiologically-based, short-term surrogate outcomes that had not yet been linked to important clinical outcomes were frequently used. These can be categorised into nutritional outcomes (e.g., calorie delivery, nitrogen balance, pre-albumin, and fibronectin concentrations), markers of inflammation (e.g., T cell numbers and C-reactive protein levels), and metabolic indices, (e.g., catecholamine levels). While it was assumed that an improvement in one of these surrogate outcomes may have a beneficial effect on survival, muscle mass, or clinical outcomes, this had not been confirmed. Later trials measured clinical outcomes such as infections, duration of ventilation, length of stay in the ICU, and survival, but many were not adequately powered to detect an effect.

2.5. Interpretation of Research and Translation into Clinical Practice

A further limitation was the way medical evidence was interpreted and translated into clinical practice. For example, small observational studies were able to demonstrate that high calorie delivery (3000–4000 calories/day) from glucose and protein reduced urinary nitrogen loss. It was consequently assumed that this would also reduce muscle mass loss, and thus the practise was widely introduced without an understanding of whether this had positive or deleterious effects on other parameters, particularly patient-centred outcomes such as survival or recovery. Another widespread assumption was that energy delivery should match energy expenditure. With this in mind, and because the measurement of energy expenditure was not easy in a clinical setting, multiple equations were derived to calculate energy expenditure, none of which proved reliably accurate in subsequent comparative studies. Progress in understanding of trial methodology and statistical analysis has occurred alongside progress in medical therapies, and this has formed the basis for advances in research processes.

3. Study Design in the Present

Over the last 10 years, research into critical care nutrition has become a focus for large research groups with international reputations for well-conducted, high-quality trials. Recognising the pitfalls of previous work and the predominance of observational evidence or underpowered studies in guiding practice, it became obvious that these smaller, less robust studies generated hypotheses that needed to be tested in a more rigorous way. In addition, trialists brought lessons from the conduct of high-quality clinical research in other areas to the field of critical care nutrition.

3.1. Methodology

In the past 10 years, at least nine phase III trials investigating nutrition intervention in critical care have been published by leading investigators in the United Kingdom, Australia and New Zealand, Saudi Arabia, the United States, Canada, and Europe [7,44,45,46,47,48,49,50,51]. The key features of these trials are the large sample sizes (often in the thousands), the methodological quality (specifically pilot work, pre-specified protocols, statistical analysis plans, blinding, and randomisation), and rigorous trial management procedures including data safety monitoring committees and interim analyses with a priori defined stopping rules.

3.2. Population

The majority of recently conducted trials have focused on a heterogenous critically ill population, as studies involving specific populations are challenging in terms of recruitment and are hence underpowered. Moreover, subgroup analyses of the larger RCTs, while underpowered to give a definitive answer for the intervention, have been conducted to elucidate populations that may benefit from further study. In the most recent trials, a focus on specific populations with more carefully defined inclusion criteria is starting to emerge; however, this is still an area necessitating greater focus [46,52], and the effect on generalisability requires consideration.

3.3. Interventions

In keeping with the inclusion of heterogenous populations, the majority of recently conducted trials have addressed general questions regarding critical care nutrition practices (e.g., calorie targets and EN versus PN). An issue with this work is the duration of the interventions, being delivered in the acute early phase of critical illness for around 4–7 days in the majority of cases, which may reflect the brief period patients receive artificial nutrition support in the ICU compared to the overall hospital admission. This is likely associated with the populations being studied (general ICU patients), for whom the average length of stay is short (and decreasing year on year in developed countries). This has implications for an intervention such as nutrition, where, unlike that of a drug, the impact is not immediate; a patient’s “nutrition journey” does not end at ICU discharge, and a nutritional intervention applied for just a few days may not be expected to have an influence on clinical outcomes. Similarly, trials have largely treated nutrition like a pharmacological intervention or fluid, with little consideration for its complexity. What these studies have provided is high-quality evidence regarding the impact of short-term nutrition interventions in heterogenous critically ill patients for the early part of their critical illness (i.e., the time they are fed in the ICU). This provides a crucial platform for future work.

3.4. Outcomes

As is the case for study methodology, concepts from other areas of critical care medicine have been utilised in the nutrition field, with the recent inclusion of more robust clinically focused outcomes. These outcomes are ideally objective (e.g., mortality) and important to patients, clinicians, and researchers alike. However, the plausibility of whether short-term nutrition interventions are likely to influence a much later outcome should be considered. Furthermore, the effect that a nutrition intervention may have on a clinical outcome such as mortality is likely to be small, making the required sample size large, and meaning that most previous RCTs (even if well designed and conducted) may have been underpowered for the primary outcome [53]. More recently, trials have used markers of muscle change as a primary outcome, a sign that investigators are starting to consider the biological plausibility of the intervention on the outcome [54,55,56]. However, it must be acknowledged that the trials that have included these alternate outcomes are relatively small, a link between muscle change and clinically important outcomes is required, and the lack of standardised procedures used to conduct these measurements is likely to effect the appropriate interpretation of results [57].

4. Study Design in the Future

Critical care nutrition trial design is evolving rapidly, with innovative study designs, greater consideration of duration and timing of, and individual responses to, interventions, and incorporation of patient-centred outcomes.

4.1. Methodology

As survival rates in ICUs improve, the ability for a single intervention to demonstrate a beneficial effect on mortality will likely require a very large sample size (i.e., tens of thousands of participants) [58]. Nutrition interventions may be beneficial, but the measurable effects are likely to be small because of the numerous factors, both patient- and treatment-related, affecting outcomes in critically ill patients. One of the largest critical care nutrition studies to date, which randomised just under 4000 patients across 46 ICUs, was powered to detect a 3.8%–4.3% mortality difference and cost upwards of 4.5 million dollars (Australian) to conduct. Even smaller survival benefits would be useful to determine, but with a parallel RCT design in a heterogenous population, this would require very large sample sizes, and hence would likely be logistically or financially unfeasible. Trial design needs to adapt accordingly, and future critical care nutrition research is likely to see more innovative and sophisticated methodology. The use of registry data or cluster randomisation may facilitate greater patient numbers [59]; however, this comes with limitations in relation to the restricted data points and availability of key nutrition data [60]. The first multi-centre randomised critical care nutrition trial to use registry data (EFFORT trial) is currently recruiting study subjects to ascertain the influence of higher protein doses on mortality [61]. Other study methodologies that may lend themselves to nutrition interventions are adaptive or Bayesian-style designs that allow iterative changes in response to therapy benefits that arise during the trial [62,63]. While these are currently being explored in other critical care interventions [64], identification of an important short-term outcome likely to be influenced by nutrition is needed prior to consideration of these types of methodologies for critical care nutrition.
Large randomised trials are unlikely to demonstrate positive results if they are not prefaced by rigorous preliminary research; thus, large trials should be preceded by detailed mechanistic work embedded within the research programme rather than a siloed research stream [65,66]. As trials become more complex, the use of artificial intelligence [67], as well as the involvement of experienced clinician-researchers and biostatisticians, will likely be integral to study design and data interpretation. In addition, greater transparency of study methodology should include a pre-published protocol and statistical analysis plan, trial registration, and clear reporting of both results and data analysis, including those for post-hoc analyses. Access to open-source data after publication should also be considered to allow for analysis by an independent statistician to assess reproducibility.

4.2. Population

Future critical care nutrition research is likely to place a greater focus on specific clinical populations, with greater consideration of individual patient characteristics to enable personalised or individualised therapy and to account for factors such as pre-morbid nutrition status, body composition, and stage of disease. A minimum common dataset of patient population characteristics to be reported may support the advancement of this, as too may the use of metabolic biomarkers to determine those patients most likely to benefit from nutrition support [68,69]. Though nutrition risk scores have been developed, they have been poorly validated, are often inaccurate, and may reflect injury severity more than nutrition risk [70], which may reduce the number of patients needing to receive the intervention in order to see benefit.
Furthermore, as medical therapy advances, so too must nutrition therapy. Management strategies for conditions such as acute kidney injury, delirium, and burns may affect nutrition requirements or delivery, and hence nutrition support specific to these conditions requires exploration. There is also a growing population of patients that are either never mechanically ventilated or ventilated for a short period of their ICU stay only, yet international nutrition guidelines provide limited recommendations specific for this population and; hence, the nutritional needs of this population must be ascertained.

4.3. Interventions

To date, critical care nutrition interventions have been provided while the patient remains within the ICU setting [71]. Yet this reflects just a short phase of the patient’s hospitalisation, and post-ICU nutrition intake has recently been shown to be poor [72,73]. We are starting to see more focus on what happens to critically ill patients after ICU discharge (INTENT trial, clinicaltrials.gov: NCT03292237). In addition, it is important that the timing of an intervention takes into account the stage of illness and metabolic state rather than simple chronology, and includes reference to identifying the “metabolic switch” for ICU patients [69], as well as recognising the patient’s recovery trajectory in order to intervene appropriately [74]. This has been identified in the general ICU literature stating that the changing pathophysiology of a disease within the same patient needs to be considered [58]. A key aspect of this is the careful documentation and appropriate analysis of the duration during which the target intervention was received, taking into consideration the progressive delivery of nutrition in order to reach goal rates early while in the ICU, as well as weaning from nutritional support, a concept well described previously [75,76].
As nutrition knowledge progresses, we may start to see more individualised treatments, such as patient-specific nutrition formulations (e.g., chemotherapy specifically blended for the best outcome), that take into consideration individual requirements for fluid, volume, and macro- and micro-nutrients, and that change over time as individual needs evolve [77]. Bundled or synergistic therapies should also be considered, such as nutrition and physical therapy [69,78].

4.4. Outcomes

To date, large phase III RCTs in critically ill patients have used mortality as the primary outcome [9,79], yet few have reported benefit from general ICU studies [80] or nutrition-specific interventions [8]. In the next decade, we are likely to see greater emphasis placed on what is important to patients, (e.g., functional, social, and psychological recovery), increased consumer engagement [81], and incorporation of co-design that includes not only clinicians/researchers, but also patients, in the trial design [82]. The use of technology such as smartphones or wearable devices will also allow greater analysis of objective outcome measures. The inclusion of body composition outcomes has been highlighted in international research agenda [83], and technology such as continuous glucose monitors that allow minute-by-minute data points affords immediate responses to nutrition therapies in need of assessment.

5. Conclusions

In the last decade critical care nutrition research has evolved rapidly. Early physiological studies, observational data, and small comparative trials have made way for large, definitive, rigorously conducted RCTs. Each trial has added incremental evidence that has made clinical practice what it is today. A number of research agendas exist that provide tangible guidance on the future research questions considered of utmost importance in the critical care nutrition field, as well as methodological challenges to be considered by clinical researchers and trialists [6,69]. Future trials will have new challenges. They must embrace technology and intelligent study design in order to cost-effectively recruit larger numbers of patients or to target those most likely to benefit in the critical phase of their hospital journey. Critical care nutrition research will continue to advance, providing clinicians and researchers alike with a greater understanding of the nutrition practices that will ultimately deliver benefit for their patients.

Author Contributions

All authors contributed to writing—original draft preparation. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding. L.S. Chapple was supported by a National Health and Medical Research Council Early Career Fellowship, and E.J. Ridley was supported by a National Health and Medical Research Council Emerging Leader 1 Fellowship.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. McClave, S.A.; Taylor, B.A.; Martindale, R.; Warren, M.M.; Johnson, D.R.; Braunschweig, C.; McCarthy, M.C.; Davanos, E.; Rice, T.W.; Cresci, G.A.; et al. Guidelines for the Provision and Assessment of Nutrition Support Therapy in the Adult Critically Ill Patient: Society of Critical Care Medicine (SCCM) and American Society for Parenteral and Enteral Nutrition (A.S.P.E.N.). J. Parenter. Enter. Nutr. 2016, 40, 159–211. [Google Scholar] [CrossRef] [PubMed]
  2. Singer, P.; Blaser, A.R.; Berger, M.M.; Alhazzani, W.; Calder, P.C.; Casaer, M.P.; Hiesmayr, M.; Mayer, K.; Montejo, J.C.; Pichard, C.; et al. ESPEN guideline on clinical nutrition in the intensive care unit. Clin. Nutr. 2019, 38, 48–79. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Chapple, L.-A.S.; Deane, A.M.; Williams, L.T.; Strickland, R.; Schultz, C.; Lange, K.; Heyland, D.K.; Chapman, M.J. Longitudinal changes in anthropometrics and impact on self-reported physical function after traumatic brain injury. Crit. Care Resusc. J. Australas. Acad. Crit. Care Med. 2017, 19, 29–36. [Google Scholar]
  4. MacLennan, G.; Campbell, M.; Norrie, J. Methodological challenges designing pragmatic, multi-centre randomised controlled trials in critical care. Trials 2013, 14, O125. [Google Scholar] [CrossRef] [Green Version]
  5. Preiser, J.-C.; Ichai, C.; Orban, J.-C.; Groeneveld, A.B.J. Metabolic response to the stress of critical illness. Br. J. Anaesth. 2014, 113, 945–954. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Wernerman, J.; Christopher, K.B.; Annane, D.; Casaer, M.P.; Coopersmith, C.M.; Deane, A.M.; De Waele, E.; Elke, G.; Ichai, C.; Karvellas, C.J.; et al. Metabolic support in the critically ill: A consensus of 19. Crit. Care 2019, 23, 1–10. [Google Scholar] [CrossRef] [Green Version]
  7. Target Investigators for the ANZICS Clinical Trials Group; Chapman, M.; Peake, S.L.; Bellomo, R.; Davies, A.; Deane, A.; Horowitz, M.; Hurford, S.; Lange, K.; Little, L.; et al. Energy-dense versus routine enteral nutrition in the critically ill. N. Engl. J. Med. 2018, 379, 1823–1834. [Google Scholar] [CrossRef]
  8. Chapple, L.-A.S.; Summers, M.J.; Weinel, L.M.; Deane, A.M. Outcome Measures in Critical Care Nutrition Interventional Trials: A Systematic Review. Nutr. Clin. Pract. 2020, 35, 506–513. [Google Scholar] [CrossRef]
  9. Harhay, M.O.; Wagner, J.; Ratcliffe, S.J.; Bronheim, R.S.; Gopal, A.; Green, S.; Cooney, E.; Mikkelsen, M.E.; Kerlin, M.P.; Small, D.S.; et al. Outcomes and Statistical Power in Adult Critical Care Randomized Trials. Am. J. Respir. Crit. Care Med. 2014, 189, 1469–1478. [Google Scholar] [CrossRef]
  10. Riegel, C.; Koop, C.E.; Drew, J.; Stevens, L.W.; Rhoads, J.E.; Bullitt, L.; Barrus, D.; Grigger, R.P.; Barnes, M.; Barnhart, A.; et al. The Nutritional Requirements for Nitrogen Balance in Surgical Patients During the Early Postoperative Period 1. J. Clin. Investig. 1947, 26, 18–23. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  11. Cuthbertson, D.P. Observations on the Disturbance of Metabolism Produced by Injury to the Limbs. QJM Int. J. Med. 1932, 1, 233–246. [Google Scholar] [CrossRef]
  12. Kreymann, G.; Grosser, S.; Buggisch, P.; Gottschall, C.; Matthaei, S.; Greten, H. Oxygen consumption and resting metabolic rate in sepsis, sepsis syndrome, and septic shock. Crit. Care Med. 1993, 21, 1012–1019. [Google Scholar] [CrossRef] [PubMed]
  13. Herndon, D.; Stein, M.D.; Rutan, T.C.; Abston, S.; Linares, H. Failure of TPN Supplementation to Improve Liver Function, Immunity, and Mortality in Thermally Injured Patients. J. Trauma Inj. Infect. Crit. Care 1987, 27, 195–204. [Google Scholar] [CrossRef] [PubMed]
  14. McNamara, J.J.; Molot, M.D.; Wissman, D.; Collins, C.; Stremple, J.F. Intravenous hyperalimentation. An important adjunct in the treatment of combat casualties. Am. J. Surg. 1971, 122, 70–73. [Google Scholar] [PubMed]
  15. Allison, S.P. Comparison of proprietary elemental and whole-protein diets in unconscious patients with head injury. BMJ 1980, 281, 390. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  16. Twyman, D.; Young, A.B.; Ott, L.; Norton, J.A.; Bivins, B.A. High Protein Enteral Feedings: A Means of Achieving Positive Nitrogen Balance in Head Injured Patients. J. Parenter. Enter. Nutr. 1985, 9, 679–684. [Google Scholar] [CrossRef] [PubMed]
  17. Gadek, J.E.; DeMichele, S.J.; Karlstad, M.D.; Pacht, E.R.; Donahoe, M.; Albertson, T.E.; Van Hoozen, C.; Wennberg, A.K.; Nelson, J.L.; Noursalehi, M. Effect of enteral feeding with eicosapentaenoic acid, gamma-linolenic acid, and antioxidants in patients with acute respiratory distress syndrome. Crit. Care Med. 1999, 27, 1409–1420. [Google Scholar] [CrossRef]
  18. Eyer, S.D.; Micon, L.T.; Konstantinides, F.N.; Edlund, D.A.; Rooney, K.A.; Luxenberg, M.G.; Cerra, F.B. Early Enteral Feeding Does Not Attenuate Metabolic Response after Blunt Trauma. J. Trauma Inj. Infect. Crit. Care 1993, 34, 639–644. [Google Scholar] [CrossRef]
  19. Demetriou, A.A. Nutritional Outcome and Pneumonia in Critical Care Patients Randomized to Gastric versus Jejunal Tube Feedings. J. Parenter. Enter. Nutr. 1993, 17, 191–192. [Google Scholar] [CrossRef]
  20. Montecalvo, M.A.; Steger, K.A.; Farber, H.W.; Smith, B.F.; Dennis, R.C.; Fitzpatrick, G.F.; Pollack, S.D.; Korsberg, T.Z.; Birkett, D.H.; Hirsch, E.F.; et al. Nutritional outcome and pneumonia in critical care patients randomized to gastric versus jejunal tube feedings. Crit. Care Med. 1992, 20, 1377–1387. [Google Scholar] [CrossRef]
  21. Hadfield, R.J.; Sinclair, D.G.; Houldsworth, P.E.; Evans, T.W. Effects of enteral and parenteral nutrition on gut mucosal permeability in the critically ill. Am. J. Respir. Crit. Care Med. 1995, 152, 1545–1548. [Google Scholar] [CrossRef] [PubMed]
  22. Moore, F.A.; Moore, E.E.; Jones, T.N.; McCroskey, B.L.; Peterson, V.M. TEN versus TPN following Major Abdominal Trauma—Reduced Septic Morbidity. J. Trauma Inj. Infect. Crit. Care 1989, 29, 916–923. [Google Scholar] [CrossRef] [PubMed]
  23. Ziegler, F.; Ollivier, J.M.; Cynober, L.; Masini, J.P.; Coudray-Lucas, C.; Levy, E.; Giboudeau, J. Efficiency of enteral nitrogen support in surgical patients: Small peptides v non-degraded proteins. Gut 1990, 31, 1277–1283. [Google Scholar] [CrossRef] [PubMed]
  24. Heimburger, D.C.; Geels, W.J.; Bilbrey, J.; Redden, D.T.; Keeney, C. Effects of small-peptide and whole-protein enteral feedings on serum proteins and diarrhea in critically ill patients: A randomized trial. J. Parenter. Enter. Nutr. 1997, 21, 162–167. [Google Scholar] [CrossRef] [PubMed]
  25. Kalfarentzos, F.; Kokkinis, K.; Leukaditi, K.; Maroulis, J.; Onoufriou, A.; Alexopoulos, K. Comparison between two fat emulsions: Intralipid 30 cent vs intralipid 10 cent in critically ill patients. Clin. Nutr. 1998, 17, 31–34. [Google Scholar] [CrossRef]
  26. Tappy, L.; Schwarz, J.-M.; Schneiter, P.; Cayeux, C.; Revelly, J.-P.; Fagerquist, C.K.; Jequier, E.; Chiolero, R. Effects of isoenergetic glucose-based or lipid-based parenteral nutrition on glucose metabolism, de novo lipogenesis, and respiratory gas exchanges in critically ill patients. Crit. Care Med. 1998, 26, 860–867. [Google Scholar] [CrossRef]
  27. Iapichino, G.; Gattinoni, L.; Solca, M.; Radrizzani, D.; Zucchetti, M.; Langer, M.; Vesconi, S. Protein sparing and protein replacement in acutely injured patients during TPN with and without amino acid supply. Intensive Care Med. 1982, 8, 25–31. [Google Scholar] [CrossRef]
  28. Ball, M.J.; White, K. Metabolic effects of intravenous medium- and long-chain triacylglycerols in critically ill patients. Clin. Sci. 1989, 76, 165–170. [Google Scholar] [CrossRef]
  29. Calon, B.; Pottecher, T.; Frey, A.; Ravanello, J.; Otteni, J.C.; Bach, A.C. Long-chain versus medium and long-chain triglyceride-based fat emulsion in parental nutrition of severe head trauma patients. Infusionstherapie 1990, 17, 246–248. [Google Scholar]
  30. Ball, M.J.; Ball, M.J. Parenteral nutrition in the critically ill: Use of a medium chain triglyceride emulsion. Intensive Care Med. 1993, 19, 89–95. [Google Scholar] [CrossRef]
  31. Planas, M.; Porta, I.; Sagristá, M.L.; Mora, M.; Padró, J.B.; Picó, M. Fatty acid composition of platelet membrane lipids after administration of two different fat emulsions in critically ill patients. Intensive Care Med. 1999, 25, 395–398. [Google Scholar] [CrossRef] [PubMed]
  32. Woude, P.V.; Morgan, R.E.; Kosta, J.M.; Davis, A.T.; Scholten, D.J.; Dean, R.E. Addition of branched-chain amino acids to parenteral nutrition of stressed critically ill patients. Crit. Care Med. 1986, 14, 685–688. [Google Scholar] [CrossRef] [PubMed]
  33. Cerra, F.B.; Mazuski, J.; Teasley, K.; Nuwer, N.; Lysne, J.; Shronts, E.; Konstantinides, F. Nitrogen retention in critically ill patients is proportional to the branched chain amino acid load. Crit. Care Med. 1983, 11, 775–778. [Google Scholar] [CrossRef]
  34. Schmitz, J.E.; Dölp, R.; Grünert, A.; Ahnefeld, F.W. Effect of solutions with different amounts of branched-chain amino acids on the plasma amino acid pattern and energy metabolism of surgical intensive care patients. Infus. Klin. Ernahr. 1982, 9, 100–107. [Google Scholar]
  35. Mendez, C.; Jurkovich, G.J.; Garcia, I.; Davis, D.; Parker, A.; Maier, R.V. Effects of an Immune-enhancing Diet in Critically Injured Patients. J. Trauma Inj. Infect. Crit. Care 1997, 42, 933–941. [Google Scholar] [CrossRef]
  36. Jones, C.; Palmer, T.; Griffiths, R.D. Randomized clinical outcome study of critically ill patients given glutamine-supplemented enteral nutrition. Nutrition 1999, 15, 108–115. [Google Scholar] [CrossRef]
  37. Atkinson, S.; Sieffert, E.; Bihari, D. A prospective, randomized, double-blind, controlled clinical trial of enteral immunonutrition in the critically ill. Crit. Care Med. 1998, 26, 1164–1172. [Google Scholar] [CrossRef]
  38. Cerra, F.B.; Lehmann, S.; Konstantinides, N.; Dzik, J.; Fish, J.; Konstantinides, F.; Licari, J.J.; Holman, R.T. Improvement in immune function in ICU patients by enteral nutrition supplemented with arginine, RNA, and menhaden oil is independent of nitrogen balance. Nutrition 1991, 7, 193–199. [Google Scholar] [PubMed]
  39. Doig, G.S.; Simpson, F.; Finfer, S.; Delaney, A.; Davies, A.R.; Mitchell, I.; Dobb, G. Effect of evidence-based feeding guidelines on mortality of critically ill adults: A cluster randomized controlled trial. JAMA 2008, 300, 2731–2741. [Google Scholar] [CrossRef]
  40. Martin, C.M.; Doig, G.S.; Heyland, D.K.; Morrison, T.; Sibbald, W.J. Multicentre, cluster-randomized clinical trial of algorithms for critical-care enteral and parenteral therapy (ACCEPT). Can. Med. Assoc. J. 2004, 170, 197–204. [Google Scholar] [CrossRef] [Green Version]
  41. Serpa, L.F.; Kimura, M.; Faintuch, J.; Ceconello, I. Effects of continuous versus bolus infusion of enteral nutrition in critical patients. Rev. Hosp. Clín. 2003, 58, 9–14. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  42. De Arruda, I.S.F.; De Aguilar-Nascimento, J.E. Benefits of early enteral nutrition with glutamine and probiotics in brain injury patients. Clin. Sci. 2004, 106, 287–292. [Google Scholar] [CrossRef]
  43. Alberda, C.; Gramlich, L.; Meddings, J.; Field, C.J.; McCargar, L.; Kutsogiannis, D.; Fedorak, R.N.; Madsen, K. Effects of probiotic therapy in critically ill patients: A randomized, double-blind, placebo-controlled trial. Am. J. Clin. Nutr. 2007, 85, 816–823. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  44. Heyland, D.; Muscedere, J.; Wischmeyer, P.E.; Cook, D.; Jones, G.; Albert, M.; Elke, G.; Berger, M.M.; Day, A.G. A randomized trial of glutamine and antioxidants in critically ill patients. N. Engl. J. Med. 2013, 368, 1489–1497. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  45. Harvey, S.E.; Parrott, F.; Harrison, D.A.; Sadique, M.Z.; Grieve, R.D.; Canter, R.R.; McLennan, B.K.; Tan, J.C.; Bear, D.E.; Segaran, E.; et al. A multicentre, randomised controlled trial comparing the clinical effectiveness and cost-effectiveness of early nutritional support via the parenteral versus the enteral route in critically ill patients (CALORIES). Health Technol. Assess. 2016, 20, 1–144. [Google Scholar] [CrossRef] [PubMed]
  46. Reignier, J.; Boisramé-Helms, J.; Brisard, L.; Lascarrou, J.-B.; Hssain, A.A.; Anguel, N.; Argaud, L.; Asehnoune, K.; Asfar, P.; Bellec, F.; et al. Enteral versus parenteral early nutrition in ventilated adults with shock: A randomised, controlled, multicentre, open-label, parallel-group study (NUTRIREA-2). Lancet 2018, 391, 133–143. [Google Scholar] [CrossRef]
  47. Arabi, Y.M.; Aldawood, A.S.; Haddad, S.H.; Al-Dorzi, H.M.; Tamim, H.M.; Jones, G.; Mehta, S.; McIntyre, L.; Solaiman, O.; Sakkijha, M.H.; et al. Permissive Underfeeding or Standard Enteral Feeding in Critically Ill Adults. N. Engl. J. Med. 2015, 372, 2398–2408. [Google Scholar] [CrossRef]
  48. Casaer, M.P.; Mesotten, D.; Hermans, G.; Wouters, P.J.; Schetz, M.; Meyfroidt, G.; Van Cromphaut, S.; Ingels, C.; Meersseman, P.; Muller, J. Early versus late parenteral nutrition in critically ill adults. N. Engl. J. Med. 2011, 365, 506–517. [Google Scholar] [CrossRef] [Green Version]
  49. Heidegger, C.-P.; Berger, M.M.; Graf, S.; Zingg, W.; Darmon, P.; Costanza, M.C.; Thibault, R.; Pichard, C. Optimisation of energy provision with supplemental parenteral nutrition in critically ill patients: A randomised controlled clinical trial. Lancet 2013, 381, 385–393. [Google Scholar] [CrossRef]
  50. Rice, T.W.; Wheeler, A.P.; Thompson, B.T.; Steingrub, J.S.; Hite, R.D.; Moss, M.; Morris, A.H.C.; Dong, N.; Rock, P. Initial Trophic vs Full Enteral Feeding in Patients With Acute Lung Injury: The EDEN Randomized Trial. JAMA 2012, 307, 795–803. [Google Scholar] [CrossRef] [Green Version]
  51. Davies, A.; Morrison, S.S.; Bailey, M.J.; Bellomo, R.; Cooper, D.J.; Doig, G.S.; Finfer, S.; Heyland, D.K. A multicenter, randomized controlled trial comparing early nasojejunal with nasogastric nutrition in critical illness. Crit. Care Med. 2012, 40, 2342–2348. [Google Scholar] [CrossRef]
  52. Heyland, D.K.; Dhaliwalm, R.; Day, A.; Drover, J.; Cote, H.; Wischmeyer, P. Optimizing the Dose of Glutamine Dipeptides and Antioxidants in Critically Ill Patients: A Phase I Dose-Finding Study. J. Parenter. Enter. Nutr. 2007, 31, 109–118. [Google Scholar] [CrossRef]
  53. Summers, M.J.; Chapple, L.-A.S.; McClave, S.A.; Deane, A.M. Event-rate and delta inflation when evaluating mortality as a primary outcome from randomized controlled trials of nutritional interventions during critical illness: A systematic review. Am. J. Clin. Nutr. 2016, 103, 1083–1090. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  54. McNelly, A.S.; Bear, D.E.; Connolly, B.A.; Arbane, G.; Allum, L.; Tarbhai, A.; Cooper, J.A.; Hopkins, P.A.; Wise, M.P.; Brealey, D.; et al. Effect of intermittent or continuous feed on muscle wasting in critical illness: A Phase 2 clinical trial. Chest 2020, 158, 183–194. [Google Scholar] [CrossRef] [PubMed]
  55. Fetterplace, K.; Deane, A.M.; Tierney, A.; Beach, L.J.; Knight, L.D.; Presneill, J.; Rechnitzer, T.; Forsyth, A.; Gill, B.M.T.; Mourtzakis, M.; et al. Targeted Full Energy and Protein Delivery in Critically Ill Patients: A Pilot Randomized Controlled Trial (FEED Trial). J. Parenter. Enter. Nutr. 2018, 42, 1252–1262. [Google Scholar] [CrossRef] [PubMed]
  56. Casaer, M.P.; Berghe, G.V.D. Comment on Protein Requirements in the Critically Ill: A Randomized Controlled Trial Using Parenteral Nutrition. J. Parenter. Enter. Nutr. 2016, 40, 763. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  57. Weinel, L.M.; Summers, M.J.; Chapple, L.-A.S. Ultrasonography to measure quadriceps muscle in critically ill patients: A literature review of reported methodologies. Anaesth. Intensive Care 2019, 47, 423–434. [Google Scholar] [CrossRef]
  58. Latour-Pérez, J. Clinical research in critical care: Difficulties and perspectives. Med. Intensiva 2018, 42, 184–195. [Google Scholar] [CrossRef]
  59. The PEPTIC Investigators for the Australian and New Zealand Intensive Care Society Clinical Trials Group; Alberta Health Services Critical Care Strategic Clinical Network; Irish Critical Care Trials Group. Effect of stress ulcer prophylaxis with proton pump inhibitors vs histamine-2 receptor blockers on in-hospital mortality among ICU patients receiving invasive mechanical ventilation: The PEPTIC Randomized Clinical Trial. JAMA 2020, 323, 616–626. [Google Scholar] [CrossRef]
  60. Cooke, C.R.; Iwashyna, T.J. Using Existing Data to Address Important Clinical Questions in Critical Care. Crit. Care Med. 2013, 41, 886–896. [Google Scholar] [CrossRef] [Green Version]
  61. Heyland, D.K.; Patel, J.; Bear, D.E.; Sacks, G.; Nixdorf, H.; Dolan, J.; Aloupis, M.; Licastro, K.; Rd, V.J.; Rice, T.W.; et al. The Effect of Higher Protein Dosing in Critically Ill Patients: A Multicenter Registry-Based Randomized Trial: The EFFORT Trial. J. Parenter. Enter. Nutr. 2019, 43, 326–334. [Google Scholar] [CrossRef] [PubMed]
  62. Kalil, A.C.; Sun, J. Bayesian methodology for the design and interpretation of clinical trials in critical care medicine: A primer for clinicians. Crit. Care Med. 2014, 42, 2267–2277. [Google Scholar] [CrossRef] [PubMed]
  63. Yarnell, C.J.; Granton, J.T.; Tomlinson, G. Bayesian Analysis in Critical Care Medicine. Am. J. Respir. Crit. Care Med. 2020, 201, 396–398. [Google Scholar] [CrossRef]
  64. Writing Committee for the REMAP CAP Investigators. Effect of hydrocortisone on mortality and organ support in patients with severe COVID-19: The REMAP-CAP COVID-19 corticosteroid domain randomized clinical trial. JAMA 2020, 324, 1317–1329. [Google Scholar] [CrossRef] [PubMed]
  65. Goodman, S.N.; Gerson, J. Mechanistic Evidence in Evidence-Based Medicine: A Conceptual Framework; Agency for Healthcare Research and Quality: Rockville, MD, USA, 2013. [Google Scholar]
  66. Deaton, A.; Cartwright, N. Understanding and misunderstanding randomized controlled trials. Soc. Sci. Med. 2018, 210, 2–21. [Google Scholar] [CrossRef] [PubMed]
  67. Harrer, S.; Shah, P.; Antony, B.; Hu, J. Artificial Intelligence for Clinical Trial Design. Trends Pharmacol. Sci. 2019, 40, 577–591. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  68. Stoppe, C.; Wendt, S.; Mehta, N.M.; Compher, C.; Preiser, J.-C.; Heyland, D.K.; Kristof, A.S. Biomarkers in critical care nutrition. Crit. Care 2020, 24, 1–10. [Google Scholar] [CrossRef] [PubMed]
  69. Arabi, Y.M.; Casaer, M.P.; Chapman, M.; Heyland, D.K.; Ichai, C.; Marik, P.E.; Martindale, R.G.; McClave, S.A.; Preiser, J.-C.; Reignier, J.; et al. The intensive care medicine research agenda in nutrition and metabolism. Intensive Care Med. 2017, 43, 1239–1256. [Google Scholar] [CrossRef]
  70. Heyland, D.K.; Rd, R.D.; Jiang, X.; Day, A. Identifying critically ill patients who benefit the most from nutrition therapy: The development and initial validation of a novel risk assessment tool. Crit. Care 2011, 15, R268. [Google Scholar] [CrossRef] [Green Version]
  71. Lambell, K.J.; Tatucu-Babet, O.A.; Chapple, L.-A.; Gantner, D.; Ridley, E.J. Nutrition therapy in critical illness: A review of the literature for clinicians. Crit. Care 2020, 24, 1–11. [Google Scholar] [CrossRef] [Green Version]
  72. Chapple, L.-A.S.; Deane, A.M.; Heyland, D.K.; Lange, K.; Kranz, A.J.; Williams, L.T.; Chapman, M.J. Energy and protein deficits throughout hospitalization in patients admitted with a traumatic brain injury. Clin. Nutr. 2016, 35, 1315–1322. [Google Scholar] [CrossRef] [PubMed]
  73. Ridley, E.J.; Parke, R.L.; Davies, A.R.; Bailey, M.; Hodgson, C.L.; Deane, A.M.; McGuinness, S.; Cooper, D.J. What Happens to Nutrition Intake in the Post-Intensive Care Unit Hospitalization Period? An Observational Cohort Study in Critically Ill Adults. J. Parenter. Enter. Nutr. 2019, 43, 88–95. [Google Scholar] [CrossRef] [PubMed]
  74. Iwashyna, T.J. Trajectories of Recovery and Dysfunction after Acute Illness, with Implications for Clinical Trial Design. Am. J. Respir. Crit. Care Med. 2012, 186, 302–304. [Google Scholar] [CrossRef] [Green Version]
  75. Hartl, W.H.; Bender, A.; Scheipl, F.; Kuppinger, D.; Day, A.G.; Küchenhoff, H. Calorie intake and short-term survival of critically ill patients. Clin. Nutr. 2019, 38, 660–667. [Google Scholar] [CrossRef] [PubMed]
  76. Heyland, D.K.; Cahill, N.; Day, A. Optimal amount of calories for critically ill patients: Depends on how you slice the cake! Crit. Care Med. 2011, 39, 2619–2626. [Google Scholar] [CrossRef]
  77. Maslove, D.M.; Lamontagne, F.; Marshall, J.C.; Heyland, D.K. A path to precision in the ICU. Crit. Care 2017, 21, 79. [Google Scholar] [CrossRef] [Green Version]
  78. Heyland, D.K.; Day, A.; Clarke, G.J.; Hough, C.; Files, D.C.; Mourtzakis, M.; Deutz, N.; Needham, D.M.; Stapleton, R. Nutrition and Exercise in Critical Illness Trial (NEXIS Trial): A protocol of a multicentred, randomised controlled trial of combined cycle ergometry and amino acid supplementation commenced early during critical illness. BMJ Open 2019, 9, e027893. [Google Scholar] [CrossRef] [Green Version]
  79. Gaudry, S.; Messika, J.; Ricard, J.-D.; Guillo, S.; Pasquet, B.; Dubief, E.; Boukertouta, T.; Dreyfuss, D.; Tubach, F. Patient-important outcomes in randomized controlled trials in critically ill patients: A systematic review. Ann. Intensive Care 2017, 7, 1–11. [Google Scholar] [CrossRef]
  80. Ospina-Tascon, G.A.; Buchele, G.L.; Vincent, J.L. Multicenter, randomized, controlled trials evaluating mortality in intensive care: Doomed to fail? Crit. Care Med. 2008, 36, 1311–1322. [Google Scholar] [CrossRef]
  81. Burns, K.E.A.; Misak, C.; Herridge, M.; Meade, M.; Oczkowski, S. Patient and Family Engagement in the ICU. Untapped Opportunities and Underrecognized Challenges. Am. J. Respir. Crit. Care Med. 2018, 198, 310–319. [Google Scholar] [CrossRef]
  82. Haines, K.J.; Holdsworth, C.; Cranwell, K.; Skinner, E.H.; Holton, S.; Macleod-Smith, B.; Bates, S.; Iwashyna, T.J.; French, C.; Booth, S.; et al. Development of a Peer Support Model Using Experience-Based Co-Design to Improve Critical Care Recovery. Crit. Care Explor. 2019, 1, e0006. [Google Scholar] [CrossRef] [PubMed]
  83. Compher, C.; Jain, A.K.; Nichol, P.F.; Blackmer, A.; Earthman, C.; Evans, D.C.; McCarthy, M.S.; Taylor, B.E.; Mehta, N. Research Agenda 2018: The American Society for Parenteral and Enteral Nutrition. J. Parenter. Enter. Nutr. 2018, 42, 838–844. [Google Scholar] [CrossRef] [PubMed]
Table 1. Summary of critical care nutrition study design in the past, present, and future.
Table 1. Summary of critical care nutrition study design in the past, present, and future.
PastPresentFuture
Study methodologySmall physiological, observational, and single-centre comparative trials; poorly defined and underdeveloped processesLarge phase III trials (usually in the 1000s);
a priori defined protocols, outcomes, and interim analyses
Sophisticated statistical techniques;
larger sample sizes (tens of thousands);
adaptive trial designs
PopulationSmall, specific populations often from specialist centresGeneral, heterogenic populationsHomogenous;
selected based on anticipated response to nutrition intervention (e.g., malnourished);
includes non-invasively ventilated cohorts
InterventionCalories; early vs. late initiation; route of delivery e.g., gastric vs. jejunal, EN vs. PN, protein- vs. glucose-based PN, medium- vs. long-chain triglycerides and branched chain amino acids; immunonutritionGeneral interventions for general questions;
focused only on the period in ICU
Synergistic;
patient-specific;
based on mechanisms;
extension of interventions beyond ICU discharge considering illness trajectory
OutcomesCalorie delivery; nitrogen balance; incidence of infection; mortality (but underpowered to show an effect in the latter)Robust clinical outcomes such as mortalityPatient recovery;
functional outcomes;
valid surrogate markers
Abbreviations: EN = enteral nutrition; ICU = intensive care unit; PN = parenteral nutrition.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Chapple, L.-a.S.; Ridley, E.J.; Chapman, M.J. Trial Design in Critical Care Nutrition: The Past, Present and Future. Nutrients 2020, 12, 3694. https://0-doi-org.brum.beds.ac.uk/10.3390/nu12123694

AMA Style

Chapple L-aS, Ridley EJ, Chapman MJ. Trial Design in Critical Care Nutrition: The Past, Present and Future. Nutrients. 2020; 12(12):3694. https://0-doi-org.brum.beds.ac.uk/10.3390/nu12123694

Chicago/Turabian Style

Chapple, Lee-anne S., Emma J. Ridley, and Marianne J. Chapman. 2020. "Trial Design in Critical Care Nutrition: The Past, Present and Future" Nutrients 12, no. 12: 3694. https://0-doi-org.brum.beds.ac.uk/10.3390/nu12123694

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop