Skip to main content

Exploring mechanisms of excess mortality with early fluid resuscitation: insightsfrom the FEAST trial

Abstract

Background

Early rapid fluid resuscitation (boluses) in African children with severe febrileillnesses increases the 48-hour mortality by 3.3% compared with controls (nobolus). We explored the effect of boluses on 48-hour all-cause mortality byclinical presentation at enrolment, hemodynamic changes over the first hour, andon different modes of death, according to terminal clinical events. We hypothesizethat boluses may cause excess deaths from neurological or respiratory eventsrelating to fluid overload.

Methods

Pre-defined presentation syndromes (PS; severe acidosis or severe shock,respiratory, neurological) and predominant terminal clinical events(cardiovascular collapse, respiratory, neurological) were described by randomizedarm (bolus versus control) in 3,141 severely ill febrile children with shockenrolled in the Fluid Expansion as Supportive Therapy (FEAST) trial. Landmarkanalyses were used to compare early mortality in treatment groups, conditional onchanges in shock and hypoxia parameters. Competing risks methods were used toestimate cumulative incidence curves and sub-hazard ratios to compare treatmentgroups in terms of terminal clinical events.

Results

Of 2,396 out of 3,141 (76%) classifiable participants, 1,647 (69%) had a severemetabolic acidosis or severe shock PS, 625 (26%) had a respiratory PS and 976(41%) had a neurological PS, either alone or in combination. Mortality wasgreatest among children fulfilling criteria for all three PS (28% bolus, 21%control) and lowest for lone respiratory (2% bolus, 5% control) or neurological(3% bolus, 0% control) presentations. Excess mortality in bolus arms versuscontrol was apparent for all three PS, including all their component features. Byone hour, shock had resolved (responders) more frequently in bolus versus controlgroups (43% versus 32%, P <0.001), but excess mortality with boluseswas evident in responders (relative risk 1.98, 95% confidence interval 0.94 to4.17, P = 0.06) and 'non-responders' (relative risk 1.67, 95% confidenceinterval 1.23 to 2.28, P = 0.001), with no evidence of heterogeneity(P = 0.68). The major difference between bolus and control arms wasthe higher proportion of cardiogenic or shock terminal clinical events in bolusarms (n = 123; 4.6% versus 2.6%, P = 0.008) rather than respiratory (n =61; 2.2% versus 1.3%, P = 0.09) or neurological (n = 63, 2.1% versus1.8%, P = 0.6) terminal clinical events.

Conclusions

Excess mortality from boluses occurred in all subgroups of children. Contrary toexpectation, cardiovascular collapse rather than fluid overload appeared tocontribute most to excess deaths with rapid fluid resuscitation. These resultsshould prompt a re-evaluation of evidence on fluid resuscitation for shock and are-appraisal of the rate, composition and volume of resuscitation fluids.

Trial registration

ISRCTN69856593

Peer Review reports

Background

In the Fluid Expansion as Supportive Therapy (FEAST) trial, African children with shockrandomized to early rapid fluid resuscitation (20 to 40 ml/kg boluses) with normalsaline or 5% human albumin had a 3.3% increased absolute risk of death by 48 hourscompared with no-bolus controls [1]. Followingpublication, commentary papers, letters and discussion groups have speculated on reasonsfor this surprising result [2–7], given that bolus resuscitation is the gold standard forshock-management in well-resourced countries (albeit based on weak levels of evidence)[8].

FEAST was a pragmatic trial conducted in African hospitals without ventilationfacilities. It included children with shock caused by a heterogeneous group ofconditions including sepsis [9] and malaria[10] but excluded those withgastroenteritis and severe malnutrition [11].Consistency of adverse outcome was shown over all sites and in all possible subgroups,with no benefit of boluses observed for any working diagnosis [1], for any definition of shock [7, 12–14], or presence or absence of anemia or under-nutrition. Asreasons for harm caused by boluses remained unclear, we undertook further analyses toexplore possible mechanisms and modes of death in children randomized to bolusresuscitation versus control. We hypothesized that boluses may cause excess deaths fromneurological or respiratory events, particularly those relating to fluid overload. Weexplored the effect of boluses on 48-hour mortality according to signs and symptoms atenrolment (presentation syndrome, PS); predominant clinical syndrome in each patientprior to death (terminal clinical event, TCE); and changes in vital signs, measuredprospectively at pre-specified times.

Methods

Trial design and population

The methods and the outcome of the trial have been reported in detail [1]. In brief, children, aged 60 days to 12 years, withsevere febrile illness (classed as impaired consciousness (prostration or coma)and/or respiratory distress (increased work of breathing) plus clinical evidence ofimpaired perfusion (one of capillary refill time >2 seconds, lower limbtemperature gradient, weak radial pulse volume or severe tachycardia) at six centersin Kenya, Tanzania and Uganda were enrolled into two strata according to systolicblood pressure [1]. Stratum A included 3,141children without severe hypotension who were randomized to immediate bolus of 20ml/kg (increased to 40 ml/kg after protocol amendment [1]) of 5% albumin (albumin-bolus: 1,050 children) or 0.9%saline (saline-bolus: 1,047 children), or no-bolus (control, maintenance fluids 4ml/kg/hour: 1,044 children). The saline-bolus and albumin-bolus arms, but not thecontrol arm, received an additional 20 ml/kg bolus at one hour if impaired perfusionpersisted. In all three arms, further 40 ml/kg boluses of study fluid (saline for thecontrol arm) were only prescribed beyond one hour if severe hypotension developed(see definition below). Stratum B included 29 children with FEAST entry criteria plussevere hypotension (defined as systolic blood pressure <50 mmHg if <12 months;<60 mmHg if 1 to 5 years; <70 mmHg if > 5 years) who were randomized toalbumin or saline boluses of 40 to 60 ml/kg only. The primary endpoint was 48-hourmortality. Children with severe malnutrition, gastroenteritis, trauma, surgery orburns were excluded.

Baseline and follow-up data collection

Trial clinicians completed a structured clinical case report form at admission. Avenous blood sample was taken for immediate biochemical analyses with a handheldblood analyzer (iSTAT, Abbott Laboratories, Abbott Park, IL, USA), hemoglobin wasmeasured with HemOcue (Ängelholm, Sweden), glucose and lactate were measuredwith an On-Call glucometer and a Lactate Pro meter respectively and HIV antibodytesting. Blood smears for malaria parasites were prepared for immediate reading andsubsequent quality control. Standardized clinical reviews were conducted at 1, 4, 8,24 and 48 hours including vital signs and hemodynamic monitoring. A working clinicaldiagnosis was recorded at 48 hours as well as history of prior neurodevelopmentalprogress and any pre-admission neurological deficits. Children were managed ongeneral pediatric wards; mechanical ventilation (other than short-term 'bag-and-mask'support) was not available. Basic infrastructural support for emergency care as wellas oxygen saturation and automatic blood pressure monitors were provided for eachsite. All trial patients received intravenous antibiotics, antimalarial drugs (forthose with falciparum malaria) and intravenous maintenance fluids (2.5 to 4ml/kg/hour as per national guidelines) until the child was able to retain oralfluids. Antipyretics, anticonvulsants and treatment for hypoglycemia (blood sugar<2.5 mmol/l) were administered according to nationally agreed protocols. Childrenwith a hemoglobin level <5 g/dl were transfused with 20 ml/kg of whole blood over4 hours.

Assignment of presentation syndrome and terminal clinical event

Prior to and throughout the trial, clinical staff received onsite training in triageand emergency life support management to optimize case recognition, implementsupportive management and ensure protocol adherence. Throughout the hospitaladmission, severe adverse events were reported immediately and clinical features ofpulmonary edema and raised intracranial pressure, and evidence of hypovolemia andallergic events were actively solicited. An independent clinician removed allreferences to randomized arm or fluid management prior to review by the EndpointReview Committee (ERC), which included an independent chair (JE), five independentpediatricians (experienced in high dependency care and/or working in Africa), andcenter principal investigators. The ERC had access to 'blinded' clinical narratives,bedside vital observations (below), iSTAT results (baseline, 24 hours), microbiology,malaria and HIV status, and concomitant treatments. They adjudicated (blind torandomized arm) on whether fatal and non-fatal events could be related to bolusinterventions and the main causes of death [1]. In addition, the ERC chairperson and another non-center ERCmember reviewed all deaths occurring within 48 hours; using pre-specified criteria,they stratified all children by presentation syndrome (PS) and classified thepredominant clinical mode of death (TCE).

Presentation syndromes definitions

Severe shock or acidosis presentationwas any one of blood lactate>5 mmol/l [8], base excess >-8 mmol/l[10]; World Health Organization shockdefinition (all of cold hands or feet; capillary refill time >3 seconds; weak andfast pulse) [14] or moderate hypotension(systolic blood pressure (SBP) 50 to 75 mmHg in children aged <12 months, 60 to 75mmHg in children aged 1 to 5 years, and 70 to 85 mmHg in children aged >5years).

Respiratory presentation

hypoxia (oxygen saturation <92% measured by pulse oximetry) [15] plus one of history of cough, chest indrawingor crepitations [16].

Neurological presentation

coma or seizures at or immediately preceding hospital admission [16].

Terminal clinical event syndrome definitions

Cardiogenic/cardiovascular collapse

signs of shock at the point of demise - severe tachycardia or bradycardia plus oneof prolonged capillary refill time >2 seconds, cold peripheries or low SBP. Ifhypoxia was also present then this mode of TCE was a consensus view among ERCmembers, that circulatory failure was deemed to be the primary problem.

Respiratory

Ongoing or development of hypoxia (PaO2 <90%) with chest signs(crepitations or indrawing). Primary cause of death assigned as pneumonia and/orinvolving possible pulmonary edema.

Neurological

Possible raised intracranial pressure (high SBP or relative bradycardia) or, inchildren with severely reduced conscious level (Blantyre Coma Score ≤2[17]), focal neurological signs,abnormal pupil response to light or posturing at the point of demise.

Children whose deaths were unwitnessed were assigned 'unknown' TCE.

Analysis

The analysis focused on children in stratum A, in which 86% of deaths occurred within48 hours. Stratum B (mortality 62% (18 out of 29)) was not included because allchildren received boluses [1]. Albumin- andsaline-bolus arms were combined, as mortality was very similar in both. Allcomparisons between combined bolus and control arms were performed according tointention-to-treat, and all statistical tests were two-sided.

PS prevalence was described by randomized arm and 48-hour mortality was compared byrandomized arm within each PS. Forest plots were constructed to show comparisonsbetween arms for 48-hour mortality according to PS and all individual features ofeach PS. Hazard ratios for the comparison between bolus and no-bolus arms wereestimated for different levels of oxygen saturation and hemoglobin from Coxproportional hazard models, with a single indicator for treatment group, the level ofthe parameter and an interaction between treatment group and parameter. Competingrisks methods were used to estimate cumulative incidence curves and sub-hazard ratiosto compare the two treatment groups in terms of TCEs [18].

Oxygen saturation, axillary temperature, heart rate, respiratory rate, SBP, glucosevalues and a composite measure of shock or impaired perfusion (defined as any of thefollowing: capillary refill time >2 seconds, lower limb temperature gradient, weakradial pulse volume or severe tachycardia (<12 months, >180 bpm; 1 to 5 years,>160 bpm; >5 years, >140 bpm)) were summarized for survivors over time (atbaseline, 1, 4, 8, 24 and 48 hours) with box and whisker plots and bar charts.Treatment groups were compared in terms of mortality after one hour, conditional onchanges in shock and hypoxia parameters 1-hour post-randomization. Our analysesfocused only on early changes where the number of deaths was similar acrossrandomized arms; thereafter, because of excess deaths in bolus arms, results would besubject to survivorship bias.

Ethics statement

Ethics Committees of Imperial College London, Makerere University Uganda, MedicalResearch Institute, Kenya and National Medical Research Institute, Tanzania approvedthe protocol.

Results

In FEAST stratum A, 3,141 children were randomized between 13 January 2009 and 13January 2011 (1,050 albumin-bolus, 1,047 saline-bolus, 1,044 control) (Figure 1). Baseline characteristics were similar across arms and median agewas 24 months (interquartile range 13 to 38 months). Overall, 2,398 (76%) had impairedconsciousness (including 457 (15%) with unarousable coma), 1,172 (37%) had convulsionsand 2,585 (83%) had respiratory distress. Plasmodium falciparum malariaparasitemia was present in 1,793 out of 3,123 (57%); severe anemia (hemoglobin <5g/dl) in 987 out of 3,054 (32%); 1,070 out of 2,079 (52%) had a base deficit >8mmol/L; 1,159 out of 2,981 (39%) had a lactate level >5mmol/l; 126 out of 1,070(12%) had bacteremia (positive blood culture); and 10 out of 292 (3%) had meningitis(positive cerebrospinal fluid culture).

Figure 1
figure 1

Patient flow. 1 Inclusion criteria: Children aged >60 daysand <12 years with severe febrile illness including impaired consciousness(prostration or coma) and/or respiratory distress (increased work of breathing)were screened for clinical evidence of impaired perfusion (shock) to be eligiblefor the trial. Impaired perfusion was defined as any one of the following: CRT 3or more secs, lower limb temperature gradient, a weak radial pulse volume orsevere tachycardia: (<12 months: >180 beats per minute (bpm); 12 months to 5years: >160bpm; >5 years: >140 bpm). 2 Exclusion criteria:Evidence of severe acute malnutrition (visible severe wasting or kwashiorkor);gastroenteritis; chronic renal failure, pulmonary edema or other conditions inwhich volume expansion is contraindicated; non-infectious causes of severe illness(68); if they already received an isotonic volume resuscitation. 3Otherreasons for exclusion: child unable to return for follow-up (111), enrolled in adifferent study (65), no trial packs/fluid or blood (47), previously enrolled toFEAST (17), died (11), other (181), missing reason (26). 4Severehypotension defined as systolic blood pressure <50mmHg if <12m; <60mmHgif 1-5years; <70mmHg if >5years- eligible children with severe hypotension wereenrolled into FEAST B (see text) 5Child was not febrile (had no feveror history of fever). 6One child had severe hypotension and one childdid not have impaired perfusion.

Presentation syndromes

Of the 3,141 children in stratum A, 2,396 (76%) could be classified into a single PSor a combination; 633 (20%) cases had missing base excess (589) or lactate (32) orblood pressure (12), thus precluding classification to shock or acidosis PS, but halfof these had additional respiratory and/or neurological presentations (Figure 2a,b; Table S1 in Additional file 1). In112 children (4%), information was missing on two or more PS. Of the 2,396 childrenwith full information, 1,647 (69%) had severe metabolic acidosis or severe shock, 625(26%) had respiratory presentations and 976 (41%) had neurological presentations,alone or in combination. The distribution of PS was balanced across randomized arms(Table S1 in Additional file 1).

Figure 2
figure 2

Mortality at 48 hours by presentation syndrome. (a) Completeinformation; n = 2,396. (b) Incomplete information; n = 745. 48-hourmortality by presentation syndrome and in bolus (albumin and saline) andcontrol (no bolus) arms for those for which severe shock or acidosis (n = 633),or respiratory syndrome (n = 105) or neurological syndrome (n = 8) could not beascertained. Areas are proportional to the size of subgroups. B: bolusarm; C: control arm.

Mortality by presenting syndrome

Mortality was greatest among children fulfilling criteria for all three PS (28%bolus, 21% control) and combined shock or acidosis and respiratory presentations(19% bolus, 18% control). The greatest differences in mortality between bolus andcontrol groups was among those with all three PS (n = 205) and those with severeshock or acidosis PS alone (n = 698; 10% bolus, 3% control). These two groupsrepresented 37% (898 out of 2,396) of classifiable cases. Mortality was lowest forrespiratory presentation alone (2% bolus, 5% control) or neurological presentationalone (3% bolus, 0% control) (Figure 2a). A small number,363 out of 2,396 (15%), had only FEAST entry criteria; three children died in thisgroup (2% bolus; 0% control).

We found no evidence that the excess 48-hour mortality in bolus arms versus thecontrol arm differed by PS (Figure 3) or by individualclinical components of each PS (Figures 4, 5 and 6) (all P-values for heterogeneity≥0.2). The exception was hypoxia (oxygen saturations <92%), present in856 children (27%) at admission. As expected, hypoxia was a strong predictor ofhigher subsequent mortality, but there was statistically significant evidence thatthe excess mortality with boluses was greater in the subgroup without hypoxia atpresentation (bolus versus control, relative risk (RR) 1.94, 95% confidenceinterval (CI) 1.31 to 2.89) than in the subgroup with hypoxia (RR 1.13, 95%CI 0.79to 1.16; heterogeneity P = 0.04, Figure 4). This isalso demonstrated with oxygen saturation as a continuous variable in Figure S2 inAdditional file 1. Conversely, and as reportedpreviously, the degree of anemia at admission had no significant impact on theeffect of boluses on overall mortality [1, 7], excess harm being evident in the bolus armsversus control across the whole range of baseline hemoglobin values (Figure S3 inAdditional file 1).

Figure 3
figure 3

Mortality risk at 48 hours for bolus compared to no bolus by presentationsyndromes at baseline. Forest plots comparing effect of bolus versusno bolus for each baseline presentation syndrome (respiratory, neurologicalor severe shock or acidosis); children could be assigned to more than onesyndrome.

Figure 4
figure 4

Mortality risk at 48 hours for bolus compared to no bolus by individualrespiratory symptoms/signs at baseline.

Figure 5
figure 5

Mortality risk at 48 hours for bolus compared to no bolus by individualneurological signs/symptoms at baseline.

Figure 6
figure 6

Mortality risk 48 hours for bolus compared to no bolus by shock-relatedor systemic signs at baseline.

Terminal clinical events

In stratum A, 345 out of 3,141 children (11%) died; of these, 297 deaths (86%)occurred within 48 hours. Primary working diagnoses, recorded by clinicians andreported previously [1], included malaria 142(48%), pneumonia or respiratory etiology 41 (14%); septicemia 27 (9%), anemia 27(9%), meningitis 15 (5%), encephalitis 7 (2%), other diagnosis 12 (4%) andinsufficient information 26 (9%). The ERC adjudicated 265 single and 32 (11%)combined TCEs: 247 (83%) were judged to have a primary cardiogenic, respiratory orneurological TCE [1].

A cardiovascular or shock TCE was the most frequent overall (n = 123 (41%));neurological and respiratory TCEs occurred in 63 children (21%) and 61 children(20.5%) respectively (Figure 7); in 18 children the TCE wasunknown. As expected, TCE generally aligned with PS (Table S2 in Additional file1).

Figure 7
figure 7

Cumulative incidence of mortality for bolus combined and no bolus arms byterminal clinical events for 297 children who died within 48 hours.

Terminal clinical event-specific mortality by randomization arm

The major difference between bolus and control arms was the higher proportion ofdeaths adjudicated as having a cardiogenic or shock TCE in bolus arms, 96 (4.6%)compared with 27 (2.6%) in the control arm (sub-hazard ratio 1.79, 95%CI 1.17 to2.74, P = 0.008, Figure 7). This difference waseven greater when 39 modes of death in 39 children who died in the first hour(when bolus administration was incomplete) were excluded (79 (3.8%) compared with19 (1.8%) respectively of modes of death were cardiogenic (sub-hazard ratio 2.09,95%CI 1.27 to 3.45, P = 0.004)). Of note, and as expected, 25 out of 39early deaths (64%) were cardiogenic.

We found no evidence for increased risk of neurological events (putative 'cerebraledema') with boluses: there were 44 neurological TCEs in bolus arms (2.1%) versus19 (1.8%) in the control arm (P = 0.6). Respiratory TCEs (putative'pulmonary edema') were marginally more common in bolus arms: 47 (2.2%) versus 14(1.3%); P = 0.09 (Figure 7). No significantdifferences were found between albumin or saline boluses for any TCE.

The cumulative incidence of death by TCE for all children by bolus versus controlarms is shown in Figure 7, where, for clarity, single andcombined TCEs are redistributed so that cardiogenic and neurological TCEs areincluded with cardiogenic alone, and neurological and respiratory (largelyterminal lung aspiration in a comatose child) are included with neurologicalalone. Cumulative incidence for individual and combined TCE categories is shown inFigure S4a,b in Additional file 1.

Terminal clinical events according to bolus volume, malaria status andhemoglobin

The effect of a bolus had similar patterns on TCEs in children receiving 20 ml/kgand 40 ml/kg (that is, before and after the protocol amendment), among childrenwith and without malaria, and in those with and without severe anemia. In allgroups, cardiogenic TCEs accounted for the greatest excess in mortality in thebolus versus control groups, with no evidence of heterogeneity (allP-values >0.1) (Tables S3a,b,c in Additional file 1).

Changes in hemodynamics, vital status and laboratory parameters over time

Box and whisker plots of individual bedside vital status observations, includingheart rate, respiratory rate oxygen saturation, consciousness level andhypoglycemia (blood glucose <3 mmol/L) showed improvement over time, with fewdifferences between bolus and control arms (Figure S1 in Additional file 1). The exception was the composite measure of impairedperfusion (first box and whiskers plot in panel in Figure S1 in Additional file1), which by one hour had resolved more frequently inthe bolus than control arms; 43% of bolus-recipients had no sign of impairedperfusion compared with only 32% in the control arm (P ≤0.001).

Hemodynamic responses and changes in oxygen status at one hour

The mortality at 48 hours was significantly higher among 1,881 children withpersistent impaired perfusion (see analysis section for definition) at one hour(non-responders) compared with 1,198 responders (shock-resolution) (10% versus 4%,P <0.001, Table S4a in Additional file 1). However, despite greater improvements in perfusion in the bolus armat one hour, excess mortality in bolus versus control arms was evident innon-responders (RR 1.67, 95%CI 1.23 to 2.28, P = 0.001) as well asresponders (RR 1.98, 95%CI 0.94 to 4.17, P = 0.06), with no evidence thatthese were different (heterogeneity P = 0.68, Table S4a in Additionalfile 1).

Children with baseline hypoxia who remained hypoxic at one hour had increased riskof subsequent mortality compared with those whose hypoxia resolved (18% versus 7%,P <0.001, Table S4b in Additional file 1). There was no evidence to indicate that boluses were associated withincreased mortality in the children with persistent hypoxia compared with controlchildren (RR 0.71, 95%CI 0.43 to 1.18). Among children whose hypoxia had resolvedby one hour, the RR of mortality for bolus versus control was 1.45 (95%CI 0.73 to2.85, heterogeneity P = 0.1, Table S4b in Additional file 1).

A total of 175 out of 2144 children without hypoxia at baseline (8%) developedhypoxia by one hour and, as expected, had higher mortality compared with those whodid not develop hypoxia (15% versus 5%, bottom panel of Table S4b in Additionalfile 1). Slightly more children in the bolus arms thanin the control arm (129 (9%) versus 46 (7%)) developed hypoxia by one hour.However, excess risk of death in the bolus versus control arms was observed amongboth children who developed hypoxia (RR 1.96, 95%CI 0.71 to 5.39) and those whoremained non-hypoxic (RR 2.64, 95%CI 1.53 to 4.54). Thus, overall, there was noevidence that development of de novo hypoxia by one hour impacted on theexcess mortality in fluid bolus versus control arms (P-value forheterogeneity = 0.63, Table S4b in Additional file 1).

Discussion

In this paper, we have explored possible mechanisms for the excess death rate amongchildren randomized to receive rapid boluses of 20 to 40 ml/kg of 5% albumin or 0.9%saline fluid resuscitation compared with no-bolus controls. We found no evidence thatexcess 48-hour mortality associated with boluses differed by type of PS, by individualconstituent components of each syndrome, or by baseline hemoglobin level. Remarkably, inevery subgroup we examined, there was consistent evidence of harm by boluses.Paradoxically, the syndromes where most concern has been expressed over trial inclusion(respiratory or neurological alone and/or less severe shock criteria) not only had lowermortality overall, but also tended to have smaller differences between bolus andcontrol, although care must be taken with interpretation of smaller subgroups. The onlyexception was hypoxia, present in a quarter of children at admission, which surprisinglyappeared to be associated with significantly less harm from boluses. There appears to beno good rationale for this finding, which could have occurred by chance.

Even though this trial was conducted in settings with limited resources and no access tointensive care, the conduct of the trial complied to the highest standards of goodclinical practice including adherence to intervention strategy and completeness offollow-up, 100% source document monitoring and the robust and blinded methodology fordetermining PS and TCE. An intention-to-treat analysis with no need for imputation formissing data minimized the likelihood of bias and underpinned the magnitude andimportance of the unexpected findings of the trial and further analyses.

Noteworthy is that, consistent with global clinical experience, we observed a superiorresolution of impaired perfusion by one hour in the bolus arms compared with the controlarm. However, importantly, this did not translate into a superior outcome when comparedwith children with continued impaired perfusion: in both cases, boluses resulted inhigher mortality. Mortality excess with boluses among children with and without hypoxiaat baseline occurred to a similar extent irrespective of oxygen saturation status at onehour. Although de novo development of hypoxia at one hour was more common inthe bolus arms, it was not associated with a significant increase in 48-hour mortality -suggesting that, if fluid boluses were causing pulmonary edema (and hypoxia), this wasnot a unifying mechanism for increased mortality from boluses. Moreover, there waslittle evidence that fluid overload was the mechanism for excess deaths with bolusesfrom our analyses of neurological or respiratory TCEs. Overall, cardiovascular collapsewas the main TCE and contributed most substantially to the excess mortality in the bolusarms compared to control, peaking at 2 to 11 hours post-bolus. Whilst it is possiblethat subtle effects of fluid overload on the lungs or brain could have been missed, ourfindings do not lend support to this hypothesis, particularly as the ERC review processwas blind to randomization and used pre-specified TCE definitions.

A limitation of our trial was that we were unable to undertake invasive or point-of-carecontinuous monitoring to provide greater insight into TCEs, as in high-income intensivecare settings. However, the availability of patient-centered variables from our bedsideobservations and laboratory data from most children at baseline has enabled furthercharacterization of the trial participants into clinically relevant presentations (PS)in the context of where the trial was conducted. It has provided a more in-depthunderstanding of the spectrum of clinical groupings of children enrolled in the trialand the degree of adverse outcome fluid boluses had in these groups. Our bedsideobservations at pre-defined times following randomization, as well as adjudication ofcause of death by the ERC, blind to randomized arm and using pre-specified criteria(TCE), provide more speculative data but remain informative because they are largelyoperator-independent. These methodologies are central to the internal validity of ouranalysis, which sought to minimize systematic bias.

Whereas initial improvement in circulatory status following bolus resuscitation isconsistent with global clinical experience [8, 12, 19], the observation of excesssubsequent cardiovascular collapse and excess mortality, even in the early responders,was only made possible because of exemplary adherence to randomized allocation, and in atrial protocol in which further boluses were given to only (very few) childrendeveloping severe hypotension [12]. Thepossibility that fluid resuscitation lead to hemodilution [20, 21] in already anemic children, reducingoxygen delivery to the myocardium [22] andleading to ischemia and cardiac dysfunction, is largely ruled out by the lack ofheterogeneity in the TCEs by hemoglobin level, and by the analysis showing excess harmwith fluids across the whole range of hemoglobin values. These findings challenge thepresumption that early and rapid reversal of shock by fluid resuscitation translatesinto longer-term survival benefits [8, 23, 24] in settings where intensive carefacilities are not available. They do raise a possibility that rapid fluid resuscitationmay cause adverse effects on vascular hemodynamics and myocardial performance, drivingthe requirement for inotropic and pressor support. Rapid restoration of microcirculatoryperfusion may come at the expense of requiring other components of the sepsis bundle,including inotropes and ventilation. This possibility could be explored in futureclinical and preclinical research examining the natural history of shock in patientsmanaged by maintenance only (as in the control arm) and with fluid boluses.

Adverse effects of hyperchloremia, at the doses given in this trial, remaincontroversial [25–29].Intriguingly, the most deleterious effects of boluses were in those patients with severeacidosis at baseline, the group with the least a priori equipoise regarding thepotential benefits of fluid resuscitation - lending support to the notion of adverseeffects of the resuscitation fluids on acid-base equilibrium. Alternatively, this mayindicate lethal reperfusion injury [30] or asurge of cytokines [31] in cases with advancedshock. As previously suggested [1], shock may bean adaptive, time-dependent response sustaining children through a prolonged periodprior to hospital admission - only to die within hours of reperfusion.

Whatever the explanation, these findings raise important questions about thepathophysiological mechanisms of shock and goal-directed management, questioning whetherthe protocol-driven requirement for inotropes and vasoactive drugs has been driven, inpart, by aggressive fluid challenge [12]. The2008 Surviving Sepsis Campaign Guidelines, informed by a modified Delphi process, gradedthe pediatric recommendation (20 ml/kg boluses over 5 to 10 minutes up to 60 ml/kg) as2C, indicating a weak recommendation with low quality of evidence. The pediatric studieson which these recommendations were based included only two retrospective, observationalstudies of initial resuscitation volume on the outcome of children with putative sepsisfrom a single tertiary referral center, involving 34 and 91 children respectively[19, 24]. Theinclusion criteria for both studies were children who survived to intensive care unitadmission, but were inotrope-dependent and had pulmonary arterial catheters insitu. Higher initial fluid boluses and early shock reversal in 9 and 24 childrenin the two respective studies were associated with improved global outcome. However, thestudy design and the patient population had major limitations in terms of survivorshipbias and external validity to other settings. Other sources of evidence that werereferenced to inform fluid resuscitation guidelines include dengue shock. We suggestthese are largely irrelevant to the management of sepsis, because shock as acomplication of dengue occurs 7 to 10 days after fever defervescence, secondary to grossintravascular leakage leading to vomiting, abdominal pain, increasingly tenderhepatomegaly, narrow pulse pressure and hemoconcentration [32]. The American College of Critical Care Medicine guideline,and similar guidelines, nevertheless have been adopted by many countries worldwide wherepoint-of-care testing and intensive care facilities are available and are considered tobe best practice. These are now being recommended as standards of care for resource-poorsettings [33, 34];indicating that the FEAST trial had limited generalizability because the adverse effectsof fluid boluses were largely confined to children with malaria and/or anemia[33]. The data presented in the originalmanuscript [1], subsequence correspondence[7] and this detailed sub-analysisdefinitely counter this interpretation. A recent systematic review assessing theevidence base for fluid resuscitation in the treatment of children with shock due tosepsis or severe infection found only 13 pediatric trials. Whilst the majority of allrandomized evidence to date comes from the FEAST trial, they recommended thatimplementation of simple algorithms for children managed at hospitals with limitedresources to ensure identification of children who maybe potentially be harmed by fluidboluses [35]. The findings we report here raisequestions over the rates, volumes and types of solutions recommended in pediatricresuscitation protocols, most of which remain untested in clinical trials.

Conclusions

The results of the FEAST trial together with findings from these additional analysessuggest that rapid administration of fluid boluses increase the risk of subsequentcardiovascular collapse in children with shock, rather than increasing the risk of fatalfluid overload. The results suggest the need for further research to better understandthe pathophysiology of shock and its treatment, and the mechanisms whereby rapid fluidresuscitation increased mortality in African children.

Abbreviations

CI:

confidence interval

ERC:

Endpoint Review Committee

FEAST:

Fluid Expansion asSupportive Therapy

PS:

presentation syndromes

RR:

relative risk

SBP:

systolic bloodpressure

TCE:

terminal clinical events.

References

  1. Maitland K, Kiguli S, Opoka RO, Engoru C, Olupot-Olupot P, Akech SO, Nyeko R, Mtove G, Reyburn H, Lang T, Brent B, Evans JA, Tibenderana JK, Crawley J, Russell EC, Levin M, Babiker AG, Gibb DM: Mortality after fluid bolus in African children with severe infection. N Engl J Med. 2011, 364: 2483-2495. 10.1056/NEJMoa1101549.

    Article  CAS  PubMed  Google Scholar 

  2. Bates I: Detrimental effect of fluid resuscitation in the initial management of severelyill children in Africa. Transfus Med. 2011, 21: 289-290. 10.1111/j.1365-3148.2011.01108.x.

    Article  CAS  PubMed  Google Scholar 

  3. Duke T: What the African fluid-bolus trial means. Lancet. 2011, 378: 1685-1687. 10.1016/S0140-6736(11)60881-7.

    Article  PubMed  Google Scholar 

  4. Hilton AK, Bellomo R: Totem and taboo: fluids in sepsis. Crit Care. 2011, 15: 164-10.1186/cc10247.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Hilton AK, Bellomo R: A critique of fluid bolus resuscitation in severe sepsis. Crit Care. 2012, 16: 302-10.1186/cc11154.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Southall DP, Samuels MP: Treating the wrong children with fluids will cause harm: response to 'mortalityafter fluid bolus in African children with severe infection'. Arch Dis Child. 2011, 96: 905-906. 10.1136/archdischild-2011-300436.

    Article  CAS  PubMed  Google Scholar 

  7. Maitland K, Akech S, Russell E: Mortality after fluid bolus in African children with sepsis REPLY. N Engl J Med. 2011, 365: 1351-1353.

    CAS  Google Scholar 

  8. Dellinger RP, Levy MM, Carlet JM, Bion J, Parker MM, Jaeschke R, Reinhart K, Angus DC, Brun-Buisson C, Beale R, Calandra T, Dhainaut JF, Gerlach H, Harvey M, Marini JJ, Marshall J, Ranieri M, Ramsay G, Sevransky J, Thompson BT, Townsend S, Vender JS, Zimmerman JL, Vincent JL: Surviving Sepsis Campaign: international guidelines for management of severesepsis and septic shock: 2008. Intensive Care Med. 2008, 34: 17-60. 10.1007/s00134-007-0934-2.

    Article  PubMed  Google Scholar 

  9. Singh D, Chopra A, Pooni PA, Bhatia RC: A clinical profile of shock in children in Punjab, India. Indian Pediatr. 2006, 43: 619-623.

    PubMed  Google Scholar 

  10. Maitland K, Levin M, English M, Mithwani S, Peshu N, Marsh K, Newton CR: Severe P. falciparum malaria in Kenyan children: evidence forhypovolaemia. QJM. 2003, 96: 427-434. 10.1093/qjmed/hcg077.

    Article  CAS  PubMed  Google Scholar 

  11. Maitland K, Berkley JA, Shebbe M, Peshu N, English M, Newton CR: Children with severe malnutrition: can those at highest risk of death beidentified with the WHO protocol?. PLoS Med. 2006, 3: e500-10.1371/journal.pmed.0030500.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Brierley J, Carcillo JA, Choong K, Cornell T, Decaen A, Deymann A, Doctor A, Davis A, Duff J, Dugas MA, Duncan A, Evans B, Feldman J, Felmet K, Fisher G, Frankel L, Jeffries H, Greenwald B, Gutierrez J, Hall M, Han YY, Hanson J, Hazelzet J, Hernan L, Kiff J, Kissoon N, Kon A, Irazuzta J, Lin J, Lorts A, et al: Clinical practice parameters for hemodynamic support of pediatric and neonatalseptic shock: 2007 update from the American College of Critical Care Medicine. Crit Care Med. 2009, 37: 666-688. 10.1097/CCM.0b013e31819323c6.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Kleinman ME, Chameides L, Schexnayder SM, Samson RA, Hazinski MF, Atkins DL, Berg MD, de Caen AR, Fink EL, Freid EB, Hickey RW, Marino BS, Nadkarni VM, Proctor LT, Qureshi FA, Sartorelli K, Topjian A, van der Jagt EW, Zaritsky AL: Part 14: pediatric advanced life support: 2010 American Heart AssociationGuidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care. Circulation. 2010, 122: S876-908. 10.1161/CIRCULATIONAHA.110.971101.

    Article  PubMed  Google Scholar 

  14. WHO: Emergency Triage Assessment and Treatment (ETAT). 2005, Geneva: World Health Organization

    Google Scholar 

  15. Majumdar SR, Eurich DT, Gamble JM, Senthilselvan A, Marrie TJ: Oxygen saturations less than 92% are associated with major adverse events inoutpatients with pneumonia: a population-based cohort study. Clin Infect Dis. 2011, 52: 325-331. 10.1093/cid/ciq076.

    Article  PubMed  Google Scholar 

  16. WHO: Hospital Care for Children: Guidelines for the Management of Common Illnesseswith Limited Resources. 2005, Geneva: World Health Organization.

    Google Scholar 

  17. Molyneux ME, Taylor TE, Wirima JJ, Borgstein A: Clinical features and prognostic indicators in paediatric cerebral malaria: astudy of 131 comatose Malawian children. Q J Med. 1989, 71: 441-459.

    CAS  PubMed  Google Scholar 

  18. Fine PM, Gray RJ: A proportional hazards model for the subdistribution of a competing risk. J Am Stat Assoc. 1999, 94: 496-509. 10.1080/01621459.1999.10474144.

    Article  Google Scholar 

  19. Han YY, Carcillo JA, Dragotta MA, Bills DM, Watson RS, Westerman ME, Orr RA: Early reversal of pediatric-neonatal septic shock by community physicians isassociated with improved outcome. Pediatrics. 2003, 112: 793-799. 10.1542/peds.112.4.793.

    Article  PubMed  Google Scholar 

  20. Kumar A, Anel R, Bunnell E, Habet K, Neumann A, Wolff D, Rosenson R, Cheang M, Parrillo JE: Effect of large volume infusion on left ventricular volumes, performance andcontractility parameters in normal volunteers. Intensive Care Med. 2004, 30: 1361-1369.

    Article  PubMed  Google Scholar 

  21. Tait AR, Larson LO: Resuscitation fluids for the treatment of hemorrhagic shock in dogs: effects onmyocardial blood flow and oxygen transport. Crit Care Med. 1991, 19: 1561-1565. 10.1097/00003246-199112000-00020.

    Article  CAS  PubMed  Google Scholar 

  22. Weisel RD, Charlesworth DC, Mickleborough LL, Fremes SE, Ivanov J, Mickle DA, Teasdale SJ, Glynn MF, Scully HE, Goldman BS: Limitations of blood conservation. J Thorac Cardiovasc Surg. 1984, 88: 26-38.

    CAS  PubMed  Google Scholar 

  23. Rivers E, Nguyen B, Havstad S, Ressler J, Muzzin A, Knoblich B, Peterson E, Tomlanovich M: Early goal-directed therapy in the treatment of severe sepsis and septic shock. New Engl J Med. 2001, 345: 1368-1377. 10.1056/NEJMoa010307.

    Article  CAS  PubMed  Google Scholar 

  24. Carcillo JA, Davis AL, Zaritsky A: Role of early fluid resuscitation in pediatric septic shock. JAMA. 1991, 266: 1242-1245. 10.1001/jama.1991.03470090076035.

    Article  CAS  PubMed  Google Scholar 

  25. Guidet B, Soni N, Della Rocca G, Kozek S, Vallet B, Annane D, James M: A balanced view of balanced solutions. Crit Care. 2010, 14: 325-10.1186/cc9230.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Gunnerson KJ, Kellum JA: Acid-base and electrolyte analysis in critically ill patients: are we ready forthe new millennium?. Curr Opin Crit Care. 2003, 9: 468-473. 10.1097/00075198-200312000-00002.

    Article  PubMed  Google Scholar 

  27. Story DA, Morimatsu H, Bellomo R: Hyperchloremic acidosis in the critically ill: one of the strong-ion acidoses?. Anesth Analg. 2006, 103: 144-148. 10.1213/01.ane.0000221449.67354.52.

    Article  PubMed  Google Scholar 

  28. Kaplan LJ, Kellum JA: Fluids, pH, ions and electrolytes. Curr Opin Crit Care. 2010, 16: 323-331. 10.1097/MCC.0b013e32833c0957.

    Article  PubMed  Google Scholar 

  29. Rehm M, Orth V, Scheingraber S, Kreimeier U, Brechtelsbauer H, Finsterer U: Acid-base changes caused by 5% albumin versus 6% hydroxyethyl starch solution inpatients undergoing acute normovolemic hemodilution: a randomized prospectivestudy. Anesthesiology. 2000, 93: 1174-1183. 10.1097/00000542-200011000-00007.

    Article  CAS  PubMed  Google Scholar 

  30. Colletti LM, Remick DG, Burtch GD, Kunkel SL, Strieter RM, Campbell DA: Role of tumor necrosis factor-alpha in the pathophysiologic alterations afterhepatic ischemia/reperfusion injury in the rat. J Clin Invest. 1990, 85: 1936-1943. 10.1172/JCI114656.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Tracey KJ, Lowry SF: The role of cytokine mediators in septic shock. Adv Surg. 1990, 23: 21-56.

    CAS  PubMed  Google Scholar 

  32. Wills BA, Nguyen MD, Ha TL, Dong TH, Tran TN, Le TT, Tran VD, Nguyen TH, Nguyen VC, Stepniewska K, White NJ, Farrar JJ: Comparison of three fluid solutions for resuscitation in dengue shock syndrome. N Engl J Med. 2005, 353: 877-889. 10.1056/NEJMoa044057.

    Article  PubMed  Google Scholar 

  33. Dunser MW, Festic E, Dondorp A, Kissoon N, Ganbat T, Kwizera A, Haniffa R, Baker T, Schultz MJ: Recommendations for sepsis management in resource-limited settings. Intensive Care Med. 2012, 38: 557-574. 10.1007/s00134-012-2468-5.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Ralston ME, Day LT, Slusher TM, Musa NL, Doss HS: Global paediatric advanced life support: improving child survival inlimited-resource settings. Lancet. 2013, 381: 256-265. 10.1016/S0140-6736(12)61191-X.

    Article  PubMed  Google Scholar 

  35. Ford N, Hargreaves S, Shanks L: Mortality after fluid bolus in children with shock due to sepsis or severeinfection: a systematic review and meta-analysis. PloS One. 2012, 7: e43953-10.1371/journal.pone.0043953.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Pre-publication history

Download references

Acknowledgements

The study group would like to thank the children and families who participated in thetrial and the other members of the trial management group (listed below). We wouldalso like to thank Professor Rinaldo Bellomo for helpful comments on an earlierdraft.

The study was supported by a grant (G0801439) from the Medical Research Council,United Kingdom (provided through the MRC DFID concordat). Baxter Healthcare Sciencesdonated the resuscitation fluids (5% albumin and 0.9% saline). The funders (MedicalResearch Council) and Baxter Healthcare Sciences had no role in study design, datacollection and analysis, decision to publish, or preparation of the manuscript.

FEAST Management Group: KEMRI-Wellcome Trust Clinical Trials Facility, Kilifi,Kenya: Kathryn Maitland (Chief Principal Investigator (PI)), Mukami J. Mbogo(Trial Manager), Gilbert Ogetii; Malaria Consortium, Kampala, Uganda: JamesTibenderana, Lilian Akello, Moses Waweru; Data Management Group: NaomiWaithira, Trudie Lang, Roma Chilengi, Greg Fegan; Medical Research CouncilClinical Trials Unit, London, UK: Abdel Babiker (Trial Statistician),Elizabeth Russell (statistician), Margaret Thomason, Diana Gibb; ImperialCollege, London: Michael Levin.

Centers: Uganda: Mulago National Referral Hospital, Kampala: Sarah Kiguli(Chief PI Uganda), Robert O. Opoka (PI), Mariam Namutebi (Study Site Coordinator;SSC), Daniel Semakula, Ahmed Ddungu, Jalia Serwadda. Soroti Regional ReferralHospital: Charles Engoru (PI), Denis Amorut (SSC), Vincent Okuuny, RonaldWokulira, Moses Okiror, Steven Okwi; Mbale Regional Referral Hospital: PeterOlupot-Olupot (PI), Paul Ongodia (SCC), Julius Nteziyaremye, Martin Chebet, ConneliusMbulalina, Tony Ssenyondo, Anna Mabonga, Emmanuela Atimango; St. Mary's Hospital,Lacor: Richard Nyeko (PI), Benedict Otii (SSC), Sarah Achen, Paska Lanyero,Ketty Abalo, Paul Kinyera. Kenya: Kilifi District Hospital: SamuelO. Akech (PI), Molline Timbwa (SSC), Ayub Mpoya, Mohammed Abubakar, Mwanamvua Boga,Michael Kazungu. Tanzania: Teule Designated District Hospital, Muheza: George Mtove (PI), Hugh Reyburn (Co-PI), Regina Malugu (SSC), Ilse C EHendriksen, Jacqueline Deen, Selemani Mtunguja.

Pediatric Emergency Triage Assessment and Treatment training team: Hans-JorgLang, Mwanamvua Boga, Natalie Prevatt, Mohammed Shebe, Jackson Chakaya, JaphethKarisa.

Endpoint Review Committee: Jennifer Evans (Chair), Diana Gibb, JaneCrawley, Natalie Young; Serious adverse events reviewers: BernadetteBrent, Ayub Mpoya.

Author information

Authors and Affiliations

Authors

Consortia

Corresponding author

Correspondence to Kathryn Maitland.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

KM conceived the trial and participated in the design of the sub-study and wrote thefirst draft of the manuscript; ECG participated in the design of the study and performedthe statistical analysis; JAE chaired the ERC, and participated in the design of thestudy and the writing of the manuscript; SK coordinated the clinical trial withinUganda, and participated in the design of the sub-study and writing of the manuscript;PO was a member of ERC, and participated in the design of the study and writing of themanuscript; SOA was a member of the ERC, and participated the design of the study andwriting of the manuscript; ROO was a member of the ERC, and participated in the datacollection and interpretation of data; CE participated in the coordination of the study,was a member of the ERC, and participated in the data collection and interpretation ofdata; RN participated in the coordination of the study and was a member of the ERC, andparticipated in the data collection and interpretation of data; GM participated in thecoordination of the study, was a member of the ERC, and participated in the datacollection and interpretation of data; HR participated in the design of the study andwriting of the manuscript; BB carried out the independent review of serious adverseevents and was involved in the writing of the manuscript; JN was involved in thecoordination of data collection and interpretation; AM carried out the independentreview of serious adverse events and was involved in the writing of the manuscript; NPparticipated in the training of study clinicians and coordination of data collection anddata interpretation; CMD participated in the data collection and interpretation of data;DS participated in the data collection and interpretation of data; AD participated inthe data collection and interpretation of data; VO participated in the data collectionand interpretation of data; RW participated in the data collection and interpretation ofdata; MT participated in the training of study clinicians, data collection andinterpretation of data; BO participated in the data collection and interpretation ofdata; ML participated the design of the study and writing of the manuscript; JCparticipated in the design of the study, the ERC and writing of the manuscript, AGBparticipated in the design of the study and performed the statistical analysis, DMGparticipated in the design of the study, the ERC and writing of the manuscript. Allauthors read and approved the final manuscript.

Electronic supplementary material

12916_2012_685_MOESM1_ESM.DOC

Additional file 1: Figure S1: Box and whisker plots and bar charts of bedside vital statusobservations, oxygen saturation and hypoglycemia by arm (fluid armscombined) versus control arm. Table S1: Admission presentationsyndromes by randomization arm (FEAST A only). Figure S2 : Hazard ratiosand 95% confidence intervals of boluses compared to no bolus for mortality overdifferent levels of oxygen saturation at baseline. Figure S3: Hazardratios and 95% confidence intervals for boluses compared to no bolus formortality over different levels of hemoglobin at baseline. Table S2:Terminal clinical events for 297 children that died within 48 hours by baselinepresentation. Figure S4a: Cumulative incidence for bolus and no-bolusarms by cardiogenic only, neurological only or respiratory only terminalclinical events. Figure S4b: Cumulative incidence by bolus versus nobolus for combined causes: respiratory and neurological, cardiogenic andneurological, and unknown or other terminal clinical events. Table S3a:Terminal clinical event by 48 hours by randomization arm (bolus versus nobolus) and by whether they were enrolled before or after the protocolamendment. Table S3b: Terminal clinical event by 48 hours byrandomization arm (bolus versus no bolus) by malaria status. Table S3c:Terminal clinical events in those with anemia and those without anemia atbaseline. Percentages are out of all those enrolled in that arm within theanemia group or non-anemia group (totals are at the top of each column).Table S4a: Mortality at 48 hours with and without persisting featuresof shock at one hour. Table S4b: Mortality at 48 hours in those with orwithout hypoxia at baseline. (DOC 476 KB)

Authors’ original submitted files for images

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative CommonsAttribution License (http://creativecommons.org/licenses/by/2.0), whichpermits unrestricted use, distribution, and reproduction in any medium, provided theoriginal work is properly cited.

Reprints and permissions

About this article

Cite this article

Maitland, K., George, E.C., Evans, J.A. et al. Exploring mechanisms of excess mortality with early fluid resuscitation: insightsfrom the FEAST trial. BMC Med 11, 68 (2013). https://doi.org/10.1186/1741-7015-11-68

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/1741-7015-11-68

Keywords