Email updates

Keep up to date with the latest news and content from BMC Medical Research Methodology and BioMed Central.

Open Access Research article

Comparison of two alternative study designs in assessment of medicines utilisation in neonates

Georgi Nellis12*, Irja Lutsar1, Heili Varendi2, Karolin Toompere3, Mark A Turner45, Jennifer Duncan6 and Tuuli Metsvaht7

Author Affiliations

1 Institute of Microbiology, Tartu University, Tartu, Estonia

2 Neonatal Unit, Children´s Clinic, Tartu University Hospital, Tartu, Estonia

3 Department of Public Health, Tartu University, Tartu, Estonia

4 Department of Women’s and Children’s Health, Institute of Translational Medicine, University of Liverpool, Liverpool, UK

5 Neonatal Unit, Liverpool Women’s NHS Foundation Trust, Liverpool, UK

6 Research and Development, Alder Hey Children's NHS Foundation Trust, Liverpool, UK

7 Paediatric Intensive Care Unit, Clinic of Anaesthesiology and Intensive Care, Tartu University Hospital, Tartu, Estonia

For all author emails, please log on.

BMC Medical Research Methodology 2014, 14:89  doi:10.1186/1471-2288-14-89


The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1471-2288/14/89


Received:24 July 2013
Accepted:9 July 2014
Published:16 July 2014

© 2014 Nellis et al.; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.

Abstract

Background

Estimates of prevalence are known to be affected by the design of cross-sectional studies. A pan-European study provided an opportunity to compare the effect of two cross-sectional study designs on estimates of medicines use.

Methods

A Service evaluation survey (SES) and a web-based point-prevalence study (PPS) were conducted as part of a European study of neonatal exposure to excipients. Neonatal units from all European Union countries plus Iceland, Norway, Switzerland and Serbia were invited to participate. All medicines prescribed to neonates were recorded during three-day and one-day study periods in the SES and PPS, respectively. In the PPS individual demographic and prescription data were also collected.

To compare the probabilities that a particular medicine would be reported by each study multilevel mixed effects logistic regression models with crossed random effects were applied. The relationship between medicines exposure at the unit and individual levels in the PPS data was assessed using polynomial regression with square root transformation.

Results

Of 31 invited countries 20 and 21 with 115 and 89 units joined the SES and PPS, respectively. Out of 5,572,859 live births in invited countries in 2010 a higher proportion was covered by units participating in the SES compared to the PPS (11% vs 6%, respectively; OR 1.89; 95% CI 1.87-1.89). A greater number of active pharmaceutical ingredients (API), manufacturers and trade names were registered in the SES compared to the PPS. High correlation between the two studies in frequency of use for each specified API was seen (R2 = 0.86). The average probability of a department to use a given API was greater in the SES compared to the PPS (OR 2.36; 95% CI 2.05-2.73) with higher frequency of use and longer average duration of prescription further increasing the difference. The polynomial regression model described the correlation between APIs exposure on unit and individual level well (R2 = 0.93).

Conclusion

The simple data structure and longer study period of the SES resulted in improved recruitment and higher likelihood of capture for a given API. The frequency of use at the unit level appears a good surrogate of individual exposure rates.

Keywords:
Pharmacoepidemiologic methods; Cross-sectional studies; Data collection; Drug/excipient exposure

Background

Newborn babies receive a range of medicines many of which have not been studied adequately. Furthermore, exposure to medicines introduces risks and risk assessment requires information about the extent of medicines use. One neglected source of risk is the inclusion of excipients in medicines. Thus there is a need to gather information about medicines use in a way that captures data about active pharmaceutical ingredients (API) and the excipients. Variability in excipient content between formulations of the same API needs to be considered.

Cross-sectional studies are attractive for studies of medicines use [1-7]. In principle, data can be collected at unit or individual level. Unit level data involve listing all the products used on a unit over a specified period of time. These data are simple to collect and can be integrated with processes for medicines supply (such as sales or dispensing activity). Unit level data indicate which medicines are used (including how many distinct products, APIs and excipients are used) and will indicate whether there are geographical differences in medicines use. In combination with demographic data about the units this method can provide indicative estimates of market size. Individual level data require an extra step of data collection. This approach captures denominator data and generates more precise estimates of market size while allowing for stratification according to important clinical variables such as age of the baby. Point prevalence studies (PPS) have been used in antibiotic consumption studies [4,5,8]. Their extremely short duration may underestimate exposure to less frequently used medicines. Unit level studies can have a longer duration but may not be manageable in a multinational setting [9]. Although a recent analysis has described the impact of cross-sectional study designs on disease incidence/prevalence estimates [10], little is known about the effects of study design in research about medicines use. We are not aware of any head to head comparison of different methods and thus the question how to navigate between different methods in medicine exposure research remains unanswered.

Within the European Study of Neonatal Excipient Exposure (ESNEE) we planned to use both unit and individual level data collection in order to maximise yield for two different study questions. [11]. This gave us the opportunity to compare the two study designs – a short duration PPS and longer lasting service evaluation survey (SES). We used APIs as the unit of analysis in this methodological study.

Accordingly, we aimed (1) to describe the implementation of each method; (2) to assess the extent of discordance between the methods by comparing the probabilities that particular medicines would be reported by each method; (3) to explore the correlations between medicines exposure at unit and individual levels.

Methods

The SES and PPS were performed as multicentre observational surveys of routine clinical practice. During the SES all medicines prescribed to neonates (≤28 days of postnatal age) were recorded on pre-formulated Excel spreadsheets over three consecutive days within a fixed period of time from May 30th to September 30th, 2011. PPS data collection was performed in a web-based database within one day during one of three fixed two-week study periods from January to February; March or May to June, 2012. Each participating unit was free to choose the most appropriate day(s) for data collection. In both surveys printable data collection forms were provided as an alternative to electronic data insertion.

Participating units and study population

The study aimed to cover all 27 European Union countries plus Iceland, Norway, Switzerland and Serbia. A network of national contact persons (Lead Contacts) was built by the ESNEE consortium. Search for national Lead Contacts was carried out through national professional/scientific societies of neonatology, paediatrics and/or perinatology as well as through personal contacts of the ESNEE consortium members or other FP-7-funded consortia (NeoMero, TINN) [12,13]. The Lead Contact was then asked to recruit as many hospitals and units providing neonatal care in the country as possible.

At the outset, a PPS sample size estimation was planned. It was intended to base this on cluster sampling analysis stratified by country, assuming conservatively a 15% neonatal admission rate of all live births [14-17] and a response rate of 50-70% of invited units as described in previous neonatal surveys [18-21]. Based on the Eurostat NUTS (or equivalent if NUTS classification not available for the country) regional distribution of the population and reported nation-wide birth rates, the number of potentially available neonatal admissions would be calculated and the representative sample size estimated for each country and region [22]. A complete list of institutions involved in neonatal care would be created for each randomly chosen NUTS2 region with subsequent involvement of all units to cover a whole region with proportional representation of different unit levels. Alternatively, if a full list of neonatal units was not available and/or the response rate in the SES would suggest critically low recruitment, all contacts provided by Lead Contacts were to be invited to participate in the PPS.

All general neonatal, intermediate and neonatal intensive care (NICU) as well as mixed paediatric and neonatal intensive care units with more than 50% of admissions consisting of neonates were eligible with stratification according to the level of care: level 1 with special neonatal care; level 2 with high dependency care, short term intensive care and low birth weight care, and level 3 with comprehensive intensive care for extremely low birth weight infants available [23,24]. Departments, offering different levels of care were classified according to the highest level they provided.

Data collection

In both studies all eligible neonates in the unit at 8 a.m. of the study day and in the SES additionally those admitted within the next 72 h, were included. The SES recorded the number of neonates receiving any prescription during the study period. In the PPS individual demographic data including gender, gestational age (GA), birth weight, 1st and 5th minute Apgar score, current body weight, postnatal age and organ dysfunctions, were recorded for all neonates receiving any prescriptions active on the study morning at 8 a.m.

Prescription data

All prescriptions, including micronutrients, iron, vitamins, parenteral nutrition solutions and topical agents were recorded. Blood products, glucose and electrolyte solutions, vaccines, nursery care topical agents, herbal medicines and food including breast milk fortifiers were excluded. The following information was collected for every medicine prescribed: trade name, manufacturer, API, strength, pharmaceutical dosage form and route of administration. Additionally individual dosing regimen together with prescription starting date was collected in the PPS.

Extemporaneous or compounded forms of medicines were not included in described analysis. The use of caffeine and morphine was reported differently in the two studies. UK units using ´special´ manufactured [25] preparations, rather than extemporaneous formulations used in the rest of European countries, were over-represented in the PPS. Accordingly, these two APIs were excluded from the analysis.

In both studies the data were pooled for analysis without any comment on treatment strategies of individual participating units.

Data management and statistical analysis

SES data collection forms were checked for legibility with two rounds of queries regarding missing data (response rate 52%). Each medication was classified according to brand name, manufacturer and pharmaceutical dosage form. The list of medicines obtained in the SES was used to prepopulate choices in the PPS database. To minimize the risk of data entry errors all ´newly appearing´ medicines in the PPS were added to the database by the ESNEE team.

Statistical analysis was performed with Stata Software (ver. 12.1). Descriptive statistics were used as appropriate to describe the variability of medicine exposure in both studies.

To identify the effect size of the study method, the average probabilities of the units to use any given API in SES vs PPS were compared by multilevel mixed effects logistic regression models with crossed random effects [26]. Since the sample of departments in the two studies was partly overlapping, department and API were added to the model as a random effect when comparing the average odds of using an API between the two study methods. The outcome variable was a binary indicator showing whether an API was used in a specific unit in a specific study. The analysis was limited to agents used in more than one unit in the SES. All models were adjusted for geographic region (Northern, Southern, Western and Eastern Europe according to the United Nations Statistics department [27]) and unit level. These variables were identified as potential confounders in an exploratory population averaged Poisson regression model with robust standard errors [28] analysis including 10 most often used APIs. Furthermore the effect of unit size (described by number of annual admissions), frequency of a given medicine use (described by proportion of units using it) and the duration of the prescription (described by the average time from the start of prescription to the PPS study day) on the estimated method effect size was explored.

To study the relationship between the number of units using each API and the number of prescriptions for that API polynomial regression model with square root transformation was used based on the PPS data.

Ethical considerations

The SES did not collect any personal data and so did not require ethics committee approval in any of the participating countries. For participation in the PPS Ethics Committee approval was obtained in compliance with the respective national guidelines. In compliance with local authorities no consent for individual patients was sought, as all data were collected in routine clinical practice and anonymised before leaving study sites.

Results

Characterisation of participating units

The cluster randomised approach to sample size calculation described in Methods section, was not feasible because no country had a full list of neonatal units. Accordingly, all country Lead Contacts were invited to participate in the PPS with as many units as possible. Of 31 invited countries 20 and 21 (Austria, Belgium, Bulgaria, England, Estonia, Greece, Hungary, Ireland, Italy, Latvia, Lithuania, Norway, Portugal, Romania, Serbia, Slovenia, Spain and Switzerland in both; Denmark and Poland only in the SES; Malta, France and the Netherlands only in the PPS) with 115 and 89 neonatal units joined the SES and PPS, respectively. The number of units per country varied from one to 20. In the SES compared to the PPS a higher proportion of annual live births was covered overall (11% vs 6% of 5,572,859 live births in participating countries in 2010, respectively; OR 1.89; 95% CI 1.87-1.89) and in each European region (Table 1). Regional distribution varied with higher representation of Eastern compared to Western Europe (10% and 6% of 968,910 and 1,987,904 live births, respectively; OR 1.80; 95% CI 1.78-1.81) and lower compared to Northern and Southern European countries (12% and 18% of 1,357,998 and 1,258,047; OR 0.52; 95% CI 0.51-0.52 and 0.84; 0.83-0.84, respectively) in the SES. The same regional distribution was seen in the PPS (data not shown). No difference in the prevalence of different levels of care between the two studies was observed. However, the prevalence of teaching hospitals was lower in the SES as compared to the PPS (OR 0.47; 95% CI 0.24-0.92) (Table 1). Similarly, the proportion of all annual admissions in units covered by the study was higher in the SES than the PPS (3% and 2% of 90235 and 61392 admissions, respectively; OR 1.58; 95% CI 1.48-1.68).

Table 1. Characteristics of participating units in the SES and the PPS

Consumption of medicines in the PPS and the SES

More neonates in the SES received at least one drug prescription compared to the PPS (OR 1.34; 95% CI 1.18-1.53) (Table 1). In total, a greater numbers of APIs, manufacturers and trade names were registered during the SES than the PPS (Table 2). Slightly lower prevalence of enteral and higher prevalence of topical formulations was noted in the SES compared to the PPS with no difference in parenteral medicines use. In the PPS the number of prescriptions per neonate was inversely related to GA the average (SD) count per neonate being 4.53 (2.33); 3.95 (2.59); 2.68 (1.91) and 2.31 (1.93) in the GA bands of <28; 28–31; 32–36 and >36 weeks, respectively.

Table 2. Medicines consumption in the SES and the PPS

Correlation of medicine use between SES and PPS

Altogether 99 APIs were used in more than one unit in the SES and were included into further analysis. As shown in Figure 1 almost all APIs were more frequently used in the SES compared to the PPS but the 95% CIs were overlapping for the majority.

thumbnailFigure 1. Proportions of units prescribing frequently used active substances in the SES and the PPS. The SES and PPS data presented as triangles and circles, respectively. Active substances that were used in more than one unit in the SES (n = 99) are presented. Each number on the x-axis identifies the individual active substance. Upper 95% CI in the SES and lower 95% CI in the PPS are shown in error bars.

In multilevel mixed effects logistic regression models with crossed random effects the average probability of the departments to use each of the most common API was higher in the SES compared to the PPS (OR 2.36; 95% CI 2.05-2.73; p < 0.0005). The frequency of use and average duration of prescription further increased the likelihood of being registered in favor of the SES by 1.01 times (95% CI 1.01-1.02; p < 0.0005) and 1.02 times (95% CI 1.00-1.05; p = 0.047) per each additional percent in frequency and day in duration of prescription, respectively. Size of the department did not influence the capture probability (OR 0.99; 95% CI 0.99-1.00; p = 0.652).

As shown in Figure 2 a high correlation in the frequency of medicine use between the SES and the PPS was observed (R2 = 0.86).

thumbnailFigure 2. Correlation in the frequency of medicine use between the SES and the PPS on unit level. Correlation between the SES and the PPS for active substances used in more than 1 unit in the SES is shown. Trendline for 99 active ingredients: y = 0.6019x – 0.0053; R2 = 0.8605.

We next evaluated the correlation between the number of prescriptions per given API and the number of units using that API in the PPS. As shown on Figure 3 there was a high correlation between these two variables in polynomial regression model with square root transformation (R2 = 0.93).

thumbnailFigure 3. Correlation between medicines exposure at unit and individual levels. Number of units using each specified active ingredient (n = 99) in relation to the number of prescriptions in the PPS was observed. Polynomial regression trendline for active ingredients used in more than one unit in the SES is shown. √number of prescriptions = 0.421 *(number of departments) – 0.004 *(number of departments)2 + 0.485; R2 = 0.93.

Discussion

By applying two different study designs in comparable target populations we have demonstrated that simple data structure and longer study period used in the SES improved recruitment and the likelihood of capture of medicines consumption. The probability of capture for a given API was further enhanced by higher frequency of use and longer duration of prescription. This meant that the SES gave a more comprehensive list of trade names, APIs and manufacturers than the PPS. The list of most frequently used APIs was overlapping in 90% with high correlation of use frequencies between the two studies.

The strength of both methods described here is the simplicity and uniformity of data collection. A two-step approach, as used in the ESNEE project may allow simultaneous coverage of both unit-level and individual-level data in neonatal excipient exposure studies, where hardly any multi-country data are available. We have recruited so far one of the largest international cohorts of neonatal units from 21 European countries revealing a surprisingly wide list of medicines. The number of trade names applied in NICU setting was also high. The response rate of 71% on country level as well as the number of prescriptions per neonate [9,29] are similar to those reported previously [18-21]. Against the background of rising safety concerns of neonatal medicine/excipient exposure, the wide trade name list from the SES allows identification of substitution possibilities, while individual exposure data from the PPS provide a more precise quantification of the problem.

Different study designs can be applied in pharmacoepidemiology [30,31]. Cohort studies have the advantage of allowing data collection over prolonged time periods which is more likely to capture rare exposures. However, the expense and duration make it hard to implement cohort designs in multinational studies [9]. Case–control studies allow rich data collection when a limited number of medicines/excipients are targeted and data collection is limited to a few centers. The PPS and SES, in contrast, are easy to perform in multinational settings. They are time and resource saving and to some extent cover each other`s shortcomings like under- or overestimated exposure rates of rarely used medicines or need for individual data [30]. Alternately, nested study designs with longitudinal data collection allow linkage between different datasets and may prove the best option. However, to the best of our knowledge, it has hardly been used in paediatric pharmacoepidemiological studies.

When planning medicine/excipient exposure studies the research question to be answered needs to be balanced against the implications of study design. We recognize that a 3-day survey is actually very short in duration. However, even this small longitudinal component resulted in a more comprehensive list of medicines due to the better recruitment when simple data structure was used. On the other hand, when individual exposures of frequently used medicines are targeted, the increasing data volume (both structure and duration) needs to be weighed against evolving decrease in compliance that may lead to underestimation of prevalence [32]. It has been shown that the longer the recall period, the greater the imprecision (especially underestimation) of prevalence estimates [33,34], although this finding may be more relevant in non-medical informants [10]. Furthermore, not only optimal recruitment but also sample size maintenance over the course of a study becomes an issue [35].

Additionally, economic evaluation plays a role in the choice of a specific study design. There is considerable overlap between acceptable methodologies and those sanctioned by health economists [36]. Although pharmacoeconomic analysis remains beyond the scope of this study, a few comments can be made. There were no major cost-differences in conducting these studies with the exception of those associated with the development of the PPS database. Web-based collection with automated control of data completeness was applied to improve compliance and quality of reporting. We believe that wherever sufficient to answer the research question, simple data structure is favourable in terms of resource allocation. However, when a more complex format is required, reasonable additional expenditure on data collection tools may result in an improved cost-benefit ratio.

We have also shown a high correlation between APIs exposure on unit and individual level. This should allow an indirect individual exposure assessment from less demanding approach like SES under the condition of limited resources. We realize, that the estimates in the frequency of medicine use in the PPS are systematically lower compared to the SES. However, knowing the relationship between the two an estimate for one can be extrapolated from the other. Nevertheless, such calculations should be applied with caution and to estimate the exposure only for the studied population as a whole. In neonatal studies stratification by GA age allows a more meaningful risk assessment due to the strong dependence of the maturity of metabolic pathways on the postmenstrual age of the newborn [37,38]. It has been shown previously and also by us that medicines utilization pattern in extreme prematurity is different from that near or at term [9,29,39,40]. Therefore, the PPS approach should be preferred to obtain individual exposure estimates in neonates of different GA.

Some limitations of the present study need to be noted. First, the quality of data collection was dependent on the personnel entering the data. In both studies data insertion was done by medical specialists according to standardized protocols. As for the quality of reported data, in the SES most queries concerned demographic data; only a few queries were related to the information about medicines. Second, neonatal units were given the option of selecting the most appropriate day(s) for data collection. This could lead to underestimation of medicine use because less busy day(s) probably were more likely to be chosen. However, we believe that eventually this approach captured information on a larger number of neonates and medicines through improved compliance and a larger number of participating units. Third, the slightly different representation by country required exclusion of two APIs from final analyses due to differences in regional handling/approach. Fourth, the SES and the PPS were done at different times. Nested design with detailed individual data collection (PPS) performed as part of the SES was not feasible, as an interim between the two studies was needed to prepopulate the PPS database from the SES list of medicines. This could potentially reduce the overlap between the studies. In that situation our data about overlap represent a “worst case scenario”. The true concordance in API reporting may be higher than described by us. Finally, about two thirds of the 31 invited countries agreed to participate with relative inhomogeneity of the regional distribution of participating units. Although the inability to randomly select hospitals probably caused overrepresentation of teaching hospitals (61% and 77% in the SES and PPS, respectively, compared to less than 10%, reported in national hospital statistics [41]), data on the distribution of the levels of neonatal care will allow appropriate adjustments. Both SES and PPS have achieved representative sample size on patient level, the margin of error being 1.73% and 2.63%, respectively (CI 95%; response distribution 50%; population size 5.4 million live births). However, it would be valid under the condition of randomisation that remained unfeasible in our studies. Still, the current analysis gives a notion of good coverage of the target population in the SES and PPS.

Conclusions

The less demanding approach of the SES allows better recruitment of units for longer periods resulting in a more comprehensive list of medicines and trade names. As we have shown, the frequency of use on unit level appears a good surrogate of individual exposure rates. However, more detailed information can be collected in a PPS. Investigators need to balance the advantages of a PPS with the risk of bias, which depends on the frequency as well as duration of medicine use. Simultaneous use of both methods with merged data analysis will likely result in optimal coverage of both aspects of the problem.

Abbreviations

API: Active Pharmaceutical Ingredient; ESNEE: European Study of Neonatal Excipient Exposure; GA: Gestational Age; NICU: Neonatal Intensive Care Unit; NUTS: Nomenclature of Territorial Units for Statistics; PPS: Point Prevalence Study; SES: Service Evaluation Survey.

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

GN participated in the design and conduct of the study and writing the manuscript. IL participated in the design and coordination of the study and drafting/revising the manuscript. HV participated in the design and conduct of the study. KT performed the statistical analysis. MT conceived of the ESNEE project, and participated in its design and coordination, and has been involved in revising the manuscript. JD participated in the design and conduct of the study. TM participated in the design and conduct of the study and has been involved in drafting and revising the manuscript. All authors read and approved the final manuscript.

Acknowledgements

We thank all national contact persons who provided lists of neonatal units in their countries: Bernhard Resch (Austria); Pieter De Cock (Belgium); Nelly Jekova (Bulgaria); Elisabeth Iyore (Denmark); Pascal Vaconsin (France); Kosmas Sarafidis (Greece); Aranka Vegso (Hungary); Noreen O'Callaghan (Ireland); Rocco Agostino (Italy); Daiga Kviluna (Latvia); Rasa Tameliene (Lithuania); Rene F. Kornelisse (Netherlands); Dag Bratlid (Norway); Almerinda Pereira (Portugal); Maria Livia Ognean (Romania); Milica Bajcetic (Serbia); Darja Paro (Slovenia); Elizabeth Valls (Spain); Per Nydert (Sweden); Hans Ulrich Bucher (Switzerland); Maria Cordina (Malta).

We also thank the whole ESNEE project consortium for support: S. Graham, A. Rieutord, T. Storme, P. Vaconsin, A.J. Nunn, J. McElnay, J. Millership, S. Yakkundi, H. Mulla, H. Pandya, J.Lass.

Funding

ESNEE is funded through ERA-NET PRIOMEDCHILD by the following national agencies: MRC from United Kingdom, Estonian Science Foundation from Estonia, Agence Nationale de la Recherche from France.

References

  1. Roberts JA, De Waele JJ, Dimopoulos G, Koulenti D, Martin C, Montravers P, Rello J, Rhodes A, Starr T, Wallis SC, Lipman J: DALI: Defining Antibiotic Levels in Intensive care unit patients: a multi-centre point of prevalence study to determine whether contemporary antibiotic dosing for critically ill patients is therapeutic.

    BMC Infect Dis 2012, 12:152. OpenURL

  2. Zarb P, Amadeo B, Muller A, Drapier N, Vankerckhoven V, Davey P, Goossens H: Antifungal therapy in European hospitals: data from the ESAC point-prevalence surveys 2008 and 2009.

    Clin Microbiol Infect Off Publ Eur Soc Clin Microbiol Infect Dis 2012, 18:E389-E395. OpenURL

  3. Ingram PR, Seet JM, Budgeon CA, Murray R: Point-prevalence study of inappropriate antibiotic use at a tertiary Australian hospital.

    Intern Med J 2012, 42:719-721. OpenURL

  4. Amadeo B, Zarb P, Muller A, Drapier N, Vankerckhoven V, Rogues A-M, Davey P, Goossens H: European Surveillance of Antibiotic Consumption (ESAC) point prevalence survey 2008: paediatric antimicrobial prescribing in 32 hospitals of 21 European countries.

    J Antimicrob Chemother 2010, 65:2247-2252. OpenURL

  5. Ansari F, Erntell M, Goossens H, Davey P: The European surveillance of antimicrobial consumption (ESAC) point-prevalence survey of antibacterial use in 20 European hospitals in 2006.

    Clin Infect Dis an Off Publ Infect Dis Soc Am 2009, 49:1496-1504. OpenURL

  6. Zarb P, Amadeo B, Muller A, Drapier N, Vankerckhoven V, Davey P, Goossens H: Identification of targets for quality improvement in antimicrobial prescribing: the web-based ESAC Point Prevalence Survey 2009.

    J Antimicrob Chemother 2011, 66:443-449. OpenURL

  7. Sequi M, Campi R, Clavenna A, Bonati M: Methods in pharmacoepidemiology: a review of statistical analyses and data reporting in pediatric drug utilization studies.

    Eur J Clin Pharmacol 2013, 69:599-604. OpenURL

  8. Zarb P, Goossens H: European Surveillance of Antimicrobial Consumption (ESAC).

    Drugs 2011, 71:745-755. OpenURL

  9. Lass J, Käär R, Jõgi K, Varendi H, Metsvaht T, Lutsar I: Drug utilisation pattern and off-label use of medicines in Estonian neonatal units.

    Eur J Clin Pharmacol 2011, 67:1263-1271. OpenURL

  10. Schmidt W-P, Arnold BF, Boisson S, Genser B, Luby SP, Barreto ML, Clasen T, Cairncross S: Epidemiological methods in diarrhoea studies–an update.

    Int J Epidemiol 2011, 40:1678-1692. OpenURL

  11. Turner M: European Study of Neonatal Exposure to Excipients (ESNEE).

    Infant 2011, 7:1-4. OpenURL

  12. NEOMERO - European multicenter network to evaluate pharmacokinetics, safety and efficacy of Meropenem in neonatal sepsis and meningitis http://www.neomero.org/ webcite

  13. TINN European Survey on the Use of Antibiotics in NICUs http://www.tinn-project.org/ webcite

  14. Labenne M, Michaut F, Gouyon B, Ferdynus C, Gouyon J-B: A population-based observational study of restrictive guidelines for antibiotic therapy in early-onset neonatal infections.

    Pediatr Infect Dis J 2007, 26:593-599. OpenURL

  15. Dani C, Corsini I, Piergentili L, Bertini G, Pratesi S, Rubaltelli FF: Neonatal morbidity in late preterm and term infants in the nursery of a tertiary hospital.

    Acta Paediatr (Oslo, Norw 1992) 2009, 98:1841-1843. OpenURL

  16. Tracy SK, Tracy MB, Sullivan E: Admission of term infants to neonatal intensive care: a population-based study.

    Birth 2007, 34:301-307. OpenURL

  17. Schiariti V, Klassen AF, Houbé JS, Synnes A, Lisonkova S, Lee SK: Perinatal characteristics and parents’ perspective of health status of NICU graduates born at term.

    J Perinatol Off J Calif Perinat Assoc 2008, 28:368-376. OpenURL

  18. Losacco V, Cuttini M, Greisen G, Haumont D, Pallás-Alonso CR, Pierrat V, Warren I, Smit BJ, Westrup B, Sizun J: Heel blood sampling in European neonatal intensive care units: compliance with pain management guidelines.

    Arch Dis Child Fetal Neonatal Ed 2011, 96:F65-F68. OpenURL

  19. Zeitlin J, Papiernik E, Bréart G: Regionalization of perinatal care in Europe.

    Semin Neonatol SN 2004, 9:99-110. OpenURL

  20. Greisen G, Mirante N, Haumont D, Pierrat V, Pallás-Alonso CR, Warren I, Smit BJ, Westrup B, Sizun J, Maraschini A, Cuttini M: Parents, siblings and grandparents in the Neonatal Intensive Care Unit. A survey of policies in eight European countries.

    Acta Paediatr (Oslo, Norw 1992) 2009, 98:1744-1750. OpenURL

  21. Cuttini M, Rebagliato M, Bortoli P, Hansen G, de Leeuw R, Lenoir S, Persson J, Reid M, Schroell M, de Vonderweid U, Kaminski M, Lenard H, Orzalesi M, Saracci R: Parental visiting, communication, and participation in ethical decisions: a comparison of neonatal unit policies in Europe.

    Arch Dis Child Fetal Neonatal Ed 1999, 81:F84-F91. OpenURL

  22. NUTS - Nomenclature of territorial units for statistics [http://epp.eurostat.ec.europa.eu/portal/page/portal/nuts_nomenclature/introduction webcite]

  23. Stark AR: Levels of neonatal care.

    Pediatrics 2004, 114:1341-1347. OpenURL

  24. Modi N: British association of perinatal medicine. London, UK; 2011. [Categories of Care 2011] OpenURL

  25. Manufacturer’s and wholesale dealer's licences http://www.mhra.gov.uk/Howweregulate/Medicines/Licensingofmedicines/Manufacturersandwholesaledealerslicences/index.htm webcite

  26. Raudenbush SW: A crossed random effects model for unbalanced data with applications in cross-sectional and longitudinal research.

    J Educ Stat 1993, 18:321-349. OpenURL

  27. United Nations Statistics Division - composition of macro geographical (continental) regions, geographical sub-regions, and selected economic and other groupings http://unstats.un.org/unsd/methods/m49/m49regin.htm#europe webcite

  28. Zou GY, Donner A: Extension of the modified Poisson regression model to prospective studies with correlated binary data.

    Stat Methods Med Res 2013, 22:661-670. OpenURL

  29. Neubert A, Lukas K, Leis T, Dormann H, Brune K, Rascher W: Drug utilisation on a preterm and neonatal intensive care unit in Germany: a prospective, cohort-based analysis.

    Eur J Clin Pharmacol 2010, 66:87-95. OpenURL

  30. Turner MA, Duncan J, Shah U, Metsvaht T, Varendi H, Nellis G, Lutsar I, Yakkundi S, McElnay J, Pandya H, Mulla H, Vaconsin P, Storme T, Rieutord A, Nunn AJ: Risk assessment of neonatal excipient exposure: Lessons from food safety and other areas.

    Adv Drug Deliv Rev 2014, 73C:89-101. OpenURL

  31. Verhamme K, Sturkenboom M: Study designs in paediatric pharmacoepidemiology.

    Eur J Clin Pharmacol 2011, 67(Suppl 1):67-74. OpenURL

  32. White HL, Macpherson AK: Capturing paediatric injury in Ontario: differences in injury incidence using self-reported survey and health service utilisation data.

    Inj Prev 2012, 18:33-37. OpenURL

  33. Melo MCND, Taddei JAD AC, Diniz-Santos DR, May DS, Carneiro NB, Silva LR: Incidence of diarrhea: poor parental recall ability.

    Brazilian J Infect Dis an Off Publ Brazilian Soc Infect Dis 2007, 11:571-579. OpenURL

  34. Alam N, Henry FJ, Rahaman M: Reporting errors in one-week diarrhoea recall surveys: experience from a prospective study in rural Bangladesh.

    Int J Epidemiol 1989, 18:697-700. OpenURL

  35. Knottnerus JA, Tugwell P: Ongoing progress in research on drug exposure and adverse events.

    J Clin Epidemiol 2012, 65:1029-1030. OpenURL

  36. Gagnon JP, Rindress D: Pharmacoeconomics: Identifying The Issues. USA: International Society for Pharmacoeconomics and Outcomes Research; 1998. OpenURL

  37. LeBel M, Ferron L, Masson M, Pichette J, Carrier C: Benzyl alcohol metabolism and elimination in neonates.

    Dev Pharmacol Ther 1988, 11:347-356. OpenURL

  38. Hiller JL, Benda GI, Rahatzad M, Allen JR, David H, Carlson CV, Reynolds JW, Culver DH, Carison CV: Benzyl alcohol toxicity: impact on mortality and intraventricular hemorrhage among very low birth weight infants the online version of this article, along with updated information and services, is located on the world wide web at: benzyl alcohol toxic.

    Pediatrics 1986, 77:500-506. OpenURL

  39. Warrier I, Du W, Natarajan G, Salari V, Aranda J: Patterns of drug utilization in a neonatal intensive care unit.

    J Clin Pharmacol 2006, 46:449-455. OpenURL

  40. Clark RH, Bloom BT, Spitzer AR, Gerstmann DR: Reported medication use in the neonatal intensive care unit: data from a large national data set.

    Pediatrics 2006, 117:1979-1987. OpenURL

  41. Zarb P, Coignard B, Griskeviciene J, Muller A, Vankerckhoven V, Weist K, Goossens M, Vaerenberg S, Hopkins S, Catry B, Monnet D, Goossens H, Suetens C: The European Centre for Disease Prevention and Control (ECDC) pilot point prevalence survey of healthcare-associated infections and antimicrobial use.

    Euro Surveill Bull Eur sur les Mal Transm = Eur Commun Dis Bull 2012., 17(46)

    pii=20316

    OpenURL

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-2288/14/89/prepub