Email updates

Keep up to date with the latest news and content from BMC Medical Research Methodology and BioMed Central.

Open Access Research article

Do health care institutions value research? A mixed methods study of barriers and facilitators to methodological rigor in pediatric randomized trials

Michele P Hamm16*, Shannon D Scott23, Terry P Klassen4, David Moher5 and Lisa Hartling1

Author affiliations

1 Alberta Research Centre for Health Evidence, Department of Pediatrics, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, AB, Canada

2 Faculty of Nursing, University of Alberta, Edmonton, AB, Canada

3 Department of Pediatrics, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, AB, Canada

4 Department of Pediatrics and Child Health, Faculty of Medicine, University of Manitoba, Winnipeg, MB, Canada

5 Ottawa Hospital Research Institute, Ottawa, ON, Canada

6 Department of Pediatrics, University of Alberta, ECHA 4-482B, 11405-87 Avenue, Edmonton, AB, T5G 1C9, Canada

For all author emails, please log on.

Citation and License

BMC Medical Research Methodology 2012, 12:158  doi:10.1186/1471-2288-12-158


The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1471-2288/12/158


Received:10 February 2012
Accepted:15 October 2012
Published:18 October 2012

© 2012 Hamm et al.; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

Pediatric randomized controlled trials (RCTs) are susceptible to a high risk of bias. We examined the barriers and facilitators that pediatric trialists face in the design and conduct of unbiased trials.

Methods

We used a mixed methods design, with semi-structured interviews building upon the results of a quantitative survey. We surveyed Canadian (n=253) and international (n=600) pediatric trialists regarding their knowledge and awareness of bias and their perceived barriers and facilitators in conducting clinical trials. We then interviewed 13 participants from different subspecialties and geographic locations to gain a more detailed description of how their experiences and attitudes towards research interacted with trial design and conduct.

Results

The survey response rate was 23.0% (186/807). 68.1% of respondents agreed that bias is a problem in pediatric RCTs and 72.0% felt that there is sufficient evidence to support changing some aspects of how trials are conducted. Knowledge related to bias was variable, with inconsistent awareness of study design features that may introduce bias into a study. Interview participants highlighted a lack of formal training in research methods, a negative research culture, and the pragmatics of trial conduct as barriers. Facilitators included contact with knowledgeable and supportive colleagues and infrastructure for research.

Conclusions

A lack of awareness of bias and negative attitudes towards research present significant barriers in terms of conducting methodologically rigorous pediatric RCTs. Knowledge translation efforts must focus on these issues to ensure the relevance and validity of trial results.

Keywords:
Clinical trials as topic; Risk of bias; Pediatrics; Mixed methods

Background

“We as an institution, as a profession, don’t actually sell research as being an important thing that we do in hospitals. And it should be.”

There is a growing body of literature documenting the methodological limitations of published randomized controlled trials (RCTs) in pediatrics [1-7]. Of particular concern is the evidence that RCTs in child health are susceptible to a high risk of bias, increasing the likelihood that reported treatment benefits and/or harms are being exaggerated [8-10]. In order to ensure clinical relevance and to prevent unnecessary and wasteful research, it is crucial that measures are taken to maximize the internal validity of studies that are conducted [11,12]. The global investment in research is enormous, with funding of $100 billion annually [11], plus the time and effort committed by the researchers, clinicians, and children and families. When participants agree to take part in a trial, they expect that the study will be conducted and reported to the highest standard to accurately answer the research question. When this expectation is met, trial results are important in providing children with the best possible treatment; however, when biased research is conducted instead, research dollars and professionals’ time are wasted, and the children’s contributions are unavailing.

Evidence describing the negative impact of bias on RCTs and how to minimize it is available [13-22], as is research on a number of specific challenges inherent in conducting RCTs in pediatrics [23-27], such as recruitment and consent procedures. However, the research-practice gap regarding methodological rigor in this population has not yet been addressed.

As the first step in the development of a knowledge translation strategy to address the reduction of bias in pediatric RCTs, the objective of this study was to determine and describe the barriers and facilitators that pediatric trialists face in the design and conduct of unbiased trials, with an emphasis on the Canadian context. Quantitative survey and qualitative interview data were collected to gain a broad perspective of the problem and researchers’ experiences.

Methods

Design

We used an explanatory mixed methods design, with semi-structured interviews building upon the results of a quantitative survey. We connected data from the two phases to provide detailed descriptions of the barriers and facilitators pediatric trialists face in designing and conducting studies with high internal validity [28]. We obtained ethical approval from the Health Research Ethics Board at the University of Alberta.

Data collection

Quantitative: survey

We sent an Internet-based survey (SurveyMonkey) to a sample of Canadian and international pediatric trialists between September 2010 and February 2011. We searched the Cochrane Central Register of Controlled Trials for pediatric RCTs published in 2008 and 2009 and identified 7,535 articles. Corresponding authors of all relevant trials identified using a Canadian-specific search filter (n=90), and a geographically representative sample of 600 international trialists chosen from a randomly ordered list were invited to participate in the survey. We sent an invitation e-mail with a link to the survey and two reminder e-mails separated by two week intervals. The questionnaire was informed by the Cochrane Collaboration’s Risk of Bias tool [29], the BARRIERS Scale [30], and the framework proposed by Cabana et al. [31], described in Table 1.

Table 1. Tools used for survey development

Twelve methodologists and clinicians evaluated the questionnaire for appropriateness and accuracy of content and 5 national and international pediatric trialists completed pilot testing for clarity and ease of use; we made revisions based on their feedback. The survey included 23 questions and pilot testing indicated that it would take approximately 15 minutes to complete. We developed items to determine: 1) researcher knowledge and awareness of bias; and 2) perceived barriers and facilitators in conducting clinical trials.

Due to a low response rate using the sample described above (154/644; 23.9%), we expanded the survey population to recruit participants from the membership of the Maternal Infant Child and Youth Research Network (MICYRN), a Canadian network linking investigators from 17 academic health centres involved in pediatric clinical research. We identified potential respondents through a publicly available network inventory maintained by MICYRN (http://www.micyrn.ca/Networks.html) and invited all individuals listed as network contacts via email to participate in both the survey and the interview portion of the study (n=163). The survey included an item asking whether respondents would be willing to be contacted for an interview, and if so, to provide their name and preferred means of initial contact. As a result of problems with access to SurveyMonkey, we administered this wave via REDCap, an alternate secure, online application for managing surveys.

Qualitative: interviews

Due to low participation rates from survey respondents, we augmented recruitment with members of MICYRN with trial experience and referrals from participants and established pediatric trialists. We used purposive sampling based upon pediatric subspecialty and geographic location, aiming to reach saturation, which typically occurs around 12 participants [33]. Interviews followed a semi-structured format built upon the results of the survey and were focused on participants’ experiences and attitudes towards conducting pediatric research and how these interacted with the appropriate design and conduct of methodologically sound trials. Each participant was sent an electronic consent form that they signed and returned via fax or email prior to the conduct of the interview. Interviews were 30 to 60 minutes and conducted by telephone by the lead author between April and July 2011. All interviews were recorded and transcribed verbatim.

Data analysis

We analyzed survey data descriptively, using means and standard deviations, medians and interquartile ranges (IQR), or proportions where appropriate. We used content analysis to code interviews, identifying categories in the data and patterns in beliefs and values that could help explain the potential for bias in pediatric RCTs [34]. Coding was conducted by the lead author in consultation with the rest of the study team. We conducted qualitative data collection and analysis concurrently, following an iterative process. We integrated the survey and interview data at the data interpretation phase [35], using the method of connecting data [28]. We used Stata and NVivo to manage quantitative and qualitative data, respectively.

Results

The survey response rate was 23.0% (186/807) and 13 interviews were conducted. Characteristics of the survey and interview participants are described in Tables 2 and 3 and Additional file 1. Survey results were similar across geographic boundaries and were therefore combined and used as a whole to inform a detailed examination of how barriers and facilitators manifest in Canadian research. Results are presented according to their classification as individual, institutional, or policy level factors. Themes are outlined in Table 4.

Additional file 1. a: Demographics of survey population – original sample. b: Demographics of survey population – MICYRN sample.

Format: DOC Size: 37KB Download file

This file can be viewed with: Microsoft Word ViewerOpen Data

Table 2. Demographics of survey population

Table 3. Characteristics of interview participants

Table 4. Interview themes and relevance to risk of bias

Individual factors

Survey findings indicated that 68.1% of respondents agree that bias is a problem in pediatric RCTs and 72.0% reported that they felt there was sufficient evidence to support the need to change some aspects of how RCTs are conducted. However, knowledge of bias among respondents was variable. There was no consistency in responses to questions which asked the respondent to rate the degree to which they agreed that a study design factor would introduce bias into a study. Identification of specific biases was strongest for sequence generation, blinding, and selective outcome reporting, while there was more uncertainty surrounding identification of problems with allocation concealment, incomplete outcome data, and “other sources of bias” (see Additional file 2). Despite this range of awareness of issues relevant to bias, 94.2% of respondents felt confident in their ability to evaluate the quality of published trials.

Additional file 2. Survey results.

Format: DOC Size: 73KB Download file

This file can be viewed with: Microsoft Word ViewerOpen Data

The interviews highlighted two important themes regarding barriers and facilitators at the individual level: knowledge and training regarding research methods, and engagement or ownership in the research process (Table 5). While most survey respondents indicated that bias is a problem, the interview data suggested that trialists often do not have the knowledge to first, recognize, and second, address bias in their studies. They often mentioned a lack of formal training, instead relying on skills learned on the job.

Table 5. Perspectives on individual-level factors

Conversely, a sense of ownership can contribute to a rigorous study design. Actively taking responsibility for the direction of the trial was seen as an opportunity for the investigators to generate enthusiasm, gain support, and educate colleagues about the rationale for rigorous methodology and how it impacts the ability to accurately answer the research question.

"“Listen to what [your colleagues] need to execute the study so that when you develop your protocol, you’ve built that into the approach. Or, if you couldn’t, you’ve at least had that dialogue with them about how scientifically you can’t be as flexible as might be ideal… so that they at least understand the rationale.”"

Institutional factors

While 93.0% of survey respondents demonstrated an interest in learning about and staying current with literature describing and analyzing research methods, only 50.3% felt that they were able to do so due to other constraints. Logistical issues such as meeting institutional requirements (29.2%) and having sufficient staff (30.4%) were identified as challenges, while access to knowledgeable colleagues (92.8%) was identified as the most significant facilitator (see Additional file 2).

Consistently, interview respondents felt that environmental factors within their institutions were not conducive to research, often as the result of perceptions of research (Table 6). They reported the underlying culture to be overwhelmingly negative towards research at all levels, with resistance from trainees, nurses, physicians, administrators, and the pharmaceutical industry. While respondents felt that the products of research tend to be valued once they are demonstrated to improve practice, they stated that there is little appreciation of the methods necessary to achieve that goal. Logistically, respondents felt that having staff dedicated to research would improve this situation, as research procedures would then not consistently be placed at the lowest priority. Recognition of the division between the paradigms of clinical care (e.g., protecting the interests and comfort of patients) and clinical research (e.g., maintaining clinical equipoise) was also identified as a challenge. Clinical investigators often have difficulty with this distinction, which allows for the possibility of compromised trial protocols.

Table 6. Perspectives on institution-level factors

"“I think a lot of investigators really have a hard time separating what decision they would make clinically from what decision they would make as part of a trial… because the feeling is I want to be convenient to the family, and I really know this stuff because I’m an expert in this clinical area, and I don’t think they realize that there’s a pretty clear demarcation between what you do in clinical care and what you do as research.”"

The degree to which institutional barriers were perceived as a threat to research varied according to the size of the site of the respondent. A clear distinction was noted between larger research-intensive institutions and other sites in which researchers struggled due to a lack of infrastructure. Trialists from the former viewed the conduct of research much more positively and generally felt that they had the necessary resources available to conduct rigorous trials, while those from the latter reported greater levels of difficulty positioning their research as an important part of the clinical landscape.

"“Space, resources, and training for research assistants, research managers, graduate students… there’s all sorts of hurdles and headaches around those things that I think most established clinical research programs… already know how to make the system work.”"

Similar to the survey results, cohesive study teams with positive working relationships were reported as the most significant facilitator to conducting rigorous trials. At the institutional level, this often included combining the expertise of experienced trialists, methodologists, and the staff that would be responsible for implementing the trial; however, this integration was more common at sites with more support for research. Positive relationships were also mentioned in the context of subspecialties, with productive research networks across sites enabling researchers to benefit from a collective expertise, as well as facilitating study-specific elements such as the conduct of multi-centre trials. A final facilitator, which was more prominent in certain institutions than others, was a reliable internal review process. Respondents viewed this as a major asset when it was available, but many felt that existing processes were fragmented and inconsistent.

Policy factors

All interview respondents felt a lack of incentive to conduct pediatric trials. Difficulty in securing funding was frequently mentioned, but beyond that, participants reported that it is challenging to justify including financial support for a methodologist in grant application budgets in an environment where the funds awarded are anticipated to be less than requested. In this context, there is less assurance that study teams can include the necessary expertise. Meeting the requirements of several research ethics boards also presents a challenge, particularly with the preponderance of multi-centre trials in pediatrics. With separate ethics approval processes at each institution, the process can be lengthy and protocols are oftentimes changed to meet the inconsistent requests of the individual review panels (Table 7).

Table 7. Perspectives on policy-level factors

Specific biases and pediatric-specific challenges

Addressing specific biases, survey and interview respondents reported challenges with blinding most frequently, which included the cost of providing a placebo, difficulties in blinding non-pharmacological interventions, and blinding all relevant parties, including parents. Other issues that were mentioned included difficulty getting adequate follow up in settings without an established clinician-patient relationship, parental resistance to randomization, and group imbalances due to small sample size.

Discussion

Bias is a recognized concern among pediatric trialists; however they may be lacking the knowledge, willingness, or resources to properly address it. Internal validity did not emerge as a primary concern in the analyses, being overshadowed by issues related to the pragmatics of running a trial and the generalizability of the results. While these issues warrant a great deal of consideration, it is crucial that studies start out being methodologically rigorous as this is a prerequisite for generalizability.

The major barriers to minimizing risk of bias in trials were related to awareness and environment. With little emphasis on research methodology in clinical curricula, many investigators are not adequately prepared to design trials with high levels of internal validity or to recognize and attend to issues as they arise. The existing ad hoc training system very likely contributes to an emphasis on certain areas and a deficit in others, as demonstrated by the disproportionate focus by respondents on issues related to external validity (i.e., generalizability of study results), despite being questioned on issues relevant to internal validity (i.e., avoiding bias through methodologically rigorous design). Additionally, the predominantly negative attitudes surrounding the research process reinforce the acceptance of sub-optimal RCTs. While research findings may be valued, more effort is required to ensure that the importance of high quality research is recognized at all stages, and by all stakeholders. Pediatric oncology is often cited as a model in developing an environment that fosters research. Available infrastructure and consistency in study protocols has resulted in the successful integration of research and clinical care leading to marked improvements in survival and other outcomes [24,36,37]. By embracing research as a critical component of providing best care, rather than viewing it as an imposition, investigators and clinicians in oncology have shown that setting a standard for conducting rigorous trials is an achievable goal with tangible benefits and impressive health outcomes.

Positive relationships that support the development of an interest in research are particularly relevant in an environment in which most training is dependent on mentorship and reinforcement from experienced trialists. Within this context, clinician-scientists have a key role in bridging the gap between the worlds of research and clinical practice. Combining knowledge of proper methodology with an appreciation for the demands of the clinical setting will increase the likelihood of producing both valid and realistic trials. Research networks such as the Canadian Critical Care Trials Group (CCCTG) and Pediatric Emergency Research Canada (PERC) have been quite successful in using this strategy, facilitating high quality trials by promoting a positive culture of research, providing access to individuals with expertise, and offering support and collegiality [38,39].

With a solid evidence base demonstrating the gaps in methodological quality in pediatric RCTs [1-10], the research agenda must now focus on knowledge translation. Using barriers and facilitators identified by the target end-users, it will be important to develop tailored strategies to overcome the gap between what is known about methodological processes and how trials are designed and conducted in practice. This is one of the stated aims of StaR Child Health, an international initiative dedicated to improving the quality of pediatric clinical research [40].

Strengths and limitations

An advantage of this study is that it combines the breadth of survey responses with the depth of interview responses, allowing for a detailed picture of the barriers and facilitators pediatric trialists face in the conduct of methodologically rigorous trials. While the response rate to the survey was low, bringing into question the representativeness of the sample, it was in line with evidence that both electronic surveys [41] and physician surveys [42] are associated with low responses. However, respondents represented a wide range of pediatric specialties, training backgrounds, and geographies, helping to give shape to the subsequent interviews. While they may have represented researchers with a higher level of interest in methodology, the survey responses were used to form the interviews, in which researchers with trial experience were of interest so as to be able to account for the barriers and facilitators in pediatric research with first-hand knowledge. The recruitment of additional survey participants from the membership of MICYRN slightly changed the balance of geographical representation; however the Canadian context was weighted heavily throughout the study, therefore our emphasis was unchanged. The response rate to requests for interview participation was also low, but with our expanded recruitment strategy we were able to achieve saturation. The interviews emphasized the Canadian context and therefore can be used to inform future developments within the national health care and research framework, as well as provide considerations relevant to other settings.

Conclusions

Clinical research is inherently challenging, but these results can be used to focus efforts on improving the validity of trials that are conducted. The evidence is clear that improvement is necessary in pediatric RCTs and a substantial body of knowledge has accumulated around how to minimize bias. Before trials can improve, though, awareness of bias and attitudes towards research must be addressed. Research must be reframed as a valuable component of health care education, practice, and decision-making.

Abbreviations

CCCTG: Canadian Critical Care Trials Group; IQR: Interquartile range; MICYRN: Maternal Infant Child and Youth Research Network; PERC: Pediatric Emergency Research Canada; RCT: Randomized controlled trial.

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

MPH collected and analyzed the data, drafted the manuscript, and integrated critical feedback from the other authors. All of the authors were involved in the development of the proposal and interpretation of the data. All of the authors provided feedback on the revisions to the manuscript and approved the final version submitted for publication.

Acknowledgements

MPH was supported by a KT Canada Fellowship Award from Knowledge Translation Canada. SDS was supported by a New Investigator Award from the Canadian Institutes of Health Research and a Population Health Investigator Award from the Alberta Heritage Foundation for Medical Research. The funder played no role in the study design; in the collection, analysis, and interpretation of data; in writing the manuscript; or in the decision to submit the manuscript for publication.

References

  1. Moss RL, Henry MCW, Dimmitt RA, Rangel S, Geraghty N, Skarsgard ED: The role of prospective randomized clinical trials in pediatric surgery: state of the art?

    J Pediatr Surg 2001, 36:1182-1186. PubMed Abstract | Publisher Full Text OpenURL

  2. Welk B, Afshar K, MacNeily AE: Randomized controlled trials in pediatric urology: room for improvement.

    J Urol 2006, 176:306-310. PubMed Abstract | Publisher Full Text OpenURL

  3. Dulai SK, Slobogean BLT, Beauchamp RD, Mulpuri K: A quality assessment of randomized clinical trials in pediatric orthopaedics.

    J Pediatr Orthop 2007, 27:573-581. PubMed Abstract | Publisher Full Text OpenURL

  4. Uman LS, Chambers CT, McGrath PJ, Kisely S, Matthews D, Hayton K: Assessing the quality of randomized controlled trials examining psychological interventions for pediatric procedural pain: recommendations for quality improvement.

    J Pediatr Psychol 2010, 35:693-703. PubMed Abstract | Publisher Full Text OpenURL

  5. Nor Aripin KNB, Choonara I, Sammons HM: A systematic review of paediatric randomised controlled drug trials published in 2007.

    Arch Dis Child 2010, 95:469-473. PubMed Abstract | Publisher Full Text OpenURL

  6. Thomson D, Hartling L, Cohen E, Vandermeer B, Tjosvold L, Klassen TP: Controlled trials in children: quantity, methodological quality and descriptive characteristics of pediatric controlled trials published 1948–2006.

    PLoS One 2010, 5:e13106. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  7. DeMauro SB, Giaccone A, Kirpalani H, Schmidt B: Quality of reporting of neonatal and infant trials in high-impact journals.

    Pediatrics 2011, 128:e639. PubMed Abstract | Publisher Full Text OpenURL

  8. Hartling L, Ospina M, Liang Y, Dryden DM, Hooton N, Seida JK, Klassen TP: Risk of bias versus quality assessment of randomised controlled trials: cross sectional study.

    BMJ 2009, 339:b4012. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  9. Crocetti MT, Amin DD, Scherer R: Assessment of risk of bias among pediatric randomized controlled trials.

    Pediatrics 2010, 126:298-305. PubMed Abstract | Publisher Full Text OpenURL

  10. Hamm MP, Hartling L, Milne A, Tjosvold L, Vandermeer B, Thomson D, Curtis S, Klassen TP: A descriptive analysis of a representative sample of pediatric randomized controlled trials published in 2007.

    BMC Pediatr 2010, 10:96. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  11. Chalmers I, Glasziou P: Avoidable waste in the production and reporting of research evidence.

    Lancet 2009, 374:86-89. PubMed Abstract | Publisher Full Text OpenURL

  12. Altman DG: The scandal of poor medical research.

    BMJ 1994, 308:283-284. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  13. Als-Nielsen B, Gluud LL, Gluud C: Methodological quality and treatment effects in randomised trials: a review of six empirical studies. 12th Cochrane colloquium. Ottawa, Ontario, Canada: The Cochrane Collaboration; 2004.

    Oct 2–6

    OpenURL

  14. Pildal J, Hrobjartsson A, Jorgensen KJ, Hilden J, Altman DG, Gotzsche PC: Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials.

    Int J Epidemiol 2007, 36:847-857. PubMed Abstract | Publisher Full Text OpenURL

  15. Abraha I, Duca PG, Montedori A: Empirical evidence of bias: modified intention to treat analysis of randomised trials affects estimates of intervention efficacy.

    Z Evid Fortbild Qual Gesundhwes 2008, 102(Suppl VI):9. OpenURL

  16. Von Elm E, Rollin A, Blumle A, Senessie C, Low N, Egger M: Selective reporting of outcomes of drug trials? comparison of study protocols and published articles. 14th Cochrane colloquium. Dublin, Ireland: The Cochrane Collaboration; 2006.

    Oct 23–26

    OpenURL

  17. Dwan K, Altman DG, Amaiz JA, Bloom J, Chan AW, Cronin E, Decullier E, Easterbrook PJ, Von Elm E, Gamble C, Ghersi D, Ioannidis JP, Simes J, Williamson PR: Systematic review of the empirical evidence of study publication bias and outcome reporting bias.

    PLoS One 2008, 3:e3081. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  18. Bassler D, Ferreira-Gonzalez I, Briel M, Cook DJ, Devereaux PJ, Heels-Ansdell D, Kirpalani H, Meade MO, Montori VM, Rozenberg A, Schunemann HJ, Guyatt GH: Systematic reviewers neglect bias that results from trials stopped early for benefit.

    J Clin Epidemiol 2007, 60:869-873. PubMed Abstract | Publisher Full Text OpenURL

  19. Montori VM, Devereaux PJ, Adhikari NK, Burns KE, Eggert CH, Briel M, Lacchetti C, Leung TW, Darling E, Bryant DM, Bucher HC, Schunemann HJ, Meade MO, Cook DJ, Erwin PJ, Sood A, Sood R, Lo B, Thompson CA, Zhou Q, Mills E, Guyatt GH: Randomized trials stopped early for benefit: a systematic review.

    JAMA 2005, 294:2203-2209. PubMed Abstract | Publisher Full Text OpenURL

  20. Bekelman JE, Li Y, Gross CP: Scope and impact of financial conflicts of interest in biomedical research: a systematic review.

    JAMA 2003, 298:454-465. OpenURL

  21. Lexchin J, Bero LA, Djulbegovic B, Clark O: Pharmaceutical industry sponsorship and research outcome and quality: systematic review.

    BMJ 2003, 326:1167-1170. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  22. Sismondo S: Pharmaceutical company funding and its consequences: a qualitative systematic review.

    Contemp Clin Trials 2008, 29:109-113. PubMed Abstract | Publisher Full Text OpenURL

  23. Caldwell PHY, Butow PN, Craig JC: Pediatricians’ attitudes toward randomized controlled trials involving children.

    J Pediatr 2002, 141:798-803. PubMed Abstract | Publisher Full Text OpenURL

  24. Cohen E, Shaul RZ: Beyond the therapeutic orphan: children and clinical trials.

    Pediatr Health 2008, 2:151-159. Publisher Full Text OpenURL

  25. Rheims S, Cucherat M, Arzimanoglou A, Ryvlin P: Greater response to placebo in children than in adults: a systematic review and meta-analysis in drug-resistant partial epilepsy.

    PLoS Med 2008, 5:e166. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  26. Tishler CL, Reiss NS: Pediatric drug-trial recruitment: enticement without coercion.

    Pediatrics 2011, 127:949-954. PubMed Abstract | Publisher Full Text OpenURL

  27. Ballard HO, Shook LA, Desai NS, Anand KJ: Neonatal research and the validity of informed consent obtained in the perinatal period.

    J Perinatol 2004, 24:409-415. PubMed Abstract | Publisher Full Text OpenURL

  28. Creswell JW, Plano Clark VL: Designing and conducting mixed methods research. 2nd edition. Los Angeles, CA: Sage Publications; 2011. OpenURL

  29. Higgins JPT, Green S: Cochrane handbook for systematic reviews of interventions. Version 5.1.0. The Cochrane Collaboration; 2011.

    Available from www.cochrane-handbook.org webcite

    OpenURL

  30. Funk SG, Champagne MT, Wiese RA, Tornquist EM: BARRIERS: the barriers to research utilization scale.

    Appl Nurs Res 1991, 4:39-45. PubMed Abstract | Publisher Full Text OpenURL

  31. Cabana MD, Rand CS, Powe NR: Why don’t physicians follow clinical practice guidelines? a framework for improvement.

    JAMA 1999, 282:1458-1465. PubMed Abstract | Publisher Full Text OpenURL

  32. Kajermo KN, Boström AM, Thompson DS, Hutchinson AM, Estabrooks CA, Wallin L: The BARRIERS scale – the barriers to research utilization scale: a systematic review.

    Implement Sci 2010, 5:32. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  33. Guest G, Bunce A, Johnson L: How many interviews are enough? an experiment with data saturation and variability.

    Field Methods 2006, 18:59-82. Publisher Full Text OpenURL

  34. Hsieh HF, Shannon SE: Three approaches to qualitative content analysis.

    Qual Health Res 2005, 15:1277-1288. PubMed Abstract | Publisher Full Text OpenURL

  35. Morse J, Niehaus L: Mixed methods design: principles and procedures. Walnut Creek, CA: Left Coast Press; 2009. OpenURL

  36. Caldwell PHY, Murphy SB, Butow PN, Craig JC: Clinical trials in children.

    Lancet 2004, 364:803-811. PubMed Abstract | Publisher Full Text OpenURL

  37. Unguru Y: The successful integration of research and care: how pediatric oncology became the subspecialty in which research defines the standard of care.

    Pediatr Blood Cancer 2011, 56:1019-1025. PubMed Abstract | Publisher Full Text OpenURL

  38. Marshall JC, Cook DJ, Canadian Critical Care Trials Group: Investigator-led clinical research consortia: the Canadian critical care trials group.

    Crit Care Med 2009, 37(1 Suppl):S165-S172. PubMed Abstract | Publisher Full Text OpenURL

  39. Klassen TP, Acworth J, Bialy L, Black K, Chamberlain JM, Cheng N, Dalziel S, Fernandes RM, Fitzpatrick E, Johnson DW, Kuppermann N, Macias CG, Newton M, Osmond MH, Plint A, Valerio P, Waisman Y, PERN: Pediatric emergency research networks: a global initiative in pediatric emergency medicine.

    Pediatr Emerg Care 2010, 26:541-543. PubMed Abstract | Publisher Full Text OpenURL

  40. Hartling L, Wittmeier KDM, van der Lee JH, Klassen TP, Craig JC, Offringa M, StaR Child Health group: StaR child health: developing evidence-based guidance for the design, conduct, and reporting of pediatric trials.

    Clin Pharmacol Ther 2011, 90:727-731. PubMed Abstract | Publisher Full Text OpenURL

  41. Wilson PM, Petticrew M, Calnan M, Nazarteh I: Effects of a financial incentive on health researchers’ response to an online survey: a randomized controlled trial.

    J Med Internet Res 2010, 12:e13. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  42. VanGeest JB, Johnson TP, Welch VL: Methodologies for improving response rates in surveys of physicians.

    Eval Health Prof 2007, 30:303-321. PubMed Abstract | Publisher Full Text OpenURL

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-2288/12/158/prepub