Email updates

Keep up to date with the latest news and content from BMC Public Health and BioMed Central.

Open Access Research article

Advances in childhood immunisation in South Africa: where to now? Programme managers’ views and evidence from systematic reviews

Charles Shey Wiysonge12*, Nthombenhle J Ngcobo3, Prakash M Jeena4, Shabir A Madhi5, Barry D Schoub5, Anthony Hawkridge1, Muki S Shey6 and Gregory D Hussey12

Author affiliations

1 Vaccines for Africa Initiative, Institute of Infectious Disease and Molecular Medicine, University of Cape Town, Cape Town, South Africa

2 Department of Clinical Laboratory Sciences, Division of Medical Microbiology, University of Cape Town, Cape Town, South Africa

3 Expanded Programme on Immunisation, National Department of Health, Pretoria, South Africa

4 Department of Paediatrics and Child Health, University of KwaZulu Natal, Durban, South Africa

5 National Institute for Communicable Diseases, Johannesburg, South Africa

6 Centre for the AIDS Programme of Research in South Africa (CAPRISA), University of KwaZulu-Natal, Durban, South Africa

For all author emails, please log on.

Citation and License

BMC Public Health 2012, 12:578  doi:10.1186/1471-2458-12-578

The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1471-2458/12/578


Received:12 January 2012
Accepted:31 July 2012
Published:31 July 2012

© 2012 Wiysonge et al.; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

The Expanded Programme on Immunisation (EPI) is one of the most powerful and cost-effective public health programmes to improve child survival. We assessed challenges and enablers for the programme in South Africa, as we approach the 2015 deadline for the Millennium Development Goals.

Methods

Between September 2009 and September 2010 we requested national and provincial EPI managers in South Africa to identify key challenges facing EPI, and to propose appropriate solutions. We collated their responses and searched for systematic reviews on the effectiveness of the proposed solutions; in the Health Systems Evidence, Cochrane Library, and PubMed electronic databases. We screened the search outputs, selected systematic reviews, extracted data, and assessed the quality of included reviews (using AMSTAR) and the quality of the evidence (using GRADE) in duplicate; resolving disagreements by discussion and consensus.

Results

Challenges identified by EPI managers were linked to healthcare workers (insufficient knowledge of vaccines and immunisation), the public (anti-immunisation rumours and reluctance from parents), and health system (insufficient financial and human resources). Strategies proposed by managers to overcome the challenges include training, supervision, and audit and feedback; strengthening advocacy and social mobilisation; and sustainable EPI funding schemes, respectively. The findings from reliable systematic reviews indicate that interactive educational meetings, audit and feedback, and supportive supervision improve healthcare worker performance. Structured and interactive communication tools probably increase parents’ understanding of immunisation; and reminders and recall, use of community health workers, conditional cash transfers, and mass media interventions probably increase immunisation coverage. Finally, a national social health insurance scheme is a potential EPI financing mechanism; however, given the absence of high-quality evidence of effects, its implementation should be pilot-tested and the impacts and costs rigorously monitored.

Conclusion

In line with the Millennium Development Goals, we have to ensure that our children’s right to health, development and survival is respected, protected and promoted. EPI is central to this vision. We found numerous promising strategies for improving EPI performance in South Africa. However, their implementation would need to be tailored to local circumstances and accompanied by high-quality monitoring and evaluation. The strength of our approach comes from having a strong framework for interventions before looking for systematic reviews. Without a framework, we would have been driven by what reviews have been done and what is easily researchable; rather than the values and preferences of key immunisation stakeholders.

Background

The Expanded Programme on Immunisation (EPI), which is one of the most powerful and cost-effective public health programmes to improve child survival [1-4], was introduced in South Africa in 1974. The programme, however, remained fragmented because of the system of apartheid until 1995; when the national EPI was formed through the unification of all immunisation services in the country [5]. Since then, there have been significant advances in immunisation services delivery in South Africa, including the introduction of the hepatitis B virus vaccine and Haemophilusinfluenzae type b conjugate vaccine into EPI in 1995 and 1999, respectively. In addition, neonatal tetanus elimination was validated in 2002 and interruption of wild poliovirus transmission was declared in 2006 [5]. More recently, in April 2009, South Africa became the first country in Africa to introduce nationwide routine childhood vaccination against rotavirus and Streptococcus pneumoniae; at no cost to recipients.

Despite these advances, there is some evidence that EPI South Africa faces a number of challenges [6-9]. The vaccination coverage is low (with only two-thirds of children estimated to receive the full series of three doses of the diphtheria-tetanus-pertussis vaccine by one year of age) [8,10], measles outbreaks are frequent [6,9], and community knowledge of immunisation is low [7]. In order to ensure evidence-informed selection and implementation of effective healthcare interventions [11] that would overcome existing barriers and reach every child in South Africa with life-saving vaccines, we conducted an audit to elicit programme managers’ views of the challenges facing EPI and potential solutions for these challenges. Our objectives were: (1) to determine what EPI programme managers in South Africa consider as key barriers to effective implementation of the programme in the country; (2) to determine what EPI programme managers consider as the best interventions to overcome the identified barriers; and (3) to conduct a comprehensive search of peer-reviewed literature, identify, and synthesise current best evidence on the effectiveness of the interventions proposed by the EPI programme managers.

Methods

South Africa has one national and nine provincial (sub-national) EPI managers. In September 2009, we sent e-mails to the national EPI manager and the nine provincial EPI managers, requesting the managers to identify key challenges facing EPI. The two questions asked were:

1. What, in your opinion, are the five key challenges to childhood immunisation in South Africa (List the challenges from the most important challenge to the least important)?

2. What would be the solutions, in your opinion, to these challenges (Only one solution for each challenge)?

Eight days after the initial email request, we sent a follow-up reminder to non-responders. Responses were collated two weeks later. The findings of this first audit were presented at a national immunisation conference in October 2009; which was attended by all key immunisation stakeholders in South Africa [12]. The conference agenda included a critical appraisal of vaccines and vaccine administration, globally and with specific reference to South Africa. In June 2010, the two questions were again sent to all the 10 EPI managers in the country.

The exercise was initially undertaken as a programme evaluation for management purposes, and therefore no ethics review board approval was sought. However, only later we realised that the results could be of interest to a wider audience. All EPI managers who responded to the questionnaire provided consent for their responses to be published. In addition, we obtained the permission of the South African National Department of Health to publish the findings. In this paper we present the key challenges identified by the managers, the key solutions they proposed, and the findings from systematic reviews on the effects of the proposed interventions. Each respondent provided five barriers and five corresponding corrective interventions. In summarising the responses, we treated the barriers identified (five per manager) equally without applying any weighting; irrespective of whether the barriers were identified by the national or provincial managers, or whether the barrier was classified by the manager as most important or least important. We did the same for the proposed interventions (one per identified barrier).

On 30 November 2010 we searched the Health Systems Evidence database, the Cochrane Database of Systematic Reviews, the Database of Abstracts of Reviews of Effectiveness, and PubMed. One author (CSW) conducted the search using the search strategy shown in Table 1. We screened the search outputs in the order in which the databases are listed here, starting with Health Systems Evidence database and ending with PubMed. Figure 1 shows a summary of the search and selection process. When we found more than one systematic review that assessed a particular intervention, we chose the one that was more comprehensive and/or more recent. Two authors (CSW and MSS) independently screened the search results, selected relevant systematic reviews, and assessed the quality of selected reviews using the AMSTAR tool [13]. In particular, we assessed whether the authors of the review reported the study selection criteria; conducted a literature search that was comprehensive enough to avoid publication, language and indexing biases; undertook duplicate study selection and data extraction; used reliable criteria to assess the risk of bias in included studies; reported the characteristics of included studies appropriately; and combined data from included studies using reliable methods. Based on these criteria, we concluded whether the review was well conducted (i.e. reliable) or not. We have only reported data from reviews that we considered to be reliable. At each stage, the two authors compared their results and resolved any disagreements by discussion and consensus.

Table 1. Search strategy for identification of eligible reviews

thumbnailFigure 1. Flow diagram showing the search and selection of reviews.

Finally, we used the GRADE approach [14,15] to assess the quality of the evidence for the effectiveness of the proposed strategies. This method results in an assessment of the quality of a body of evidence as high, moderate, low, or very low. High quality evidence implies that “further research is very unlikely to change our confidence in the estimate of effect”. Moderate quality evidence means that “further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate”. Evidence is considered of low quality if “further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate”, and very low quality if “we have very little confidence in the effect estimate”. We began the rating of the quality of evidence with the study design; evidence from systematic reviews of randomised controlled trials as high-quality and that from systematic reviews of observational studies as low-quality. In addition, five reasons led us to downgrade the quality of evidence from systematic reviews of randomised controlled trials and three to upgrade the quality of evidence from systematic reviews of observational studies. For pooled data from randomised controlled trials, the factors that led to rating down the quality of evidence were risk of bias, heterogeneity, indirectness, imprecision, and publication bias. Regarding risk of bias, concerns that limited our confidence in the evidence include lack of allocation concealment, lack of blinding of outcome assessment, and a large loss to follow-up. Heterogeneity of effects across studies for which there were no compelling explanations also reduced our confidence in the evidence. Indirectness refers to differences between the population, intervention, comparison group and outcome of interest to us, and those included in the relevant reviews e.g. we used the evidence on strategies for improving patients’ understanding of health information as a proxy for evidence on parents’ understanding of the importance of childhood immunisation [16]. For imprecision, if we found that studies included relatively few participants and few events and thus had estimates of effects with wide confidence intervals, we rated down the quality of the evidence. Finally, we downgraded the quality of evidence if there was a high likelihood of publication bias. Furthermore, we upgraded the quality of the evidence if the pooled estimates revealed a large magnitude of effect, if we had negligible concerns about confounders, or if there was a strong dose-response gradient [14].

Results

We received responses from the national EPI manager and five provincial EPI managers in the first round (giving a total of 6 x 5 = 30 identified challenges). For the second round, we received responses from the six EPI managers who sent responses in the first round plus three additional provincial managers (i.e. a total of 9 x 5 = 45 challenges). One provincial manager did not respond. Table 2 shows that consolidation of the responses from the first round revealed the five key EPI challenges to be (in descending order): insufficient knowledge of vaccines and immunisation among staff; staff shortages and high staff turn-over; financial constraints; poor communication among stakeholders (including insufficient advocacy and insufficient social mobilisation); and sub-optimal collaboration between the public and private health sectors. The second round revealed the first four key challenges to be identical to those of the first round; as shown in Table 3. Rumours and reluctance from parents, as well as sub-optimal collaboration between the public and private health sectors occupied fifth position.

Table 2. Summary of responses from first round

Table 3. Summary of responses from second round

The strategies proposed by the EPI managers for addressing the EPI challenges are highlighted in Table 4. For example, the managers proposed training, supportive supervision, and audit and feedback for addressing the insufficient knowledge of immunisation issues among healthcare workers. Through a comprehensive search and selection process (as shown in Figure 1), we identified many well-conducted systematic reviews on strategies for improving the performance of healthcare workers; including educational meetings [17], supportive supervision or educational outreach i.e. “a personal visit by a trained person to health workers in their own settings”[18], audit and feedback i.e. “a summary of performance over a specified period of time given in a verbal or written format”[19], and printed educational materials [20]. Concerning strategies for strengthening vaccine advocacy and social mobilisation, we found reliable reviews on parent reminder and recall systems [21], use of community health workers [22], interactive communication tools [16], mass media interventions [23], and conditional cash transfers i.e. monetary transfers made to disadvantaged households on the condition that they comply with some pre-determined requirements in relation to health care, such as vaccinating their children [24]. Finally, regarding appropriate EPI funding schemes that would address the identified financial and human resource constraints, we found a comprehensive systematic review on the impacts of using social health insurance schemes to finance health care [25]. We provide a description of the objectives, type and number of studies included, participants, settings, interventions, and outcomes assessed in the included systematic reviews in Table 5.

Table 4. Strategies proposed by EPI managers for addressing barriers

Table 5. Characteristics of included systematic reviews

The rating of the quality of currently available evidence on the proposed remedial strategies is shown in Table 6. There exists high-quality evidence [14] showing that interactive educational meetings, audit and feedback, and supportive supervision can improve healthcare worker performance [17-19]. The evidence on the effectiveness of passive distribution of printed educational materials [20] in improving healthcare worker performance is of low quality. Regarding strategies aimed at increasing community demand and support for immunisation services, moderate-quality evidence shows that parent reminder and recall systems [21], use of community health workers [22], mass media interventions [23], and conditional cash transfers [24] may increase routine immunisation coverage. There also exists moderate-quality evidence that structured and interactive communication tools may increase parents’ understanding of the importance of childhood immunisation [16]. Finally, there currently exists very low quality evidence showing that a social health insurance scheme may improve the use of health services in low and middle-income countries [25].

Table 6. Summary of the quality of evidence

Discussion

Decision makers in South Africa need to use considerable judgement about how best to use limited resources they have for maintaining and improving the quality of health care in order to maximise population benefits. In making such decisions, they need to consider the potential areas for quality improvement activities, the likely benefits and costs required to introduce new quality improvement interventions, and the likely benefits and costs as a result of any changes in the behaviour of healthcare workers. With the values and preferences of programme managers as a starting point, we found that the use of interactive educational meetings and workshops, audit and feedback, supportive supervision, parent reminder and recall systems, conditional cash transfers, community health workers, mass media interventions, and interactive communication tools could be effective in improving EPI performance in South Africa. In selecting which combination of interventions to use for which community and at which time point, decision makers would need to consider population characteristics, available resources, and competing priorities. A major strength of our approach comes from having a framework for interventions (i.e. collating managers’ views of the barriers to effective implementation of the EPI programme and their proposals of effective interventions for addressing these barriers) before looking for systematic reviews of effects. If we had searched the literature for effective interventions for improving childhood immunisation, without such a framework, we would have been driven by what systematic reviews have been done and what is easily researchable; rather than the priorities of key immunisation stakeholders.

The settings and designs of the studies included in the systematic reviews that assessed the effects of interactive educational meetings, audit and feedback, and educational outreach visits for improving healthcare worker performance varied widely; but the studies consistently showed that these strategies can improve healthcare worker performance [17-19].The reviews also found that multifaceted interventions may not be any more effective than educational meetings, outreach visits, or audit and feedback alone [17-19]. The consistency of effects across different study designs and healthcare settings and conditions suggests that these findings would be applicable to the EPI in South Africa. The low quality of the evidence on the effects of passive distribution of printed educational materials [20] implies that we have limited confidence in the effectiveness of this strategy to improve the knowledge of vaccine and immunisation issues among healthcare workers in South Africa [14]. Social mobilisation for immunisation may include active community participation, contextualisation of information in the local customs and culture, and involvement of a broad range of stakeholders and the mass media. The moderate quality of the evidence on parent reminder and recall systems [21], community health workers [22], interactive communication tools [16], conditional cash transfers [24], and mass media interventions [23] is an indication that these strategies could have significant effects in mobilising communities and increasing demand for routine childhood immunisation services in South Africa.

Appropriate financing mechanisms are needed to ensure that there are sufficient funds (at all times in all health districts) to recruit and retain a sufficient number of qualified staff, procure adequate quantities of vaccines, purchase and maintain the appropriate vaccine-related equipment and other logistics; and maintain and improve the quality of care. Evidence of effects, population characteristics, societal values, and attitudes are important when selecting funding schemes which aim to ensure universal coverage of health services. Childhood immunisation services are provided free of charge at public health facilities in South Africa [5]. Therefore, sustainable EPI financing mechanisms relevant to the South African context would include the use of general tax revenues or a social health insurance scheme to cover the costs of immunisation services. Social health insurance refers to compulsory health insurance that aims to provide universal coverage. The compulsory nature of social health insurance should reduce adverse selection and enable redistributive mechanisms between poor and affluent segments of the population. However, few examples exist of social health insurance schemes operating at a large scale in low and middle-income countries; none of which provides strong evidence related to their impact [25]. The National Health Insurance (NHI), whose planned introduction in the health sector in South Africa has become a very topical issue, is a type of social health insurance scheme. The South African National Department of Health indicates that the NHI “is based on the key principles of universality, that every South African would be entitled to benefit from the services it covers, and that it would be funded partly by compulsory contributions by all persons who are earning an income and partly by tax. All these funds would be placed in a single pool. This pool would be available to fund all health care in the public and private health sector under conditions that would apply to all health care service providers.” [26].

Our study has some limitations. One provincial manager did not respond to our questions. However, given the consistency of the key challenges across the other eight provinces, we do not think that the EPI challenges in the province of the manager who did not respond would be substantially different from those of the other provinces. There is a possibility that the presentation of findings of the first round to EPI managers may have influenced the responses in the second round of the audit. However, this does not seem to have been the case because new challenges (e.g. reluctance from parents) emerged from the second round. Regardless of these potential shortcomings, we believe that this study is a good reflection of the challenges encountered in the planning, delivering, and monitoring of childhood immunisation services in South Africa. Most of the evidence of effects described above comes from studies conducted in high-income countries, and applicability to South Africa or other low and middle-income countries may be limited. There is thus a need for high-quality studies from low and middle-income countries, assessing the effects of strategies for improving immunisation services. In the meantime, implementation of such strategies in South Africa should be pilot-tested and their impacts and costs rigorously monitored and evaluated [11].

Conclusion

In line with the Millennium Development Goals, we have to ensure that our children’s right to health, development and survival is respected, protected and promoted. EPI is central to this vision. We found numerous promising strategies for improving EPI performance in South Africa. However, their implementation would need to be tailored to local circumstances and accompanied by rigorous monitoring and evaluation. The suggested interventions (by EPI programme managers) for handling immunisation challenges and the availability (or lack) of sound scientific evidence on their effectiveness, emphasise the need for partnerships between health policy makers, programme managers, and researchers in order to ensure that health decisions are always informed by the best available evidence [27,28]. Collaboration between the three stakeholders needs to be continuous, as the results of this audit show that challenges to immunisation may vary from time to time. Such policymaker-implementer-researcher partnerships are needed to create a health system that is effective, equitable, and sustainable; one in which EPI will be better able to meet its obligations.

Competing interests

The authors declare that they have no competing interests.

Authors’ contribution

CSW and GDH conceived the study, CSW and MSS collected and analysed the data, and CSW wrote the first draft of the paper. CSW, NJN, PMJ, SAM, BDS, AH, MSS and GDH made significant contributions to the interpretation of the data, and revision of the manuscript. All authors approved the final version for publication.

Acknowledgement

We are grateful to the EPI managers who responded to the audit and to the National Department of Health of South Africa for giving us permission to publish the findings. We thank the Editor of BMC Public Health and the peer reviewers (Mohammad Khan, Vittal Mogasale, and W. William Schluter) for critical review of an earlier version of this manuscript. Their comments and suggestions helped us to improve the quality of the paper.

References

  1. Bloom DE: The value of vaccination.

    Adv Exp Med Biol 2011, 697:1-8. PubMed Abstract | Publisher Full Text OpenURL

  2. Wiysonge CS, Nomo E, Mawo JN, Ticha JM: Accelerated measles control in sub-Saharan Africa.

    Lancet 2006, 367(9508):394-395. PubMed Abstract | Publisher Full Text OpenURL

  3. Cutts FT, Zaman SM, Enwere G, Jaffar S, Levine OS, Okoko JB, Oluwalana C, Vaughan A, Obaro SK, Leach A, et al.: Efficacy of nine-valent pneumococcal conjugate vaccine against pneumonia and invasive pneumococcal disease in The Gambia: randomised, double-blind, placebo-controlled trial.

    Lancet 2005, 365(9465):1139-1146. PubMed Abstract | Publisher Full Text OpenURL

  4. Andre FE, Booy R, Bock HL, Clemens J, Datta SK, John TJ, Lee BW, Lolekha S, Peltola H, Ruff TA, et al.: Vaccination greatly reduces disease, disability, death and inequity worldwide.

    Bull World Health Organ 2008, 86(2):140-146. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  5. Ngcobo N: The impact of the immunisation programme on vaccine –preventable diseases in South Africa: a review of progress over 10–15 year period.

    South Afr J Epidemiol Infect 2008, 23:9-13. OpenURL

  6. Siegfried N, Wiysonge CS, Pienaar D: Too little, too late: measles epidemic in South Africa.

    Lancet 2010, 376(9736):160. PubMed Abstract | Publisher Full Text OpenURL

  7. Zipursky S, Wiysonge CS, Hussey G: Knowledge and attitudes towards vaccines and immunization among adolescents in South Africa.

    Hum Vaccin 2010, 6(6):455-461. PubMed Abstract | Publisher Full Text OpenURL

  8. WHO:

    South Africa. WHO and UNICEF estimates of immunization coverage. 2010.

    revision. http://www.who.int/immunization_monitoring/data/zaf.pdf webcite (accessed 01 June 2012)

    OpenURL

  9. Schoub BD: Lessons from the 2009 measles epidemic in South Africa.

    S Afr Med J 2011, 101(8):519. PubMed Abstract OpenURL

  10. Burton A, Monasch R, Lautenbach B, Gacic-Dobo M, Neill M, Karimov R, Wolfson L, Jones G, Birmingham M: WHO and UNICEF estimates of national infant immunization coverage: methods and processes.

    Bull World Health Organ 2009, 87(7):535-541. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  11. Wiysonge CS, Lavis JN, Volmink J: Make the money work for health in sub-Saharan Africa.

    Lancet 2009, 373(9670):1174. PubMed Abstract | Publisher Full Text OpenURL

  12. Hussey GD, Ngcobo N, Desai F, Herbert J, Friggens J, Arnot M, Majenge M, Gouws AM, Wiysonge CS: EPI in South Africa - challenges and prospects. In Proceedings of Vaccinology Conference 2009: 19-20 October 2009;Hermanus, South Africa. Edited by Schoub BD. National Institute for Communicable Diseases, South Africa; 2009. OpenURL

  13. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM: Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews.

    BMC Med Res Methodol 2007, 7:10. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  14. Balshem H, Helfand M, Schunemann HJ, Oxman AD, Kunz R, Brozek J, Vist GE, Falck-Ytter Y, Meerpohl J, Norris S, et al.: GRADE guidelines: 3. Rating the quality of evidence.

    J Clin Epidemiol 2011, 64(4):401-406. PubMed Abstract | Publisher Full Text OpenURL

  15. Akl EA, Kennedy C, Konda K, Caceres CF, Horvath T, Ayala G, Doupe A, Gerbase A, Wiysonge CS, Segura ER, et al.: Using GRADE methodology for the development of public health guidelines for the prevention and treatment of HIV and other STIs among men who have sex with men and transgender people.

    BMC Publ Health 2012, 12(1):386. BioMed Central Full Text OpenURL

  16. Trevena LJ, Davey HM, Barratt A, Butow P, Caldwell P: A systematic review on communicating with patients about evidence.

    J Eval Clin Pract 2006, 12(1):13-23. PubMed Abstract | Publisher Full Text OpenURL

  17. Forsetlund L, Bjorndal A, Rashidian A, Jamtvedt G, O’Brien MA, Wolf F, Davis D, Odgaard-Jensen J, Oxman AD: Continuing education meetings and workshops: effects on professional practice and health care outcomes.

    Cochrane Database Syst Rev 2009, 2:CD003030. PubMed Abstract | Publisher Full Text OpenURL

  18. O’Brien MA, Rogers S, Jamtvedt G, Oxman AD, Odgaard-Jensen J, Kristoffersen DT, Forsetlund L, Bainbridge D, Freemantle N, Davis DA, et al.: Educational outreach visits: effects on professional practice and health care outcomes.

    Cochrane Database Syst Rev 2007, 4:CD000409. PubMed Abstract | Publisher Full Text OpenURL

  19. Jamtvedt G, Young JM, Kristoffersen DT, O’Brien MA, Oxman AD: Audit and feedback: effects on professional practice and health care outcomes.

    Cochrane Database Syst Rev 2006, 2:CD000259. PubMed Abstract | Publisher Full Text OpenURL

  20. Farmer AP, Legare F, Turcot L, Grimshaw J, Harvey E, McGowan JL, Wolf F: Printed educational materials: effects on professional practice and health care outcomes.

    Cochrane Database Syst Rev 2008, 3:CD004398. PubMed Abstract | Publisher Full Text OpenURL

  21. Jacobson VJ, Szilagyi P: Patient reminder and patient recall systems to improve immunization rates.

    Cochrane Database Syst Rev 2005, 3:CD003941. PubMed Abstract | Publisher Full Text OpenURL

  22. Lewin S, Munabi-Babigumira S, Glenton C, Daniels K, Bosch-Capblanch X, van Wyk BE, Odgaard-Jensen J, Johansen M, Aja GN, Zwarenstein M, et al.: Lay health workers in primary and community health care for maternal and child health and the management of infectious diseases.

    Cochrane Database Syst Rev 2010, 3:CD004015. PubMed Abstract | Publisher Full Text OpenURL

  23. Grilli R, Ramsay C, Minozzi S: Mass media interventions: effects on health services utilisation.

    Cochrane Database Syst Rev 2002, 1:CD000389. PubMed Abstract | Publisher Full Text OpenURL

  24. Lagarde M, Haines A, Palmer N: The impact of conditional cash transfers on health outcomes and use of health services in low and middle income countries.

    Cochrane Database Syst Rev 2009, 4:CD008137. PubMed Abstract | Publisher Full Text OpenURL

  25. Lagarde M, Palmer N: Evidence from systematic reviews to inform decision making regarding financing mechanisms that improve access to health services for poor people. A policy brief prepared for the International Dialogue on Evidence-Informed Action to Achieve Health Goals in Developing Countries (IDEAHealth). Alliance for Health Policy and Systems Research, Geneva; 2006.

    http://www.who.int/rpc/meetings/HealthFinancingBrief.pdf webcite (accessed 31 December 2010). In.; 2009

    OpenURL

  26. DoH SA:

    National Department of Health of South Africa. Address by Deputy Minister of Health to companies of the South African Medical Device Industry Association.

    http://www.doh.gov.za/docs/sp/2009/sp0716.html webcite (accessed 30 December 2010)

    OpenURL

  27. Wiysonge CS, Muula AS, Kongnyuy EJ, Shey MS, Hussey GD: Lessons and myths in the HIV/AIDS response.

    Lancet 2009, 374(9702):1675.

    author reply 1675-1676

    PubMed Abstract | Publisher Full Text OpenURL

  28. Wiysonge CS, Uthman OA, Ndumbe PM, Hussey GD: Individual and contextual factors associated with low childhood immunisation coverage in sub-Saharan Africa: a multilevel analysis.

    PLoS One 2012, 7(5):e37905. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-2458/12/578/prepub