Email updates

Keep up to date with the latest news and content from BMC Medical Research Methodology and BioMed Central.

Open Access Highly Accessed Research article

Using qualitative synthesis to explore heterogeneity of complex interventions

Bridget Candy1*, Michael King2, Louise Jones3 and Sandy Oliver4

Author Affiliations

1 Marie Curie Palliative Care Research Unit, UCL Mental Health Sciences Unit, University College Medical School, Charles Bell House, 67-73 Riding House Street, London W1W 7EJ, UK

2 Michael King, Professor of Primary Care Psychiatry and Co-Director of PRIMENT Clinical Trials Unit, UCL Mental Health Sciences Unit, University College Medical School, Charles Bell House, 67-73 Riding House Street, London W1W 7EJ, UK

3 Louise Jones, Head of Marie Curie Palliative Care Research Unit, UCL Mental Health Sciences Unit, University College Medical School, Charles Bell House, 67-73 Riding House Street, London W1W 7EJ, UK

4 Sandy Oliver, Professor of Public Policy, EPPI-Centre, Social Science Research Unit, Institute of Education, 20 Bedford Way, London WC1H 0AL, UK

For all author emails, please log on.

BMC Medical Research Methodology 2011, 11:124  doi:10.1186/1471-2288-11-124


The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1471-2288/11/124


Received:5 October 2010
Accepted:26 August 2011
Published:26 August 2011

© 2011 Candy et al; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

Including qualitative evidence on patients’ perspectives in systematic reviews of complex interventions may reveal reasons for variation in trial findings. This is particularly the case when the intervention is for a long-term disease, as management may rely heavily on the efforts of the patient. Inclusion though seldom happens, possibly because of methodological challenges, and when it does occur the different forms of evidence are often kept separate. To explore heterogeneity in trial findings, we tested a novel approach to integrate qualitative review evidence on patients' perspectives with evidence from a Cochrane systematic review.

Methods

We used, as a framework for a matrix, evidence from a qualitative review on patients’ perspectives on helping them manage their disease. We then logged in the matrix whether the interventions identified in a Cochrane review corresponded with the patient perspectives on how to help them. We then explored correspondence.

The Cochrane review we used included 19 trials of interventions to improve adherence to therapy in HIV/AIDS patients. The qualitative review we used included 23 studies on HIV/AIDS patients' perspectives on adherence; it translated the themes identified across the studies into recommendations in how to help patients adhere.

Both reviews assessed quality. In the qualitative review they found no difference in findings between the better quality studies and the weaker ones. In the Cochrane review they were unable to explore the impact of quality in subgroup analysis because so few studies were of good quality.

Results

Matrix tabulation of interventions and patients' perspectives identified a range of priorities raised by people infected with HIV-1 that were not addressed in evaluated interventions. Tabulation of the more robust trials revealed that interventions that significantly improved adherence contained more components considered important by patients than interventions where no statistically significant effect was found.

Conclusions

This simple approach breaks new ground in cross tabulating qualitative evidence with the characteristics of trialled interventions. In doing so it tests the assumption that patients are more likely to adhere to interventions that match more closely with their concerns. The potential of this approach in exploring varying content and rates of success in trialled complex interventions deserves further evaluation.

Background

Complex multi-component healthcare interventions are challenging to evaluate [1]; variation between trials in the nature of an intervention and its effect can make it difficult to derive conclusions in systematic reviews [2,3]. The Cochrane handbook for systematic reviews acknowledges that inclusion of evidence on patients' perspectives can potentially reveal reasons for trial variation in effect [4]. Such insight, which is commonly derived from qualitative research, may inform the development of more appropriate interventions. This may particularly be the case for interventions for a long-term condition, where management can rely heavily on the patient as a co-producer of health [5]. Including such evidence with evidence on effectiveness rarely occurs. This is possibly because of the considerable methodological challenges involved [6,7]. As there are so few worked examples, a researcher attempting to do so is forced to make subjective decisions that will affect the outcome of the project. These include how the qualitative evidence is collected and analysed, and how it is used with the trial evidence.

In the worked examples available the different forms of evidence are often not brought together in analysis [8]. Instead they are reported in separate papers, where any integration of their findings is reserved for the discussion section. However, combining analytically the different types of data has the potential to reveal more about the relationship between the different forms of evidence. A realist synthesis does this by using other evidence to help explain the success or failure of a complex intervention, such as the theories or mechanisms postulated by the trial's authors or that are generated from other sources including patients [9]. However this approach, described as 'a logic of enquiry' [10, p 32] rather than a method, lacks transparency, thereby departing from the more explicit and staged approaches to reduce bias, which have distinguished and helped legitimise systematic review methods [11].

We used a contrasting approach, which conforms more to the conventions of systematic reviewing. It uses the existing primary research on a topic that meets certain criteria. It is also explicit in how it assesses and analyses the evidence. The approach explores mixed data correspondence, by tabulating in a matrix the findings of difference sources of evidence. This way of mixing the data has been used by other researchers to combine two reviews; one on patient perspectives and one on effectiveness [12]. The researchers used two-by-two matrices to link the findings from qualitative studies about children's perspectives on healthy eating with trial evidence on the effectiveness of an intervention to promote healthy eating in children. Population of the matrix represented whether there was correspondence between the two sources of data. The matrices provided an explanation for how components of the intervention contributed to its effect; for instance, interventions that increased vegetable consumption the most were those that came closest to children's views by not focusing on the health benefits, or conflating fruit and vegetables.

Constructing an evidence matrix that goes beyond four categories (i.e. two-by-two matrices) is undertaken in Qualitative Comparative Analysis (QCA) [13]. It is the first stage in data analysis, which then proceeds, if sufficient data are available, by using algorithmic logic to identify differences in data correspondence relative to the outcome of interest. By including more evidence the matrices as they are used in QCA can explore more of the relationships between the data. This approach has been used in public health topics, for instance in exploring across countries socio-economic factors relating to suicide and traffic injury [14]. We are not aware that this way of combining data has been used to explore complex healthcare interventions.

In this study we took all evidence from a qualitative review of patient's perspectives on helping them manage their disease and logged in matrices the degree to which interventions identified in a complementary Cochrane review corresponded with those perspectives.

Method

Data sources and selection criteria

The study used data from two published systematic reviews; one a Cochrane review on the effectiveness of an intervention, the other a qualitative review on patients' perspectives. We selected the two reviews for this study based on five criteria. The first two were on the appropriateness of the topic: (1) the disease in question required long-term self-management, and (2) the reviews' research questions closely complemented each other in disease topic. The second two were on the appropriateness of the findings: (3) the effectiveness review evaluated an intervention promoting patient self-management, identified relevant trials, and the findings of the individual trials varied in size of effect, and (4) the qualitative review analysed evidence on patients' perspectives by considering how they may contribute to appropriateness of therapies. The fifth criterion was that both reviews needed to report procedures to minimise bias in their findings. Specifically for the Cochrane review we required that they only included randomised controlled trials, and the quality of the trials included was taken into account in their findings. For the qualitative review, that they assessed quality using criteria commonly included in appraisal tools, such as those included in Critical Appraisal Skills Programme for qualitative studies [15]. Such criteria could include whether measures existed to assure validity of data collection, and that data analyses were sufficiently rigorous. We also assessed the methodological quality of the qualitative and the Cochrane review. Methods and reporting standards for undertaking a review of qualitative studies are under-developed; therefore methodological quality was assessed for both reviews using the AMSTAR [16] measurement tool to assess the methodological quality of systematic reviews. This 11-item tool includes questions on whether the review processes were undertaken in duplicate, whether the characteristics of the included studies were included, and whether the scientific quality of the included studies was assessed and documented. This tool has not been specifically designed for reviews that include qualitative evidence; therefore we also followed guidelines on how to assess quality in qualitative primary research [15,17,18]. These included assessing the validity of data by triangulation, whether methods were clear on data collection and that the findings are relevant in that they can be generalised beyond settings in which they were generated.

Complementary reviews in HIV/AIDS

From searches in the Cochrane library and in Medline a complementary pair of reviews was identified that matched our inclusion criteria [19,20]. One review combined evidence from qualitative studies on what is known from the perspectives of HIV/AIDS patients about the problems associated with adhering to highly active antiretroviral therapy (HAART) [20]. The review identified 23 eligible studies, giving a combined total sample of 916 participants. The review authors' quality assessment found half the studies were rated as poor quality but, given their findings were in most respects comparable with the findings of the other studies, they were not excluded. The review authors describe the analysis of the studies as proceeding by 'reading the publication several times during which the findings were coded inductively in thematic groups and compared'. Thirteen primary themes were identified from the 23 studies and all themes, apart from one on homelessness, were identified in at least 7 studies. In their discussion of their findings the reviewers translated the themes into recommendations for promoting adherence. To help distinguish the type of recommendations they were grouped by WHO classification of risk factors influencing adherence to long-term therapies, namely relating to therapy, disease status i.e. having HIV-1, patient factors such as motivation to take therapy, healthcare team and system, and socio-economic circumstances [21].

The Cochrane systematic review evaluated the effectiveness of support and education strategies in improving HIV/AIDS patients' adherence to HAART [19]. This review included 19 randomised controlled trials (RCTs) giving a total sample of 2,159 participants. The reviewers did not combine trial data in a meta-analysis because of heterogeneity in population, intervention, and outcome assessment. They describe the populations across the trials as heterogeneous; specifically while most evaluated the intervention in a mixed general population others focused on high-risk groups, or particular populations such as women, children or an ethnic minority group. However, likewise there were differences in the study populations evaluated in the qualitative review. Whilst, all interventions were directed at the patient, and all were support and education-based interventions intended to improve adherence to HAART, they differed in content. A few involved cognitive management strategies, and one intervention indirectly targeted adherence by aiming to reduce risky sexual behaviours. Outcome assessment differed in how adherence was measured including, for example, pharmacy refill and self-report of the number of missed doses. The Cochrane reviewers found ten interventions were effective in increasing adherence to the prescribed medication; these were delivered to individuals rather than in groups, included six out of the seven interventions provided over 12 weeks, and six out of the eight interventions that aimed to improve practical medication skills such as by using memory aids. The reviewers used eight criteria to assess the validity of the trials findings and provided reasons for selection of the criteria based on design issues specific to adherence enhancing intervention studies [22] and the Consort Statement [23]. The criteria were: details on randomisation, concealment of allocation, an objective measure of adherence, intervention and control patients receiving similar trial contact time, follow-up for six months or longer, attrition of 20% of less in both trial arms at the end of the intervention and at six months follow-up, and that the data from all participants were analysed. Overall, the trials were reported as methodologically weak. The median criteria fulfilled were two out of a possible eight; three trials fulfilled none of the criteria. There were two distinct clusters of trials; a few fulfilling at least half of the eight criteria, and the majority fulfilling none or few criteria. As the reviewers were unable to undertake meta-analysis they could not explore in sensitivity analysis the impact of trial quality on their findings.

Data extracted from qualitative review

We first extracted from the qualitative reviews' discussion section the recommendations for promoting adherence from the qualitative review. These we listed as 21 recommendations (Appendix 1). In accordance with the authors of the qualitative review we grouped the recommendations by the WHO classification of risk factors influencing adherence to long-term therapy [21].

Extraction and coding of data from Cochrane review

We extracted data, from the Cochrane review, on the effectiveness of the interventions. Given that the trials measured adherence in various ways we could not standardise trial findings for the purposes of comparison. Instead we extracted data on whether the intervention was found to improve adherence significantly. One trial was excluded as it did not provide a between trial group comparison of the benefit of the intervention [24].

We planned to extract descriptions of the trialled interventions from the Cochrane review but, as the review only provided limited details on their composition, we sought this information from the original published papers reporting the trials findings. This information was central to our analysis, so we set additional inclusion criteria. Firstly, that at least a paragraph in the methods section of the paper was dedicated to describing the intervention content. Two trials were subsequently excluded [25,26]. We assessed the remaining 16 trial papers' descriptions of their interventions according to guidelines from the extension of the CONSORT statement to trials of non-pharmacologic treatment [27]. Specifically, we found all trial papers provided a description of different components of the interventions, but two did not report that the intervention was tailored to the individual and so were excluded [28,29].

Review of the 14 remaining trials revealed that all interventions aimed to improve medication management skills; this involved various techniques, for example, adaptation of dose schedules according to participants' risk factors and/or provision of adherence gadgets such as timers [30-43]. Apart from one where the comparative trial arm involved additional medication monitoring [39], all the interventions were compared with usual care. For each trial we extracted, as appropriate, the details of the intervention on to a standardised data sheet listing the 21 recommendations derived from the qualitative review. We only entered data on the trialled intervention for matches identified between components and recommendations. For example, an intervention in one trial fulfilled the recommendation 'Healthcare providers should acquire and use insight into possible factors influencing each individual patient before starting treatment' because it was described as 'In the first visit the nurse determines the specific problems that exist with respect to knowledge and understanding of HIV infection and current antiretroviral medication adherence'. We assessed each correspondence between the trial intervention components on whether it was a full or, if appropriate to the recommendation, a partial match. For example, a partial match would be reported if for the recommendation 'Clear information should be given on side effects how to manage side effect, including those that may be unpleasant and distressing', the trial authors reported that side effects were discussed but did not report that they provided information on how to manage them.

Initially, one reviewer extracted the trial data. This extraction was checked independently by two additional observers, who, although not disagreeing with the matches identified, found a small number that were missed by the other. Therefore, a further independent data extraction was undertaken by a layperson masked to the findings of the trial. Each time a passage describing the intervention matched a recommendation the layperson underlined and marked it with which recommendation it matched. This transparent method of data matching allowed us to confirm that we had not missed any matches in the unmarked text.

Matrix for configuring and comparing the data

We used the data sheet extractions to build a matrix. Each column represented a recommendation for an intervention component derived from the qualitative review, while each row represented whether an individual trial's intervention matched any of the recommendations. We marked a cell 'X' when a commonality (a correspondence) was found between a patient recommendation and an intervention component. It was marked 'x' if only a partial correspondence was identified. It was left blank when the intervention content did not correspond with a patient-based recommendation. The final cell, per trial row, indicated whether or not the intervention was found to be effective. We used the WHO classification of risk factors influencing adherence to long-term therapy [21] as a framework to group the links found between the interventions and the recommendations. See Table 1.

Table 1. Tabulation of the four higher quality trials

We built two matrices. Many of the trials in the Cochrane review were methodologically weak. Therefore in our first matrix we used as a cut-off to explore heterogeneity of effect the cluster of four trials that fulfilled at least half of the set quality criteria. We explored visually whether there were differences in configurational matches in the mixed review data between the two successful interventions and the two that had no effect. In the second matrix we entered the data extractions for all trials to explore how well the patient recommendations were reflected in the components of the evaluated interventions.

Results

The two higher quality trials in which the intervention significantly improved adherence addressed a wider range of barriers and facilitators that patients felt were important to adherence than the two higher quality trials where the intervention did not improve adherence. These latter trials did not address three types of barriers and facilitators; namely, general and overarching factors, those relating to HIV-1 status, and those relating to social circumstances (Table 1). The effective interventions both matched two recommendations that the ineffective interventions did not; specifically 'To get patients to describe their own behaviour so as to pick up any risky patterns' and 'To develop a trusting relationship with the patient'. There were also two recommendations that the effective trials either matched or partially matched and where no match was identified in the interventions found not to improve adherence. These were: 'To enquire for each patient before starting the treatment into the possible factors influencing adherence' and 'Regular discussion of the details of circumstances that lead to forgetting medication can reveal aspects that need attention to improve adherence e.g. capacity to organise life and activities, and to anticipate risky situations'. Nevertheless, there was one recommendation that was matched in the ineffective interventions that was not matched in the effective interventions, namely 'Clear information should be given on how to manage side effects, including those that may be unpleasant and distressing'.

When we brought together in the second matrix the data extraction sheets for all 14 trials, we found few trials had included recommendations relating to HIV-1 status, or for personal social circumstances (Table 2). This included, for example, the recommendations derived from the qualitative review that 'Secrecy is threatened by taking treatment, therefore the possibility of disclosure should be discussed as openness leads to higher adherence. If disclosure is not an option, the patient can be advised how to take medication in secret to avoid skipping doses' and 'To acquire insight into a patient's social support systems, and counsel the patient on how to use such support'.

Table 2. Tabulation of all trials

Discussion

We used a simple approach to explore whether qualitative review evidence can explain the findings of a systematic review of effectiveness of a complex intervention. The approach breaks new ground by comparing in a matrix data derived from a qualitative review with data from a quantitative systematic review. By tabulating these two forms of data we were able to explore how patients' views matched the various components of trialled interventions.

The authors of the Cochrane review found that interventions targeting practical medication management skills were more often linked to successful adherence outcomes than those that did not target such skills [19]. By using our approach other information about the intervention components was revealed. Firstly, successful interventions contained a broader range of components that worked on barriers and facilitators to anti-retroviral drug adherence considered important by people infected with HIV-1. This finding is perhaps of no surprise but on the other hand this has not, to our knowledge, been formally explored before. What is surprising is that not all types of priorities raised by patients had been fully evaluated in the interventions included in the most recent update of the Cochrane review.

Limitations

In this study any conclusions drawn on the differences between evaluated interventions are tentative as none of the trials was without risk of biased findings. Furthermore, conclusions cannot be drawn on which components are essential or superfluous, or on whether the recommendations not taken up in the reviewed trials would have any impact on adherence if included in further trials. The reason for bringing quantitative trial data together with qualitative evidence on patients' views is not to demonstrate a causal link; rather it is hypothesis generating by providing possible explanations for variation between trial findings.

Incomplete reporting in trial reports poses a threat to the validity of this approach, which is dependent on the quality of information provided on the interventions. We did attempt to ask for further information with the authors where an email address was provided. The approach was abandoned, as few responded and not all who responded provided information. However, to reduce the risk of incomplete reporting we only included trials that devoted a paragraph or more in their methods section to describing the intervention and which fulfilled the extension of the CONSORT Statement to trials of non-pharmacologic treatment on reporting of intervention content [27]. It could also be argued that while trial authors may not report all aspects of an intervention in the paper publishing their findings, they are likely to report the major components. While this has not been tested, a study of 80 papers describing evaluations of complex interventions found that the most common aspect missing from descriptions of treatments was detail on intervention process as opposed to content, such as timing of treatments [44]. Another limitation in this approach is our subjective evaluation of matches between the qualitative studies and the trials. However, to reduce this potential bias the matches identified were checked several times. Although neither of the first two checks disagreed with what had been extracted in the first extraction, each identified matches that were missed. The third check was an independent data extraction by a layperson who was masked to the trial findings. The transparent method of data matching in this final check minimised the likelihood that any further matches were missed. Finally, we were reliant on the skills of the researchers of the two published reviewers we used. To reduce this limitation we only selected reviews of a high standard.

Implications

Systematic reviews select and combine all high quality randomised trials of a given intervention. They provide the highest quality evidence on effectiveness of a treatment or service. The approach presented here aims, by providing more evidence on intervention components, to increase the potential learning from a systematic review of a complex intervention. This information could lead to the development of more appropriate interventions.

There are a growing number of approaches for combining evidence from different types of studies. Practical experience and worked examples of different approaches are limited. This paper adds to this literature. However, it is uncertain how many other health topics would be suitable for the approach demonstrated in this paper, particularly as qualitative syntheses remain relatively uncommon and methods for qualitative synthesis vary across a range of dimensions [45]. The qualitative review we used in this study used a thematic synthesis approach [46]. Specifically, it aimed to identify all available data and to assess the quality of the studies included. The analysis involved coding of the findings of primary studies and the organisation of these 'free codes' into themes. Other researchers have explored whether a meta-ethnography of qualitative studies could potentially inform a Cochrane review on the effectiveness of interventions to promote adherence to tuberculosis treatment. Meta-ethnography was not developed as a method of systematic review but rather as a way of combining individual studies in one analysis. It involves several stages; a central stage is to translate studies into one another by in-depth review of key concepts or themes across studies [47]. One group that used the findings of meta-ethnography with evidence from a Cochrane review on adherence to tuberculosis therapy reported difficulties in translating their qualitative synthesis into recommendations for interventions. They suggested that meta-ethnography is more suitable for generating models or higher order theories of behaviour or experiences [48]. We stress however, that where reviews of studies on patients' perspectives do not present their data as recommendations in relation to a therapy, then our approach would be difficult.

The potential of the approach tested here should be further explored. While the number of reviews of qualitative evidence on patients' perspectives is small, it is slowly growing. The approach could be used to explore whether combinations of intervention components matching patients' views vary with the trials' effect sizes. Statistical analysis of the matrix data may be restricted because of the limited data points that mixed review evidence could provide but further analysis could explore, in a review with more methodologically robust trials, the value of using the algorithmic logic of QCA to explore variation in trial findings. The potential of QCA in combining qualitative and qualitative evidence synthesis in health care research is recognised by review methodologists [7] and, while we were unable to explore its full potential in the study presented here, we are currently doing so using evidence from a Cochrane systematic review where there are more trials that can be combined with qualitative review evidence [49].

Conclusions

Our approach breaks new ground by combining qualitative and quantitative data in matrices that show whether effective interventions are more likely to contain components patients believe are important than ineffective interventions. In short, it directly tests the assumption that patients are more likely to stick to interventions that contain more of their concerns. The value of this approach in exploring heterogeneity in trial content and findings deserves further evaluation. We encourage authors publishing trial findings to provide an adequate description of their intervention. We also call for more reviews of qualitative evidence on patients' perspectives and, in particular, qualitative reviews that complement existing reviews of trials.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

All authors contributed to the conception, design and interpretation of the data. BC drafted the article and all authors revised it and approved the final version

Appendix 1: Recommendations for adherence interventions drawn from the qualitative review on HIV/AIDS patient's perspectives*

General and overarching

1. Adherence is a dynamic phenomenon in which the interplay of a number of influences varies over time. Therefore ongoing attention to adherence should have highest priority.

2. To enquire into the possible factors influencing adherence for each individual patient before starting treatment.

3. To use insight into the possible adherence factors influencing each individual patient before starting treatment.

Therapy related

4. Medication should be adapted to life rather than life to medication: e.g. the provider should encourage the use of a watch or pillbox with an alarm to remind a patient to take medication and to prevent (unwanted) disclosure.

5. Clear information should be given on side effects how to manage side effect, including those that may be unpleasant and distressing.

6. During every visit any ambivalence towards medications should be discussed.

Condition related

7. Patients' acceptance of their HIV status should be discussed on a regular basis, bearing in mind that taking medication can renew confrontation with diagnosis.

8. Secrecy is threatened by taking treatment. Therefore the possibility of disclosure should be discussed as openness leads to higher adherence: if disclosure is not an option, the patient can be advised how to take medication in secret to avoid skipping doses.

Patient related

9. Feedback about positive reactions of the body to treatment should be provided at each visit to promote adherence e.g. decreased viral load.

10. Pointing out the value of treatment to a patient's life enhances motivation.

11. Information transfer appropriate to a patient's level of understanding will lead to the patient having correct knowledge of what constitutes good adherence practice.

12. It is important to get patients to describe their own behaviour so as to pick up any risky patterns.

13. Regular discussion of the circumstances that lead to forgetting medication can reveal aspects that need attention to improve adherence e.g. capacity to organise life and activities, and to anticipate risky situations.

14. Depression or substance misuse should be managed as first priority.

Healthcare related

15. To develop a trusting relationship with the patient.

16. To give clear instructions on how to take medication.

17. To explain the relationship between adherence and viral load.

18. To offer good medical follow up.

Socio-economic related

19. To acquire insight into a patient's social support systems, and counselled the patient on how to use such support.

20. Attention to possible negative social circumstances e.g. mothers of young children may need help to fit medication into hectic schedules.

21. Social support has to be substantial and practical.

*Derived from Vervoort review [20]and grouped by WHO [21]type of adherence risk factor

Acknowledgements

Marie Curie Cancer Care funded this project. The funding body for this project had no role in the design or implementation, write-up or submission of this project. We are grateful to Drs Marc Lipman and Jason Myers for commenting on earlier drafts of this paper, Karen Harrison for checking our data extraction and Paul Tuckey for independently extracting the trial data.

References

  1. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M: Developing and evaluating complex interventions: new guidance. UK Medical Research Council; 2008. OpenURL

  2. Lenz M, Steckelberg A, Muhlhauser I: Patient education programmes and decision aids- evaluation of complex interventions.

    Av Diabetol 2008, 24:443-452. OpenURL

  3. Shepperd S, Lewin S, Struas S, Clarke M, Eccles M, Fitzpatrick R, Wong G, Sheikh A: Can we systematically review studies that evaluate complex interventions?

    Plos Medicine 2009, 6:8. Publisher Full Text OpenURL

  4. Noyes J, Popay J, Pearson A, Hannes K, Booth A: Qualitative research and Cochrane reviews. In Cochrane handbook for systematic reviews of interventions. Version 5.0.2. Volume Chapter 20. Edited by Higgins JPT, Green S. The Cochrane Collaboration; 2009. OpenURL

  5. Tudor-Hart J: The political economy of health care: a clinical perspective. Bristol, The Policy Press; 2006. OpenURL

  6. Lewin S: Methods to synthesise qualitative evidence alongside a Cochrane intervention review. [http://www.jbiconnect.org/tools/cqrmg/documents/Conference08/Synthesis.pdf] webcite

    Oceania Regional Qualitative Synthesis Symposium: The art and science of method 2008. OpenURL

  7. Pope C, Mays N, Popay J: Synthesising qualitative and quantitative health research: a guide to methods. Open University Press: Maidenhead, UK; 2007. OpenURL

  8. Pope C, Mays N: Critical reflections on the rise of qualitative research.

    BMJ 2009, 339:737-739. OpenURL

  9. Pawson R, Greenhalgh T, Harvet G, Walshe K: Realist Synthesis: an introduction. Manchester: University of Manchester; 2004. OpenURL

  10. Pawson R, Greenhalgh T, Harvet G, Walshe K: Realist review: a new method of systematic review designed for complex policy interventions.

    J Health Services Res Policy 2005, 10(Supp 1):21-34. OpenURL

  11. Sheldon T: Making evidence synthesis more useful for management and policy making.

    J Health Services Res Policy 2005, 10:1-5. OpenURL

  12. Thomas J, Harden A, Oakley A, Oliver S, Sutcliffe K, Rees R, Brunton G, Kavanagh K: Integrating qualitative research with trials in systematic reviews.

    BMJ 2004, 328:1010-1012. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  13. Ragin C: Fuzzy Set Social Science. Chicago: University of Chicago Press; 2000. OpenURL

  14. Melinder KA, Andersson R: The impact of structural factors on the injury rate in different European countries.

    Eur J Public Health 2001, (11):301-308. OpenURL

  15. CASP (Critical Appraisal Skills Programme): 10 Questions to make sense of qualitative research. [http://www.casp-uk.net/index.aspx?o=1150] webcite

    2006.

  16. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM: Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews.

    BMC Med Res Methodol 2007, 7:10. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  17. Mays N, Pope C: Qualitative research in health care. Assessing quality in qualitative research.

    BMJ 2000, 320:50-2. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  18. Giacomini MK, Cook DJ: Users' guides to the medical literature: XXIII. Qualitative research in health A.

    JAMA 2000, 284:357-62. PubMed Abstract | Publisher Full Text OpenURL

  19. Rueda S, Park-Wyllie LY, Bayoumi A, Tynan AM, Antoniou T, Rourke S, Glazier R: Patient support and education for promoting adherence to highly active antiretroviral therapy for HIV/AIDS.

    Cochrane Database of Systematic Reviews 2006., (3) OpenURL

  20. Vervoort SCJM, Borleffs JCC, Hoepelman AIM, Grypdonck MHF: Adherence in antiretroviral therapy: a review of qualitative literature.

    AIDS 2007, 21:271-281. PubMed Abstract | Publisher Full Text OpenURL

  21. World Health Organisation: Adherence to long-term therapy: evidence for action. Geneva: World Health Organisation; 2003. OpenURL

  22. Sackett DL, Snow LC: The magnitude of adherence and nonadherence. In Compliance In Health Care. Edited by Haynes RB, Taylor DW, Sackett DL. John Hopkins University Press; 1979:11-22. OpenURL

  23. Moher D, Schulz KF, Altman DG: The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials.

    Lancet 2001, 357:1191-1194. PubMed Abstract | Publisher Full Text OpenURL

  24. Jones DL, Ishii M, LaPerriere A, Stanley H, Antoni M, Ironson G, Schneiderman N, Van Splunteren F, Cassells A, Alexander K, Gousse YP, Vaughn A, Brondolo E, Tobin JN, Weiss SM: Influencing medication adherence among women with AIDS.

    AIDS CARE 2003, 15:463-474. PubMed Abstract | Publisher Full Text OpenURL

  25. Weber R, Christen L, Christen S, Tschopp S, Hansjoerg Znoj, Schnedier C, Schmitt J, Opravil M, Günthard HF, Ledergerber B: Effect of individual cognitive behaviour intervention on adherence to antiretroviral therapy: prospective randomized trial.

    Antivir Ther 2004, 9:85-95. PubMed Abstract OpenURL

  26. Wyatt GE, Longshore D, Chin D, Vargas Carmona J, Burns Loeb T, Myers HF, Warda U, Liu H, Rivkin I: The efficacy of an integrated risk reduction intervention for HIV-positive women with child sexual abuse histories.

    AIDS Behav 2004, 8:453-462. PubMed Abstract | Publisher Full Text OpenURL

  27. Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P, for the CONSORT group: Methods and process of the CONSORT Group: Example of an extension for trials assessing nonpharmacologic treatments.

    Ann Intern Med 2008, 148:w60-w66. PubMed Abstract | Publisher Full Text OpenURL

  28. Knobel H, Carmona A, Lopez JL, Gimeno JL, Saballs P, Gonzalez A, Guelar A, Diez A: Adherence to highly active antiretroviral treatment: impact of individualized assessment.

    Enfermedades Infecciosas y Microbiologia Clinica 1999, 17:78-81. PubMed Abstract OpenURL

  29. van Servellen G, Carpio F, Lopez M, et al.: Program to enhance health literacy and treatment adherence in low-income HIV-infected Latino men and women.

    AIDS Patient Care STDS 2003, 17:581-594. PubMed Abstract | Publisher Full Text OpenURL

  30. Berrien VM, Salazar JC, Reynolds E, McKay K: Adherence to antiretroviral therapy in HIV-infected pediatric patients improves with home-based intensive nursing intervention.

    AIDS Patient Care STDS 2004, 18:355-363. PubMed Abstract | Publisher Full Text OpenURL

  31. DiIorio C, Resnicow K, McDonnell M, Soet J, McCarty F, Yeager K: Using motivational interviewing to promote adherence to antiretroviral medications: a pilot study.

    J Assoc Nurses AIDS Care 2003, 14:52-62. PubMed Abstract OpenURL

  32. Fairley CK, Levy R, Rayner CR, Allardice K, Costello K, Thomas C, et al.: Randomized trial of an adherence programme for clients with HIV.

    Int J STD AIDS 2003, 14:805-809. PubMed Abstract | Publisher Full Text OpenURL

  33. Goujard C, Bernard N, Sohier N, Peyramond D, Lancon F, Chwalow J, Arnould B, Delfraissy JF: Impact of a patient education program on adherence to HIV medication: a randomized clinical trial.

    J Acquir Immune Defic Syndr 2003, 34:191-194. PubMed Abstract | Publisher Full Text OpenURL

  34. Levy RW, Rayner CR, Fairley CK, Kong DCM, Milch A, Costello K, McArthur C, Melbourne Adherence Group: Multidisciplinary HIV adherence intervention: a randomized study.

    AIDS Patient Care STDS 2004, 18:728-735. PubMed Abstract | Publisher Full Text OpenURL

  35. Murphy DA, Lu Michael C, Martin D, Hoffman D, Marelich WD: Results of a pilot intervention trial to improve antiretroviral adherence among HIV-positive patients.

    J Assoc Nurses AIDS Care 2002, 13:57-69. PubMed Abstract OpenURL

  36. Pradier C, Bentz L, Spire B, Tourette-Turgis C, Morin M, Souville M, Rebillon M, Fuzibet JG, Pesce A, Dellamonica P, Moatti JP: Efficacy of an educational and counseling intervention on adherence to highly active antiretroviral therapy: French prospective controlled study.

    HIV Clin Trials 2003, 4:121-131. PubMed Abstract | Publisher Full Text OpenURL

  37. Rathbun RC, Farmer KC, Stephens JR, Lockhart SM: Impact of an adherence clinic on behavioral outcomes and virologic response in the treatment of HIV infection: a prospective, randomized, controlled pilot study.

    Clin Ther 2005, 27:199-209. PubMed Abstract | Publisher Full Text OpenURL

  38. Rawlings MK, Thompson MA, Farthing CF, Brown LS, Racine J, Scott RC, Crawford KH, Goodwin SD, Tolson JM, Williams VC, Shaefer MS: Impact of an educational program on efficacy and adherence with a twice-daily lamivudine/zidovudine/abacavir regimen in underrepresented HIV-infected patients.

    J Acquir Immune Defic Syndr 2003, 34:174-183. PubMed Abstract | Publisher Full Text OpenURL

  39. Safren SA, Otto MW, Worth JL, Salomon E, Johnson W, Mayer K, Boswell S: Two strategies to increase adherence to HIV antiretro-viral medication: life-steps and medication monitoring.

    Behav Res Ther 2001, 39:1151-62. PubMed Abstract | Publisher Full Text OpenURL

  40. Safren SA, Hendriksen ES, Desousa N, Boswell SL, Mayer KH: Use of an on-line pager system to increase adherence to antiretroviral medications.

    Aids Care 2003, 15:787-793. PubMed Abstract | Publisher Full Text OpenURL

  41. Samet JH, Horton NJ, Meli S, Dukes K, Tripp T, Sullivan L, Freedberg KA: A randomized controlled trial to enhance antiretroviral therapy adherence in patients with a history of alcohol problems.

    Antivir Ther 2005, (10):83-93. OpenURL

  42. Smith SR, Rublein JC, Marcus C, Brock TP, Chesney MA: A medication self-management program to improve adherence to HIV therapy regimens.

    Patient Educ Couns 2003, 50:187-199. PubMed Abstract | Publisher Full Text OpenURL

  43. Tuldra A, Fumaz CR, Jose Ferrer M, Bayes R, Arno A, Balague M, Bonjoch A, Jou A, Negredo E, Paredes R, Ruiz L, Romeu J, Sirera G, Tural C, Burger D, Clotet B: Prospective randomized two-arm controlled study to determine the efficacy of a specific intervention to improve long-term adherence to highly active antiretroviral therapy.

    J Acquir Immune Defic Syndr 2000, (25):221-8. OpenURL

  44. Glaszlou P, Meats E, Heneghan C, Sheppard S: What is missing from descriptions of treatment in trials and reviews?

    BMJ 2008, 336:1472-1474. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  45. Barnett-Page E, Thomas J: Methods for the synthesis of qualitative research: a critical review.

    BMC Med Res Methodol 2009, 9:59. PubMed Abstract | BioMed Central Full Text OpenURL

  46. Thomas J, Harden A: Methods for the thematic synthesis of qualitative research in systematic review. ESRC National Centre for Research Methods, NCRM Working Paper Series. Number (10/07). [http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=188] webcite

  47. Campbell R: Evaluating meta-ethnography: a synthesis of qualitative research on lay experiences of diabetes and diabetes care.

    Social Science & Medicine 2003, 56:671. PubMed Abstract | Publisher Full Text OpenURL

  48. Atkins S, Lewin S, Smith H, Engel M, Fretheim A, Volmink J: Conducting a meta-ethnography of qualitative literature: Lessons learnt.

    BMC Med Res Methodol 2008, 8:21. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  49. Haynes RB, Ackloo E, Sahota N, McDonald HP, Yao X: Interventions for enhancing medication adherence.

    Cochrane Database of Systematic Reviews 2008., (2) OpenURL

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-2288/11/124/prepub