BMC Medical Research Methodology - Latest articles

Advantages of the nested case-control design in diagnostic research

2008-07-21

Background: Despite the benefits, it is uncommon to apply the nested case-control design in diagnostic research. We aim to show advantages of this design for diagnostic accuracy studies. Methods: We used data from a full cross-sectional diagnostic study comprising a cohort of 1295 consecutive patients who were selected on their suspicion of having deep vein thrombosis (DVT). We draw nested case-control samples from the full study population with case:control ratios of 1:1, 1:2, 1:3 and 1:4 (per ratio 100 samples were taken). We calculated diagnostic accuracy estimates for two tests that are used to detect DVT in clinical practice. Results: Estimates of diagnostic accuracy in the nested case-control samples were very similar to those in the full study population. For example, for each case:control ratio, the positive predictive value of the D-dimer test was 0.30 in the full study population and 0.30 in the nested case-control samples (median of the 100 samples). As expected, variability of the estimates decreased with increasing sample size. Conclusions: Our findings support the view that the nested case-control study is a valid and efficient design for diagnostic studies and should also be (re)appraised in current guidelines on diagnostic accuracy research.

Reliability and validity of two frequently used self-administered physical activity questionnaires in adolescents

2008-07-15

Background: To create and find accurate and reliable instruments for the measurement of physical activity has been a challenge in epidemiological studies. We investigated the reliability and validity of two different physical activity questionnaires in 71 adolescents aged 13-18 years; the WHO, Health Behaviour in Schoolchildren (HBSC) questionnaire, and the International Physical Activity Questionnaire (IPAQ, short version). Methods: The questionnaires were administered twice (8-12 days apart) to measure reliability. Validity was assessed by comparing answers from the questionnaires with a cardiorespiratory fitness test (VO2peak) and seven days activity monitoring with the ActiReg, an instrument measuring physical activity level (PAL) and total energy expenditure (TEE). Results: Intraclass correlation coefficients for reliability for the WHO HBSC questionnaire were 0.71 for frequency and 0.73 for duration. For the frequency question, there was a significant difference between genders; 0.87 for girls and 0.59 for boys (p<0.05). The intraclass correlation coefficients the IPAQ varied between 0.10 and 0.62 for the reliability. Spearman correlation coefficients for validity for both the WHO HBSC questionnaire and the IPAQ (recoded into low, moderate and high activity) measured against VO2peak were fair, ranging between 0.29 - 0.39. The WHO HBSC questionnaire measured against VO2peak for girls were acceptable, ranging between 0.30 - 0.55. Both questionnaires, except the walking question in IPAQ, showed a low correlation with PAL and TEE, ranging between 0.01 and 0.29. Conclusion: These data indicate that the WHO HBSC questionnaire had substantial reliability and were acceptable instrument for measuring cardiorespiratory fitness, especially among girls. None of the questionnaires however seemed to be a valid instrument for measuring physical activity compared to TEE and PAL in adolescents.

A coarsened multinomial regression model for perinatal mother to child transmission of HIV

Charlotte C Gard and Elizabeth R Brown — 2008-07-15

Background: In trials designed to estimate rates of perinatal mother to child transmission of HIV, HIV assays are scheduled at multiple points in time. Still, infection status for some infants at some time points may be unknown, particularly when interim analyses are conducted. Methods: Logistic regression models are commonly used to estimate covariate-adjusted transmission rates, but their methods for handling missing data may be inadequate. Here we propose using coarsened multinomial regression models to estimate cumulative and conditional rates of HIV transmission. Through simulation, we compare the proposed models to standard logistic models in terms of bias, mean squared error, coverage probability, and power. We consider a range of treatment effect and visit process scenarios, while including imperfect sensitivity of the assay and contamination of the endpoint due to early breastfeeding transmission. We illustrate the approach through analysis of data from a clinical trial designed to prevent perinatal transmission. Results: The proposed cumulative and conditional models performed well when compared to their logistic counterparts. Performance of the proposed cumulative model was particularly strong under scenarios where treatment was assumed to increase the risk of in utero transmission but decrease the risk of intrapartum and overall perinatal transmission and under scenarios designed to represent interim analyses. Power to estimate intrapartum and perinatal transmission was consistently higher for the proposed models. Conclusions: Coarsened multinomial regression models are preferred to standard logistic models for estimation of perinatal mother to child transmission of HIV, particularly when assays are missing or occur off-schedule for some infants.

Methods for the thematic synthesis of qualitative research in systematic reviews

James Thomas and Angela Harden — 2008-07-10

Background: There is a growing recognition of the value of synthesising qualitative research in the evidence base in order to facilitate effective and appropriate health care. In response to this, methods for undertaking these syntheses are currently being developed. Thematic analysis is a method that is often used to analyse data in primary qualitative research. This paper reports on the use of this type of analysis in systematic reviews to bring together and integrate the findings of multiple qualitative studies. Methods: We describe thematic synthesis, outline several steps for its conduct and illustrate the process and outcome of this approach using a completed review of health promotion research. Thematic synthesis has three stages: the coding of text 'line-by-line'; the development of 'descriptive themes'; and the generation of 'analytical themes'. While the development of descriptive themes remains 'close' to the primary studies, the analytical themes represent a stage of interpretation whereby the reviewers 'go beyond' the primary studies and generate new interpretive constructs, explanations or hypotheses. The use of computer software can facilitate this method of synthesis; detailed guidance is given on how this can be achieved. Results: We used thematic synthesis to combine the studies of children's views and identified key themes to explore in the intervention studies. Most interventions were based in school and often combined learning about health benefits with 'hands-on' experience. The studies of children's views suggested that fruit and vegetables should be treated in different ways, and that messages should not focus on health warnings. Interventions that were in line with these suggestions tended to be more effective. Thematic synthesis enabled us to stay 'close' to the results of the primary studies, synthesising them in a transparent way, and facilitating the explicit production of new concepts and hypotheses. Conclusion: We compare thematic synthesis to other methods for the synthesis of qualitative research, discussing issues of context and rigour. Thematic synthesis is presented as a tried and tested method that preserves an explicit and transparent link between conclusions and the text of primary studies; as such it preserves principles that have traditionally been important to systematic reviewing.

Language and rigour in qualitative research: Problems and principles in analyzing data collected in Mandarin

Helen J Smith, Jing Chen and Xiaoyun Liu — 2008-07-10

In collaborative qualitative research in Asia, data are usually collected in the national language, and this poses challenges for analysis. Translation of transcripts to a language common to the whole research team is time consuming and expensive; meaning can easily be lost in translation; and validity of the data may be compromised in this process. We draw on several published examples from public health research conducted in mainland China, to highlight how language can influence rigour in the qualitative research process; for each problem we suggest potential solutions based on the methods used in one of our research projects in China.Problems we have encountered include obtaining sufficient depth and detail in qualitative data; deciding on language for data collection; managing data collected in Mandarin; and the influence of language on interpreting meaning.We have suggested methods for overcoming problems associated with collecting, analysing, and interpreting qualitative data in a local language, that we think help maintain analytical openness in collaborative qualitative research. We developed these methods specifically in research conducted in Mandarin in mainland China; but they need further testing in other countries with data collected in other languages. Examples from other researchers are needed.

Relative performance of different exposure modeling approaches for sulfur dioxide concentrations in the air in rural western Canada

Igor Burstyn, Nicola M Cherry, Yutaka Yasui and Hyang-Mi Kim — 2008-07-04

Background: The main objective of this paper is to compare different methods for predicting the levels of SO2 air pollution in oil and gas producing area of rural western Canada. Month-long average air quality measurements were collected over a two-year period (2001–2002) at multiple locations, with some side-by-side measurements, and repeated time-series at selected locations. Methods: We explored how accurately location-specific mean concentrations of SO2 can be predicted for 2002 at 666 locations with multiple measurements. Means of repeated measurements on the 666 locations in 2002 were used as the alloyed gold standard (AGS). First, we considered two approaches: one that uses one measurement from each location of interest; and the other that uses context data on proximity of monitoring sites to putative sources of emission in 2002. Second, we imagined that all of the previous year's (2001's) data were also available to exposure assessors: 9,464 measurements and their context (month, proximity to sources). Exposure prediction approaches we explored with the 2001 data included regression modeling using either mixed or fixed effects models. Third, we used Bayesian methods to combine single measurements from locations in 2002 (not used to calculate AGS) with different priors. Results: The regression method that included both fixed and random effects for prediction (Best Linear Unbiased Predictor) had the best agreement with the AGS (Pearson correlation 0.77) and the smallest mean squared error (MSE: 0.03). The second best method in terms of correlation with AGS (0.74) and MSE (0.09) was the Bayesian method that uses normal mixture prior derived from predictions of the 2001 mixed effects applied in the 2002 context. Conclusion: It is likely that either collecting some measurements from the desired locations and time periods or predictions of a reasonable empirical mixed effects model perhaps is sufficient in most epidemiological applications. The method to be used in any specific investigation will depend on how much uncertainty can be tolerated in exposure assessment and how closely available data matches circumstances for which estimates/predictions are required.

Does it matter whether the recipient of patient questionnaires in general practice is the general practitioner or an independent researcher? The REPLY randomised trial

2008-06-27

Background: Self-administered questionnaires are becoming increasingly common in general practice. Much research has explored methods to increase response rates but comparatively few studies have explored the effect of questionnaire administration on reported answers. Methods: The aim of this study was to determine the effect on responses of returning patient questionnaires to the respondents' medical practice or an independent researcher to questions relating to adherence and satisfaction with a GP consultation. One medical practice in Waveney primary care trust, Suffolk, England participated in this randomised trial. Patients over 18 years initiated on a new long-term medication during a consultation with a GP were randomly allocated to return a survey from their medical practice to either their medical practice or an independent researcher. The main outcome measures were self reported adherence, satisfaction with information about the newly prescribed medicine, the consultation and involvement in discussions. Results: 274 (47%) patients responded to the questionnaire (45% medical practice, 48% independent researcher (95% CI -5 to 11%, p = 0.46)) and the groups appeared demographically comparable, although the high level of non-response limits the ability to assess this. There were no significant differences between the groups with respect to total adherence or any of the satisfaction scales. Five (4%) patients reported altering doses of medication in the medical practice group compared with 18 (13%) in the researcher group (P = 0.009, Fisher's exact test). More patients in the medical practice group reported difficulties using their medication compared to the researcher group (46 (35%) v 30 (21%); p = 0.015, Fisher's exact test). Conclusion: Postal satisfaction questionnaires do not appear to be affected by whether they are returned to the patient's own medical practice or an independent researcher. However, returning postal questionnaires relating to detailed patient behaviours may be subject to response biases and further work is needed to explore this phenomena.

Disagreement in primary study selection between systematic reviews on negative pressure wound therapy

Frank Peinemann, Natalie McGauran, Stefan Sauerland and Stefan Lange — 2008-06-26

Background: Primary study selection between systematic reviews is inconsistent, and reviews on the same topic may reach different conclusions. Our main objective was to compare systematic reviews on negative pressure wound therapy (NPWT) regarding their agreement in primary study selection. Methods: This retrospective analysis was conducted within the framework of a systematic review (a full review and a subsequent rapid report) on NPWT prepared by the Institute for Quality and Ef-ficiency in Health Care (IQWiG). For the IQWiG review and rapid report, 4 bibliographic databases (MEDLINE, EMBASE, The Cochrane Library, and CINAHL) were searched to identify systematic reviews and primary studies on NPWT versus conventional wound therapy in patients with acute or chronic wounds. All databases were searched from inception to December 2006. For the present analysis, reviews on NPWT were classified as eligible systematic reviews if multiple sources were systematically searched and the search strategy was documented. To ensure comparability between reviews, only reviews published in or after December 2004 and only studies published before June 2004 were considered. Eligible reviews were compared in respect of the methodology applied and the selection of primary studies. Results: A total of 5 systematic reviews (including the IQWiG review) and 16 primary studies were analysed. The reviews included between 4 and 13 primary studies published before June 2004. Two reviews considered only randomised controlled trials (RCTs). Three reviews considered both RCTs and non-RCTs. The overall agreement in study selection between reviews was 96% for RCTs (24 of 25 options) and 57% for non-RCTs (12 of 21 options). Due to considerable disagreement in the citation and selection of non-RCTs, we contacted the review authors for clarification (this was not initially planned); all authors or institutions responded. According to published information and the additional information provided, most differences between reviews arose from variations in inclusion criteria or inter-author study classification, as well as from different reporting styles (citation or non-citation) for excluded studies. Conclusions: The citation and selection of primary studies differ between systematic reviews on NPWT, particularly with regard to non-RCTs. Uniform methodological and reporting standards need to be applied to ensure comparability between reviews as well as the validity of their conclusions.

Integrated analysis of incidence, progression, regression and disappearance probabilities

Guan-Hua Huang — 2008-06-25

Background: Age-related maculopathy (ARM) is a leading cause of vision loss in people aged 65 or older. ARM is distinctive in that it is a disease which can transition through incidence, progression, regression and disappearance. The purpose of this study is to develop methodologies for studying the relationship of risk factors with different transition probabilities. Methods: Our framework for studying this relationship includes two different analytical approaches. In the first approach, one can define, model and estimate the relationship between each transition probability and risk factors separately. This approach is similar to constraining a population to a certain disease status at the baseline, and then analyzing the probability of the constrained population to develop a different status. While this approach is intuitive, one risks losing available information while at the same time running into the problem of insufficient sample size. The second approach specifies a transition model for analyzing such a disease. This model provides the conditional probability of a current disease status based upon a previous status, and can therefore jointly analyze all transition probabilities. Throughout the paper, an analysis to determine the birth cohort effect on ARM is used as an illustration.Results and conclusionThis study has found parallel separate and joint analyses to be more enlightening than any analysis in isolation. By implementing both approaches, one can obtain more reliable and more efficient results.

Workplace restructurings in intervention studies – a challenge for design, analysis and interpretation

2008-06-13

Background: Interventions in occupational health often target worksites rather than individuals. The objective of this paper is to describe the (lack of) stability in units of analysis in occupational health and safety intervention projects directed toward worksites. Methods: A case study approach is used to describe naturally occurring organizational changes in four, large, Nordic intervention projects that ran 3–5 years, covered 3–52 worksites, cost 0.25 mill–2.2 mill €, and involved 3–7 researchers. Results: In all four cases, high rates of closing, merging, moving, downsizing or restructuring was observed, and in all four cases at least one company/worksite experienced two or more re-organizations during the project period. If individual worksites remained, ownership or (for publicly owned) administrative or legal base often shifted. Forthcoming closure led employees and managers to seek employment at other worksites participating in the studies. Key employees involved in the intervention process often changed. Conclusion: Major changes were the rule rather than the exception. Frequent fundamental changes at worksites need to be taken into account when planning intervention studies and raises serious questions concerning design, analyses and interpretation of results. The frequent changes may also have deleterious implications for the potential effectiveness of many real life interventions directed toward worksites. We urge researchers and editors to prioritize this subject in order to improve the quality of future intervention research and preventive action.