Table 1

Description of the assessment of harmful effects in the three systematic reviews

Review

Schizophrenia

Smoking cessation

Epilepsy


Intervention evaluated

8 atypical antipsychotics.

NRT and bupropion SR.

7 newer antiepileptic drugs.

Objective / Scope

Review commissioned by the HTA programme and an update of the HTA report commissioned by NICE. The objective regarding harmful effects was to determine the incidence of specific rare adverse events to populate an economic model.

Scope provided by NICE: To review all known or unknown harmful effects that might be associated with the interventions.

Scope provided by NICE: To include adverse effects in a review of RCTs of clinical effectiveness in adults with epilepsy. The reviewers undertook a supplementary review of serious, rare and long-term harmful effects. Serious was defined by WHO criteria [25], long-term as longer than 6 months, and rare as defined by the authors of primary studies.

Study designs included

Randomised trials

RCTs of atypical antipsychotics versus alternative drug treatment or placebo in schizophrenia.

An existing Cochrane review was used as a source of summary data on adverse effects from RCTs of effectiveness [26].

Studies that assessed safety as the primary objective were included in the review of primary studies of harmful effects. This included RCTs that investigated aspects of clinical pharmacology that might impact on the drugs' tolerability and safety.

The five most commonly reported adverse effects were extracted from RCTs as part of the review of clinical effectiveness in epilepsy.

RCTs in indications other than epilepsy and dose comparisons were eligible for inclusion in the supplementary review of harmful effects.

Non-randomised studies

Cohort studies and case series with 2000 or more participants or at least 2 years follow-up, and case-control studies of any size or duration.

Uncontrolled trials, prospective and retrospective observational studies, data from adverse events monitoring systems (e.g. UK yellow card scheme) and case reports.

Non-randomised controlled trials, cohort and case-control studies, prospective case series and other uncontrolled trials, and open-label extension phases of trials. More than 300 participants had to be exposed or follow-up more than 6 months unless the study objective was to investigate a specific adverse effect. Prescription event monitoring [27], and post-marketing surveillance reports were also included.

Studies identified

6477 items screened, 924 articles retrieved, and 223 studies included:

171 RCTs, 13 cohort studies, 1 case-control study, 38 case series.

1280 items screened, 353 articles retrieved, and 123 studies included:

25 RCTs, 4 non-randomised controlled trials, 30 uncontrolled trials, before/after studies or cohort studies, 1 case-control study, 9 surveillance studies, 1 survey, 53 case reports or case series.

108 RCTs were included in the review of effectiveness, selected from 4211 items screened and 887 articles retrieved.

In the supplementary review of harmful effects 3884 items were screened, 227 articles retrieved, and 77 studies included: 2 RCTs, 2 non-randomised controlled trials, 26 uncontrolled trials, 14 open-label phases, 25 cohort studies, 1 case-control study, 4 prescription event monitoring studies, 3 post-marketing surveillance studies.

Quality assessment

Quality checklists for various study designs provided in CRD Report 4 were used [28].

The quality checklist for RCTs provided in CRD Report 4 [28], and checklists for the other study designs published elsewhere [29], were used.

Published checklists were used as a starting point. Questions were amended and others added to capture information specifically on the reliability of harmful effects data.

Findings of review

Very few studies with useful data were found, so the economic model could not be populated with incidence rates of the adverse events of interest.

Primarily the findings merely reflected the accepted side-effect profiles for NRT and bupropion SR. The review did not identify any previously unknown harmful effects.

The supplementary review of harmful effects did identify reports of potential adverse effects not reported in the RCTs of clinical effectiveness. However, these were mostly effects already documented in tertiary sources. There was insufficient evidence to attribute causality of other reported effects to the test drugs.


McIntosh et al. BMC Medical Research Methodology 2004 4:19   doi:10.1186/1471-2288-4-19

Open Data