Biometry Research Group, Division of Cancer Prevention, National Cancer Institute, Bethesda, MD 20892-7354, USA

Office of Disease Prevention, National Institutes of Health, Bethesda, MD 20892, USA

Abstract

Background

In recent years there has been increased interest in evaluating breast cancer screening using data from before-and-after studies in multiple geographic regions. One approach, not previously mentioned, is the paired availability design. The paired availability design was developed to evaluate the effect of medical interventions by comparing changes in outcomes before and after a change in the availability of an intervention in various locations. A simple potential outcomes model yields estimates of efficacy, the effect of receiving the intervention, as opposed to effectiveness, the effect of changing the availability of the intervention. By combining estimates of efficacy rather than effectiveness, the paired availability design avoids confounding due to different fractions of subjects receiving the interventions at different locations. The original formulation involved short-term outcomes; the challenge here is accommodating long-term outcomes.

Methods

The outcome is incident breast cancer deaths in a time period, which are breast cancer deaths that were diagnosed in the same time period. We considered the plausibility of the basic five assumptions of the paired availability design and propose a novel analysis to accommodate likely violations of the assumption of stable screening effects.

Results

We applied the paired availability design to data on breast cancer screening from six counties in Sweden. The estimated yearly change in incident breast cancer deaths per 100,000 persons ages 40–69 (in most counties) due to receipt of screening (among the relevant type of subject in the potential outcomes model) was -9 with 95% confidence interval (-14, -4) or (-14, -5), depending on the sensitivity analysis.

Conclusion

In a realistic application, the extended paired availability design yielded reasonably precise confidence intervals for the effect of receiving screening on the rate of incident breast cancer death. Although the assumption of stable preferences may be questionable, its impact will be small if there is little screening in the first time period. However, estimates may be substantially confounded by improvements in systemic therapy over time. Therefore the results should be interpreted with care.

Background

The paired availability design is a study design and method of analysis that reduces selection bias when using data from historical controls

Estimating efficacy, as opposed to estimating effectiveness, is important when combining estimates from different locations (hospitals or regions). If the fraction of subjects who receive intervention differs among locations, it is difficult to interpret the overall estimate of effectiveness. In contrast, the overall estimate of efficacy is not confounded by varying the fraction of subjects who receive intervention in different locations.

Heretofore the paired availability design has only been formulated for evaluating the effect of an intervention on a

Methods

Basic requirements

The first step in extending the paired availability design to the evaluation of breast cancer screening is to identify various geographic regions with a change in the availability of breast cancer screening from time period 0 to time period 1. To simplify this discussion, we presume that screening is more available in time period 1 than time period 0. The methodology is also applicable in the unlikely situation in which the reverse were true in some or all regions. The change in availability is a change in the fraction of the eligible population who are invited for screening. Following Duffy et al

Requirement 1

The time periods should be sufficiently long to give screening sufficient time to maximize (or almost maximize) its impact on breast cancer mortality rates.

Requirement 2

For each geographic region, time periods 0 and 1 should be the same length.

Requirement 3

The outcome in each time period is incident breast cancer deaths, namely deaths from breast cancer in the specified time period arising from diagnosis of breast cancer during the same time period.

Requirement 4

We consider only situations in which most screening occurs at regular intervals of the same length during each time period.

The rationale for

The rationale for

The rationale for

Potential outcomes model

For each before-and-after geographic region, our goal is to estimate the efficacy of breast cancer screening, which we define as the change in average yearly probability of incident breast cancer deaths due to the

For the sake of simplicity, we assume two conditions: (1) all-or-none behavior (i.e. an individual either receives all screens at the recommended interval or none, but does not switch back and forth), and (2) there is a single dominant screening test rather than a choice among screening tests of varying efficacy. In our application, there was only one screening modality.

Let π_{iAz}, π_{iCz}, π_{iIz}, and π_{iNz }denote the probabilities of subject types _{iAz}, _{iCz}, _{iIz }and _{iNz }denote the probability of incident breast cancer death in time period

_{i0 }= π_{iN0 }_{iN0 }+ π_{iC0 }_{iC0 }+ π_{iI0 }_{iI0 }+ π_{iA0 }_{iA0}, for time period 0,

_{i1 }= π_{iN1 }_{iN1 }+ π_{iC1 }_{iC1 }+ π_{iI1 }_{iI1 }+ π_{iA1 }_{iA1}, for time period 1. (1)

As with the standard paired availability design, to ensure identifiably we restrict the estimation of efficacy to type _{i }denote the length of follow-up for time periods 0 and 1 for region

The probability in (2) differs from a naive comparison of the effect of screening between subjects who receive screening in time period 1 and subjects who do not receive screening in time period 1. Instead Δ_{i }is the effect of receiving screening among type

Assumptions

In order to estimate (2) we require the following assumptions adapted from the standard paired availability design

Assumption 1. (Stable population)

The characteristics of the population that affect the probability of incident breast cancer death are constant over time.

Assumption 2. (Stable treatment)

The screening modality and therapy following diagnosis do not change over time.

Assumption 3. (Stable evaluation)

The outcome measure, which is incident cancer breast deaths, does not change in definition over time.

Assumption 4. (Stable preferences)

Factors affecting the decision to receive screening do not change over time.

Assumption 5. (Stable screening effects)

The effect of screening on the probability of incident breast cancer death rates does not change over time.

The basic idea _{iAz }= π_{iA}, π_{iCz }= π_{iC}, π_{iIz }= π_{iI}, and π_{iNz }= π_{iN}. In addition, by virtue _{iI }= 0. If there is no screening in time period 0, so π_{iaz }= π_{iCz }= 0, and _{iCz }= π_{iC }and π_{iNz }= π_{iN}, which is very plausible especially if one views public awareness as part of the screening intervention.

_{iN0 }= _{iN1 }≡ _{N}. However, unless there is no screening in time period 0,

As a consequence of the above assumptions (and not applying

_{i0 }= π_{iN }_{iN }+ π_{iC }_{iC0 }+ π_{iA }_{iA0}

_{i1 }= π_{iN }_{iN }+ π_{iC }_{iC1 }+ π_{iA }_{iA1}. (3)

Because _{i1 }- _{i0 }= π_{iC }(_{iC1 }- _{iC0}) + π_{iA}(_{iA1 }- _{iA0}), we obtain from (3)

If _{iA1 }= _{iA0 }, and we would obtain the standard formula, averaged over the duration of the time period, for efficacy in the paired availability design, Δ_{i }= (_{i1 }- _{i0})/(π_{iC}_{i}). We would also obtain the standard formula if there were no screening in time period 0 (and thus no type

Estimates

In order to estimate (4) we need to estimate _{iz}, π_{iC}, π_{iA}, and _{iAz}. Following the standard paired availability design we can estimate the first three parameters as follows. Let

In the ideal scenario (Scenario I) the investigators would report data _{izsy}, which is the number of subjects in region _{izs+ }and the numbers with a given outcome _{iz+y}, where "+" denotes summation over the indicated subscript. For both scenarios, we obtain the following estimates,

_{iz }= _{iz+1}/_{iz++ }= fraction of subjects in time period

_{iA }= _{i01+}/_{i0++ }= fraction who received screening in time period 0, (7)

_{iC }= _{i11+}/_{i1++ }- _{i01+}/_{i0++}

= fraction who received screening in time period 1 (a combination of types

If we had the full data _{izsy}, we could estimate _{iA0}. However because subjects in time period 1 who receive screening are a combination of types _{iA1}. We discuss how to circumvent this difficulty in the two scenarios.

Scenario I: Full reporting of data

When there are full reporting of data, we can estimate _{izs }= _{izs }= _{izs1}/_{izs+}. Under the potential outcomes model, we write

_{i00 }= π_{iN }_{iN }+ π_{ic }_{iC0}, _{i01 }= π_{iA }_{iA0},

_{i10 }= π_{iN }_{iN}, _{i11 }= π_{iC }_{iC1 }+ π_{iA }_{iA1}. (9)

We introduce an exogenous parameter

_{iA1 }= _{iA0}. (10)

We discuss specification of

The asymptotic variance is approximately

Scenario II. Limited reporting of data

With limited reporting of data we introduce a second exogenous parameter

_{i01 }= _{i0}, (13)

where

where

Using actual reported data from the limited data scenario, we checked the approximate variances in (12) and (15) by making reasonable assumptions to impute _{izsy }and then also computed the asymptotic variance using the delta method. The agreement was excellent: using the data in the example and assuming relative risk of incident cancer death of .7 for screened versus not screened, the approximate and exact asymptotic variance agreed to three significant digits.

Lead time adjustment related to prior screening

We specify a value for _{i}, the length of time of screen-detection in subgroup (_{i }+ _{i}/(_{i }+ _{i }there will be little bias as

Lead time adjustment related to age-range at diagnosis

Because incident cancer cases are defined based on age at diagnosis, there is also a subtle bias _{i1 }by _{iC }+ (1 - π_{iC}), where

and _{a }is five-year cumulative mortality following breast cancer diagnosis at age _{40 }= .167, _{50 }= .131, and _{60 }= .124, based on US population data

Combining estimates over regions

To obtain a combined estimate of efficacy over all regions, we use a simple random effects meta-analysis _{i }= 1/ ^{2 }= the larger of (_{i }_{i }- Σ_{i }_{i }_{i}) and 0, where _{i }_{i }(^{2}, _{i }_{i }/ Σ_{i}_{i}. The random-effects weights are _{r-1 }_{k-1}_{r-1 }is the value of the 97 1/2 percentile of a

Data and calculations

time period 0

time period 1

yearly change in incident breast cancer deaths per 100,000

county

number eligible

incident breast cancer deaths

fraction screened

number eligible

incident breast cancer deaths

fraction screened

estimate

standard error

Varmland

39946

36

.03

40853

37

.73

.16

Sodermanland

49526

98

.007

53760

79

.77

-7.89

4.28

Uppsala

43340

110

.03

52426

112

.82

-5.73

4.43

Vastamanland

49827

112

.14

52329

85

.92

-8.88

3.93

Orebro

52936

133

.04

54366

108

.64

-9.74

5.36

Gavleborg

56794

311

.00

56145

219

.84

-13.40

3.64

Fraction screened is from Table ^{2 }= 0, and overall estimate of -9 with 95% confidence (-14, -4) per 100,000.

Results

We applied the methodology to before- and after-data on breast cancer screening in various Swedish counties

The data in _{iz++}. Using these data, we estimated the change in the average yearly death rate of incident breast cancer among type

Estimated change (and 95% confidence intervals) in average yearly probability of incident cancer death due to receipt of screening per 100,000 type

Estimated change (and 95% confidence intervals) in average yearly probability of incident cancer death due to receipt of screening per 100,000 type

Discussion

Our methodology complements that of Duffy et al.

Another approach to the analysis of before- and after- cancer screening data is to regress the change in cancer mortality rate over time for each region on the change in screening rates over time in each region. Sometimes the change in cancer incidence rates is used as a proxy for the change in screening rates

With additional data, it may be possible to adjust for the effect of changes in therapy over time, if an additional assumption is reasonable. Suppose we had additional data on incident cancer deaths in time periods 0 and 1 in regions in which there was

Although data were reported on person-years of eligibility for screening, we did not use a survival analysis. A survival analysis can be incorporated into the potential outcomes model for all-or-none compliance

Besides this methodology based on the paired availability design, one could also analyze observational screening data using the method of periodic screening evaluation

Conclusion

The paired availability design can be extended to the evaluation of breast cancer screening by using incident breast cancer deaths as the outcome and requiring sufficiently long equal-length time periods before and after a change in availability of periodic screening. However the assumptions should be examined carefully. The assumption of stable preferences may be violated by a campaign to encourage screening, although the impact would be greatly mitigated if there were little screening in time period 0. Also the assumptions regarding changes in therapy over time may also be violated.

Authors' contributions

SGB wrote the initial draft. BSK and PCP made substantial improvements to the manuscript.

Competing interests

None declared.

Acknowledgment

We thank Ping Hu for helpful comments.

Pre-publication history

The pre-publication history for this paper can be accessed here: