Biometry Research Group, Division of Cancer Prevention, National Cancer Institute, Bethesda, Maryland, U.S.A

Information Management Services, Inc., Rockville, Maryland, USA

Office of Disease Prevention, National Institutes of Health, Bethesda, Maryland, USA

Abstract

Background

When evaluating cancer screening it is important to estimate the cumulative risk of false positives from periodic screening. Because the data typically come from studies in which the number of screenings varies by subject, estimation must take into account dropouts. A previous approach to estimate the probability of at least one false positive in

Method

By redefining the random variables, we obviate the unrealistic dropout assumption. We also propose a relatively simple logistic regression and extend estimation to the expected number of false positives in

Results

We illustrate our methodology using data from women ages 40 to 64 who received up to four annual breast cancer screenings in the Health Insurance Program of Greater New York study, which began in 1963. Covariates were age, time since previous screening, screening number, and whether or not a previous false positive occurred. Defining a false positive as an unnecessary biopsy, the only statistically significant covariate was whether or not a previous false positive occurred. Because the effect of screening number was not statistically significant, extrapolation beyond 4 screenings was reasonable. The estimated mean number of unnecessary biopsies in 10 years per woman screened is .11 with 95% confidence interval of (.10, .12). Defining a false positive as an unnecessary work-up, all the covariates were statistically significant and the estimated mean number of unnecessary work-ups in 4 years per woman screened is .34 with 95% confidence interval (.32, .36).

Conclusion

Using data from multiple cancer screenings with dropouts, and allowing dropout to depend on previous history of false positives, we propose a logistic regression model to estimate both the probability of at least one false positive and the expected number of false positives associated with

Background

When evaluating cancer screening, it is important to estimate both the benefits and harms. The major benefit is the reduction in mortality from the cancer that is the object of the screening

Gelfand and Wang (GW)

Methods

Obviating the Unrealistic Dropout Assumption

The proof obviating the assumption that dropout does not depend on previous FP's is technical and deferred to the Appendix (see

Logistic Regression

To estimate parameters, GW used a Bayesian approach with a proportional hazards model. For the clinically oriented reader, the computations can be difficult. As an alternative we propose a relatively simple logistic regression models that is appropriate for some data sets (with further elaboration in the Discussion). Our approach requires fitting two logistic regressions. The first logistic regression models the probability of FP on the first screening as a function of age. We let

_{0 }+ α_{age(i)}, (1)

where α_{age(1) }= 0 because we have constant term α_{0}. The data are a table of counts for age categories cross-classified by FP outcome (yes or no). See supplemental file.

The second logistic regression models the probability of FP on a screening after the first as a function of age at screening, time since the last screening, the number of the screening, and whether or not there was a previous FP. To obtain a parsimonious model we have made two simplifications. First we use screening number rather than chronological time. For example, in one subject screening might occur at times 0, 1, and 3, and in another subject, screening might occur at times 0, 1, 2. In terms of the model, both subjects have three screenings indexed by

As before we let

_{0 }+ β_{age(i) }+ β_{time(j) }+ β_{screen(t) }+ β_{FP(k)}. (2)

where β_{age(1) }= β_{time(1) }= β_{screen(1) }= β_{FP(1) }= 0. The data are counts for a cross classification of age interval, time interval, screening number, an indicator of previous FP, and the FP outcome. See

Click here for file

Click here for file

Estimating Cumulative Risk

We use the parameter estimates from the logistic regression to estimate the cumulative risk of an FP. Let

where

The estimated survival time until the first FP in _{ij}(

To better quantify the cumulative burden of FP's we also estimate the expected number of FP's. The formula for the estimated expected number of FP's in

_{ij}(4) =

where

_{1i }_{3|3ij}

_{1i }_{2|3ij }+ (1 - _{1i}) _{2i }_{2|2ij}

_{1i }_{1|3ij }+ (1 - _{1i}) _{2i }_{1|2ij }+ (1 - _{1i}) (1 - _{2ij}) _{3ij }_{1|1ij}

_{1i }_{0|3ij }+ (1 - _{1i}) _{2i }_{0|2ij }+ (1 - _{1i}) (1 - _{2ij}) _{3ij }_{0|1ij }+ (1 - _{1i})(1 - _{2ij}) (1 - _{3ij}) _{4ij},

and _{h|fij }is the probability of

_{3|3ij }= _{2ij}_{3ij}_{4ij},

_{2|3ij }= _{2ij}_{3ij}(1 - _{4ij}) + _{2ij}(1 - _{3ij}) _{4ij }+ (1 - _{2ij})_{3ij}_{4ij},

_{1|3ij }= _{2ij }(1 - _{3ij}) (1 - _{4ij}) + (1 - _{2ij}) _{3ij }(1 - _{4ij}) + (1 - _{2ij}) (1 - _{3ij}) _{4ij},

_{0|3ij }= _{2ij}) (1 - _{3ij}) (1 - _{4ij}),

_{2|2ij }= _{3ij}_{4ij},

_{1|2ij }= _{3ij }(1 - _{4ij}) + (1 - _{3ij}) _{4ij},

_{0|2ij }= _{3ij}) (1 - _{4ij}),

_{1|1ij }= _{4ij},

_{0|1ij }= _{4ij}.

An important special case occurs when the probabilities of FP do not vary with screening number. This case is important because it allows extrapolation to additional screenings. With _{2ij }= _{tij0 }and _{ij }= _{tij1 }for

_{ij}(_{1i}) (1 - _{2ij})^{n}, (8)

and the estimated expected number of FP's in

The most difficult part of implementation is computing the variance. The asymptotic variances are

where θ = (α_{0}, α_{age(i)}, β_{0}, β_{age(i)}, β_{time(j)}, β_{screen(t)}, β_{FP(k)}). By using computer software for symbolic derivatives

Results

We applied the methodology to data on 4 annual screenings in the Health Insurance Program of Greater New York (HIP) breast cancer screening study

Parameter estimates from the logistic regressions are presented in Table

Parameter estimates (standard errors) from logistic regression models for FP's

Narrow FP

Broad FP

initial model

final model

final model

model for initial FP

α_{0}

-4.30 (.13)

-4.14 (.06)

-2.37 (.05)

α_{age}(2)

.23 (.17)

.15 (.07)

α_{age}(3)

.29 (.17)

-.01 (.08)

α_{age}(4)

.13 (.19)

.05 (.08)

α_{age}(5)

.00 (.22)

-.02 (.09)

model for subsequent FP's

β_{0}

-4.81 (.18)

-4.89 (.06)

-2.85 (.07)

β_{age}(2)

.24 (.19)

-.02 (.07)

β_{age}(3)

.00 (.19)

-.16 (.08)

β_{age}(4)

-.32 (.21)

-.28 (.08)

β_{age}(5)

.07 (.21)

-.24 (.08)

β_{time}(2)

-.02 (.11)

-.13 (.05)

β_{time}(3)

.10 (.20)

-.22 (.09)

β_{time}(4)

.12 (.22)

-.08 (.09)

β_{screen}(3)

-.30 (.13)

-.11 (.05)

β_{screen}(4)

-.07 (.13)

-.22 (.06)

β_{FP(1)}

2.35 (.14)

2.34 (.14)

1.64 (.05)

Question is answered by the estimates in (6) and (8). Question 2 is answered by estimates in (7) and (9). For an economic analysis Question 2 is particularly useful as it would help an analyst assign monetary costs to the cumulative burden of FP's. Both questions are clearly important to the patient.

Defining a FP as an unnecessary biopsy, we could not reject a model for (1) with only a constant (deviance = .27 on 4 d.f, p = .99) nor a model for (2) with a constant and a parameter for previous FP's (deviance = 3.66 on 9 d.f., p = .93). Consequently we think it is reasonable to extrapolate beyond 4 screenings using (8) and (9). In answer to

Defining an FP as an unnecessary work-up, most of the parameter estimates were statistically significant, so we kept all the covariates in the model. Because the parameter for screening number was included, we could not extrapolate beyond the 4 screenings that was the maximum number of screenings per subject in our data. For purposes of illustration we selected

As an ancillary investigation, we also fit a logistic regression for the probability of dropout as a function of age category, time interval since last screening, screening number, false positive on the last screening, false positive on an earlier screening, and interaction between of the two false positive variables. For the HIP data, when FP was defined broadly, there was no statistically significant association between FP history and dropout. For FP defined narrowly, there was a strong association between FP history and dropout.

Discussion

Our methodology is applicable to any screening test recommended on a periodic basis for which data come from subjects with possibly different numbers of screenings. Ideally one would like data from a study in which subjects are representative of the general eligible population and clinicians are representative of the clinicians who would perform the screening in practice. Particularly when the FP is an unnecessary work-up, the clinicians may vary in the threshold used to determine a positive, as there is subjectivity to the interpretation of the test. When FP is an unnecessary biopsy, the variation among clinicians will likely be small because a high level of FP's is unacceptable [14]. In our data set there was no information on clinician. If there are data on clinicians, it should be incorporated into the analysis. If the number of clinicians is small, we suggest including a variable for clinician in the logistic regression. If the number of clinicians is larger, it is best to include a random variable for the effect of clinician. Unfortunately the simple logistic regression is not applicable and a more complicated model such as that in GW would be needed.

The assumption that dropout does not depend on future false positives could be violated if a subject drops out because of self examination results (so she goes to her regular physician) that would have led to future false positives. To avoid this violation of the assumption, one could ask women screened and women who dropped out if they found any lump on self examination. By including a covariate for lump on self examination, the dropout process depends on previous history and factors from the likelihood.

Conclusion

We made three contributions. First we showed that previous methodology of GW did not require an unrealistic assumption about the dropout process. This makes the approach much more appealing. Second, we showed how to estimate the expected number of false positives, which we think is informative, in addition to the probability of at least one false positive. Third we presented a logistic regression formulation that is applicable for some data sets and is relatively simple to implement. Our approach can be applied to many types of cancer screening tests that are recommended on a periodic basis. It is useful for both advising individuals in a clinical setting and for health resources planning.

Authors' Contribution

SGB wrote the initial draft and computed the results. DE created the appropriate tables of data from HIP and ran initial logistic regressions. BSK made substantial improvements to the manuscript.

Competing Interests

None declared

Acknowledgment

We thank Jian-Lun Xu, Victor Kipnis, and Philip Prorok for helpful comments.

Pre-publication history

The pre-publication history for this paper can be accessed here: