Reducing decision errors in the paired comparison of the diagnostic accuracy of screening tests with Gaussian outcomes
1 Center for Cancer Prevention and Control Research, University of California, Los Angeles, 650 Charles Young Drive South, Room A2-125 CHS, Los Angeles CA 90095, USA
2 Department of Preventive Medicine, University of Southern California, 440 E. Huntington Dr, 4th floor, Arcadia CA 91006, USA
3 Department of Biostatistics and Informatics, Colorado School of Public Health, University of Colorado Anschutz Medical Campus, 13001 E. 17th Place, Aurora CO 80045, USA
4 Department of Health Outcomes and Policy, 1329 SW 16th St., Gainesville FL 32608, USA
BMC Medical Research Methodology 2014, 14:37 doi:10.1186/1471-2288-14-37Published: 5 March 2014
Scientists often use a paired comparison of the areas under the receiver operating characteristic curves to decide which continuous cancer screening test has the best diagnostic accuracy. In the paired design, all participants are screened with both tests. Participants with suspicious results or signs and symptoms of disease receive the reference standard test. The remaining participants are classified as non-cases, even though some may have occult disease. The standard analysis includes all study participants, which can create bias in the estimates of diagnostic accuracy since not all participants receive disease status verification. We propose a weighted maximum likelihood bias correction method to reduce decision errors.
Using Monte Carlo simulations, we assessed the method’s ability to reduce decision errors across a range of disease prevalences, correlations between screening test scores, rates of interval cases and proportions of participants who received the reference standard test.
The performance of the method depends on characteristics of the screening tests and the disease and on the percentage of participants who receive the reference standard test. In studies with a large amount of bias in the difference in the full areas under the curves, the bias correction method reduces the Type I error rate and improves power for the correct decision. We demonstrate the method with an application to a hypothetical oral cancer screening study.
The bias correction method reduces decision errors for some paired screening trials. In order to determine if bias correction is needed for a specific screening trial, we recommend the investigator conduct a simulation study using our software.