Pediatric screening urinalysis: a difference-in-differences analysis of how a 2007 change in guidelines impacted use

Abstract

Background

Practice guidelines can promote higher-quality care, yet they are inconsistently adopted. The purpose of this study is to evaluate the impact of a 2007 American Academy of Pediatrics recommendation to discontinue routine screening urinalysis in children.

Methods

Using data from the National Ambulatory Medical Care Survey, we used a difference-in-differences approach to estimate visit-level screening urinalysis proportions before (2005-2006, n = 1,247) and after (2008-2009, n = 1,772) the 2007 AAP recommendation. We compared visits by children 4-18 years old to visits by young adults aged 19-32. Analyses were adjusted for continuous patient age, patient race/ethnicity, physician specialty, and stratified by patient gender and visit setting.

Results

The 2007 recommendation was associated with no significant change in adjusted visit-level screening urinalysis proportions in child visits (20.4% to 22.5%) compared to an increase in young adult visits (20.1% to 27.0%) – a differential impact of -4.8 percentage points (95% Confidence Interval [CI] -9.0, -0.5). In private practices, visit proportions differentially decreased by 7.6 percentage points (95% CI -13.7, -1.5) in female children and by 0.5 percentage points (95% CI -10.6, 9.6) in male children. In community health centers, visit proportions differentially decreased by 17.4 percentage points (95% CI -27.9, -6.8) in female children and by 33.5 percentage points (95% CI -47.4, -19.7) in male children.

Conclusions

A 2007 recommendation to discontinue routine screening urinalysis in children was associated with no change in use in child visits relative to an increase in use in adult visits. Overall, nearly one-quarter of child visits still included screening urinalysis.

Background

Standardized practice guidelines developed by professional societies and other health entities can help physicians make informed decisions about appropriate use of preventive services. Yet physicians who care for children, like many physicians, inconsistently utilize preventive care guidelines [1–3]. Attention to evaluating the quality of pediatric preventive care has been growing, and many previous studies [4–8] have examined whether recommended services are being delivered to children. Less attention has been devoted to evaluating whether services that are no longer recommended are being discontinued.

Recommendations regarding routine urine screening for the detection of renal or urologic disease in children have evolved over several decades. In both 1977 and 1991, the American Academy of Pediatrics (AAP) recommended routine urine screening at four time points during childhood [9]. Revised health supervision guidelines in 1995 [10] and 2000 [11] limited screening to only two age groups, five-year-olds and sexually active adolescents. Accumulated evidence now suggests that although inexpensive, screening urinalysis is a poor screening test for disease, it can lead to false positives and associated costly and invasive diagnostic evaluations, and there is limited evidence to suggest detection of abnormalities in childhood improves long-term outcomes [12–14]. In light of the lack of clear benefit associated with screening urinalysis relative to its associated costs and risks, the AAP in 2007 removed routine screening urinalysis for asymptomatic children and adolescents from its health supervision guidelines altogether [15]. In 2005, a survey of pediatricians showed that many still reported routinely screening children in non-recommended age groups [9]. Little is known about how this recent change in recommendations has impacted physician practice.

Using data from the National Ambulatory Medical Care Survey (NAMCS), a nationally representative survey assessing provision of ambulatory medical care services in the United States, we used a difference-in-differences approach to determine whether the 2007 AAP recommendation resulted in a differential impact on the proportion of child visits that included screening urinalysis compared to young adult visits. Because visit setting and patient gender may have an impact on screening urinalysis we stratified our analysis by visit setting (community health center or private practice) and by patient gender. Our findings will inform clinicians and guideline developers about current screening urinalysis practices among children, whether and how the AAP recommendation impacted physician behavior, and whether recommendations alone are sufficient in changing physician behavior.

Methods

Data source

We analyzed data from the National Ambulatory Medical Care Survey (NAMCS), a nationally representative data set of ambulatory visits to office-based physicians in the United States. Physicians, office staff, or survey administrators from the U.S. Bureau of the Census enter data, including demographic and clinical parameters, for a systematic random sample of visits; each visit is weighted to allow for extrapolation to nationally representative estimates. Conducted on an annual basis, the NAMCS offers a unique opportunity to estimate pediatric screening urinalysis prevalence before and after enactment of the 2007 AAP recommendation. Our study period of interest included two years before and two years after the AAP recommendation in 2007: from 2005-2006 and from 2008-2009. Data from 2007 were excluded to allow for an implementation period. The study was determined to be exempt from Committee Review by the Yale University Human Investigation Committee. The research has adhered to the STROBE guidelines for observational studies as outlined at http://www.strobe-statement.org.

Study design and sample

We used a difference-in-differences approach [16] to quantify the impact of the AAP’s 2007 recommendation on the proportion of pediatric visits that included screening urinalysis. The study design involves consideration of study and comparison groups, before and after an intervention is applied to only the study group, in order to help identify differences in an outcome associated with the intervention separately from differences in covariates and secular trends (i.e. trends in testing over time that are unrelated to the timing of the AAP recommendation change and time-invariant differences between the two age groups).

Our study sample included visits to physicians who commonly see children, defined for the purpose of this analysis as pediatricians and family practice physicians. Visits to internal medicine physicians were excluded because it was anticipated that they would care for only adult populations. Because the NAMCS survey instrument does not distinguish between screening and diagnostic urinalysis, we defined screening urinalysis as urinalysis ordered in the context of a preventive care visit as designated by the physician; for each surveyed visit, providers are asked to select a categorical reason for the patient’s visit (new problem, chronic problem-routine, chronic problem-flare up, pre/post surgery, or preventive care). We studied children for whom catheterization would not likely be necessary to perform a screening urinalysis, defined for this analysis as those over age 4 years. Therefore, our analysis included visits by children aged 4-18 years.

For comparison, we designated a sample of visits to represent secular trends in screening behavior. Given our interest in the AAP recommendation, an ideal comparison group would be visits by children aged 4-18 years to physicians who were not exposed to the AAP recommendation. Due to the national applicability of the AAP guidance, data for such a comparison group was not available for visits in the United States. Rather, we selected as a comparison group preventive care visits to pediatricians or family practice physicians by young adults aged 19-32 years during the study periods of interest. To our knowledge universal screening urinalysis for adults was not recommended at any point in the study period, so its pattern of use was expected to reflect secular trends in physician screening practices and was not anticipated to be affected by the AAP guidance.

Our primary outcome of interest was the proportion of visits including screening urinalysis, defined as urinalysis ordered in the context of a physician-identified preventive care visit.

Statistical analysis

We used NAMCS data to establish a baseline description of pediatric and young adult preventive care visits to pediatricians and family practitioners in 2005-2006, comparing unadjusted visit-level urinalysis proportions, patient gender, physician specialty, and visit setting. Our main analysis compared overall differences for visit-level rates of screening urinalysis in children before and after the 2007 AAP recommendation to the differences in young adults. We fit a nonlinear difference-in-differences model [16], adjusting for covariates, age category, and time period using probit regression.

Analyses were adjusted for or stratified by patient- and physician-level characteristics we anticipated might impact screening urinalysis use. We adjusted for the patient-level variable of continuous age because we anticipated it could affect prevalence of testing, given previous age-specific recommendations; analyses were also adjusted for patient race/ethnicity (collapsed into three categories, white, black, or other race/ethnicity because of small sample sizes in other categories). We also adjusted for physician specialty (pediatrician or family practitioner,) because we anticipated pediatricians may be differentially receptive to guidance from a pediatric-specific professional organization. Sample size limitations precluded a stratified analysis by physician specialty type. We anticipated practice setting may have an impact on screening urinalysis use due to factors such as laboratory access, patient insurance mix, and provider characteristics, so analyses were stratified by the system-level variable of visit setting (private practice or community health center). Because screening urinalysis may be differentially utilized between females (for purposes such as pregnancy testing) and males, we also stratified analysis by patient gender. Data from our NAMCS sample were weighted according to the National Center for Health Statistics weighting procedure to produce national estimates and adjusted for the survey design [17]. Computation was in Stata version 12 [18].

Results

Baseline characteristics

This analysis included preventive care visits from two time periods (2005-2006 and 2008-2009) involving two age groups (children 4-18 years and young adults 19-32 years). Overall, we analyzed 3,019 preventive care visits: 1,247 visits from 2005-2006 and 1,772 visits from 2008-2009. Before 2007, a majority of child and adult visits occurred in the private practice setting (95.6% and 97.3%, respectively). The majority of visits by children (75.7%) were to pediatricians while most visits by young adults (93.7%) were to family medicine practitioners; young adult visits were predominantly made by females (71.1%) (Table 1).

Table 1 Baseline characteristics of preventive care visits before 2007 (2005-2006)

Screening urinalysis before and after the 2007 AAP policy statement

Overall, the adjusted proportion of visits including screening urinalysis prior to the 2007 AAP policy statement were similar in preventive care visits by children aged 4-18 years and visits by young adults aged 19-32 years. Adjusted screening urinalysis proportions before and after the 2007 recommendation remained flat in child visits, from 20.4% before to 22.5% after, but increased in young adult visits, from 20.1% to 27.0%, resulting in a differential impact on child visits of -4.8 percentage points (95% confidence interval [CI], -9.0 to -0.5, p = 0.03).

Compared to young adults, after stratifying analyses by visit setting and patient gender, adjusted screening urinalysis proportions in female child visits to private practices differentially decreased by 7.6 percentage points (95% CI -13.7 to -1.5, p = 0.02). In male child visits to private practices, proportions differentially decreased by 0.5 percentage points (95% CI -10.6 to 9.6, p = 0.93). In community health center visits by female children, proportions differentially decreased by 17.4 percentage points (95% CI -27.9 to -6.8, p = 0.001); in community health center visits by male children, proportions differentially decreased by 33.5 percentage points (95% CI -47.4 to -19.7, p < 0.001) (Table 2). Of note, sample sizes were small for some cells, leading to imprecise estimates.

Table 2 Adjusted proportion of child visits including urinalysis before and after 2007, compared to young adult visits

Discussion

In the two year periods before and after a 2007 recommendation to discontinue routine screening urinalysis in children, we observed no change in screening urinalysis use in child visits. In adult visits, screening urinalysis rates rose, resulting in a differential decrease in child visits of nearly 5 percentage points. Our stratified analyses by visit setting and patient gender also revealed differential impacts in child visits compared to young adult visits. Overall, screening urinalysis use persisted in nearly a quarter of pediatric preventive care visits after 2007.

The increase in urinalyses observed in adult visits could be explained by several factors. A concurrent change in adult guidelines could lead to an increase in the proportion of adult visits including screening urinalysis. However, at no time during the study period was universal screening urinalysis recommended by major adult preventive care guidelines developers such as the United States Preventive Services Task Force (USPSTF) [19, 20] or the American Academy of Family Physicians (AAFP) [21, 22], making this explanation less likely. Or, it could be attributed to an increase in adult visits by subgroups for whom urinalysis may be an appropriate screening test, such as pregnant women or patients with diabetes, kidney disease, or hypertension. Similarly, routine urine chlamydia tests could have been misclassified as urinalyses. However, we did not anticipate significant differences in the proportions of these subgroups visiting physicians in either age group in the short period before or after the AAP recommendation, so find this explanation to be less likely as well. Therefore, we interpret the absolute increase in the proportion of adult visits including screening urinalysis to a general upward trend in screening tests – a trend that could be accounted for in part by increased adoption of the medical record, which has been associated with increased provision of preventive health services generally [23, 24].

The observed differential decrease in pediatric screening urinalysis in our study was consistent with the observed impact of another recently discontinued preventive care practice. In 2008, the USPSTF recommended discontinuation in older men of prostate cancer screening with the prostate-specific antigen test, which was shown to result in a small but significant impact on screening [25]. Unfortunately, discontinued routine child preventive care screening tests are not common, so comparison in the pediatric population is difficult.

Our stratified analyses by visit setting and patient gender also revealed differential impacts. We observed consistently larger differential decreases in screening urinalysis proportions in the community health center setting than in the private practice setting for both genders. This finding is consistent with previous studies demonstrating comparable or higher quality care delivery in community health center settings compared to other settings [26–28]. Community health centers have also been shown to provide cost-effective care [28], which could make it more likely that community health center practitioners would more rapidly discontinue a procedure such as urinalysis (which can be time-intensive and minimally revenue-generating) once it was no longer recommended. Our gender-stratified results were mixed and thus more difficult to interpret; while urinalysis in female child visits differentially decreased in both settings, in male visits we observed a large differential decrease in the community health center setting compared to no change in the private practice setting.

Despite observed differential decreases, the overall proportion of pediatric visits including screening urinalysis remained high. The persistence of screening urinalysis use – in nearly a quarter of all child visits - could be explained by a physician lack of awareness of the AAP guideline [1–3]. Additionally, physicians are subject to a wide array of clinical directives – a 2006 study [29] identified over 340 AAP policy statements alone, of which 57 were deemed broadly relevant to pediatric practice. Alternatively, physicians may have experienced previously cited barriers to guideline adoption, such as lack of awareness or familiarity, lack of self-efficacy in adopting a guideline, or inability to overcome the inertia of previous practice [30], many of which have been noted in studies of pediatricians and family practitioners [1–3, 31]. Another possibility is that physicians were considering other, potentially conflicting, input when making decisions about the use of screening urinalysis. The appropriate use of screening urinalysis as a routine screening test has been debated for decades and the discourse continues [32]. However, in this case, a relatively high degree of consensus regarding pediatric screening urinalysis had been reached with concordant guidelines from at least several major guideline developers, including the USPSTF [19, 20], the AAFP [21, 22], and the American Medical Association’s Guidelines for Adolescent Preventive Services [33]. Physicians may also have conducted screening urinalyses for other reasons than to detect disease, for instance to meet requirements for school entry or sports participation that had not been changed to reflect new evidence about the utility of the screening test [34].

This study’s strengths include use of a strong, quasi-experimental, difference-in-differences approach [35] to evaluate the impact of the 2007 AAP recommendation. Our research design accounts for secular time trends in test use and differential time trends in unobserved covariates between the age groups. Furthermore, it did not depend on physician report of screening practices to measure screening urinalysis rates. However, it is important to consider the limitations of this analysis as well. Because the NAMCS visit form does not distinguish between screening and diagnostic urinalyses, our sample could include visits by patients who presented for a preventive care visit who simultaneously had acute complaints that would warrant a diagnostic urinalysis – leading to misclassification. There are a number of other conditions and circumstances in which urinalyses would be warranted in the context of preventive care visits, noted above, that could have biased our results. Additionally, this sample includes only physicians; patterns of adherence may be different for other types of practitioners.

Conclusions

In conclusion, our study offers evidence that a 2007 standardized practice guideline differentially impacted routine screening urinalysis use in child compared to adult ambulatory care visits. Differential impacts were noted across visit settings and patient genders. However, screening urinalysis use persisted in nearly a quarter of pediatric preventive care visits. More research is needed to understand why and how females and males are impacted differentially by screening guidelines, what differentiates practice settings, and why physicians do or do not discontinue screening tests once they are no longer recommended.