Medical records were abstracted in 1998 at 14 sites of Harvard Vanguard Medical Associates (HVMA, formerly known as Harvard Community Health Plan, and later, the Health Centers Division of Harvard Pilgrim Health Care) in greater Boston, Massachusetts. In 1998, more than 90% of patients at HVMA had capitated health insurance coverage from a single health maintenance organization, Harvard Pilgrim Health Care.

At the time of the study, HVMA was transitioning from one electronic medical record system, the COSTAR-based Automated Medical Record System 8910 to the current system, EpicCare^® 11. EpicCare^® records constituted the majority of those reviewed for this study. Both systems have designated, coded fields for capturing the quality measures described below, and both systems have free-text fields for clinicians to enter narrative information from patient encounters. Physicians themselves enter the majority of data in EpicCare^®, either by populating specific fields (e.g., smoking status, including number of cigarettes per day and duration of smoking; alcohol consumption [yes/no and ounces per week]) or by keyboard entry of free text. Depending on the physician's style, most data entry occurs immediately upon completion of the office visit, while some occurs during the visit in the presence of the patient and some occurs at the end of a clinical session. A minority of office notes are dictated; these transcribed notes are entered only as narrative text and do not result in any coding of specific fields.

At the time of the study, 212 physicians provided primary care at HVMA, of whom 167 had completed their malpractice insurance carrier's biennial survey of clinical and teaching activities; these 167 primary care physicians (117 internists and 50 pediatricians) constitute the study sample. With three exceptions, five patient records per physician were selected at random for review; four records were reviewed for each of two physicians and six records for one physician. A total of 834 patient records were reviewed.

Data sets containing medical record review information and self-reported physician teaching and patient care activities were obtained through permission of the Harvard Risk Management Foundation. Medical record reviews were conducted as a component of quality assurance procedures to fulfill accreditation requirements of the National Center for Quality Assurance (NCQA). Trained nurse abstractors from a medical record consulting firm conducted the reviews using an abstraction instrument containing 51 questions modeled after NCQA's 1999 Health Plan Employer Data and Information Set (HEDIS) quality measures 12. Nurse abstractors reviewed both the free-text and coded-field components of the medical records.

Physicians reported the amount of direct patient care they provided in hours per week; they reported the amount of teaching they did as measured by the number of patients per month for whom they supervised care, as opposed to directly provided care. Physicians were classified as teachers if they reported supervising the care of any patients each month or supervising patient care in any months of the year. Specialty, gender and year of medical school graduation were obtained for each physician from the World Wide Web sites of the American Medical Association and the medical licensing boards of Massachusetts, New York, and California 13141516. Practice site was dichotomized according to whether or not the site was located within Boston city limits, because we anticipated significant differences in patient populations and physician practices across sites in these locations.

Before examining the data, we selected for analysis five measures from the chart review instrument that we believed were the most important indicators of quality of medical record documentation in ambulatory care. The five measures were medications, allergies, immunizations, smoking history and compliance with age-appropriate screening guidelines. All measures were dichotomous (yes/no) variables that were coded "yes" if the trained abstractors determined that the relevant information was appropriately documented. For medication documentation, we created a dichotomous (yes/no) variable that was scored "yes" if the record contained documentation of current medications either in the encounter text of the complete history and physical examination or in the medication list (i.e., medication field) of the computerized medical record. Allergy documentation was coded in a similar fashion. The other three measures – immunizations, smoking history and compliance with age-appropriate screening guidelines – are analogous to items found in the 1999 HEDIS List of Measures under "Effectiveness of Care" 12. For "compliance with age-appropriate screening guidelines," abstractors referred to Harvard Vanguard's clinical guidelines at the time of the study. These guidelines included colorectal cancer screening for adults age 50 years or older, mammography for women age 50 years or older, and Pap smears for all women. Prostate cancer screening was not included in this measure. For children, age-appropriate screening included sensory screening (vision and hearing), and lead testing. The "compliance with age-appropriate screening guideline" variable was coded "yes" if the abstractor determined that each relevant screening item had been documented.

There were no missing data. Chart abstractors coded "not applicable" for one or more measure on less than one percent of the records reviewed. We recoded this small number of "not applicable" values as "no" on the assumption that any items not coded as "yes" were lacking appropriate documentation of the specified measures.

Characteristics of physicians were compared by specialty using Student's t-test for continuous measures and by Fisher's exact test for categorical variables.

Analyses of documentation behavior were stratified by specialty because we anticipated differences between pediatricians and internists due to the nature of their patient populations, methods of practice, and the documentation measures we selected for analysis. No analyses were performed for immunization documentation by pediatricians since all pediatricians documented immunizations.

To account for clustering of patients within each physician, we used mixed effects (hierarchical) logistic modeling for documentation outcomes 17. This technique accounts for the correlation between outcomes measured on patients who share a physician 18. We determined whether physician teaching status, physician gender, years since medical school, clinical hours per week of direct patient care, practice site, patient age, and patient gender were correlated with better chart documentation for each of the five measures. We hypothesized a priori that physicians' teaching status would be correlated with the quality of documentation. Other physician characteristics were included because we expected that documentation practice would vary by clinician experience and by geography, although we did not have an a priori hypothesis about the direction of these effects.

Analyses were performed using the Glimmix Macro in Statistical Analysis System (SAS) software 1920

The study was approved by the Human Studies Committee (Institutional Review Board) of Harvard Pilgrim Health Care.

Table 1 shows the characteristics of the 117 internists and 50 pediatricians in the study. About two-thirds of internists and pediatricians practiced outside Boston City limits. While internists reported more clinical hours per week than pediatricians (34 ± 14 vs. 29 ± 9; P = 0.006), years since completing medical school were higher for pediatricians (18 ± 8 vs. 22 ± 9; P = 0.003).

The mean (± SD) age of patients was 9.3 ± 5.5 years in the pediatrics group and 57.5 ± 15.0 in the internist group. Fifty-five percent of all patients were female.

Table 2 shows the unadjusted rates of fulfillment of the five measures of medical record documentation quality for internists and for pediatricians.

No consistent patterns emerged among the physician or patient characteristics and documentation quality measures in this study (Table 3 – see Additional File). Female internists were more likely than male internists to document smoking history (odds ratio [OR], 1.90; 95% confidence interval [CI], 1.27 – 2.83) but were less likely to document drug allergies (OR, 0.51; 95% CI, 0.35 – 0.75). With each subsequent year since completing medical school, internists were less likely to document drug allergies (OR, 0.97; 95% CI, 0.95 – 0.99) and immunizations (OR, 0.91; 95% CI, 0.87–0.96). For internists, increased clinical time was associated with better documentation for smoking history (OR, 1.02; 95% CI, 1.01 – 1.03) but not for any of the other measures. As compared with internists practicing at sites within Boston city limits, those practicing outside Boston city limits were less likely to document, medications (OR, 0.47; 95% CI, 0.26 – 0.85), and immunizations (OR, 0.19; 95% CI, 0.07–0.47) but were more likely to document smoking history (OR, 1.55; 95% CI, 1.08 – 2.20).

Internists were more likely to document drug allergy status (OR, 2.48; 95% CI, 1.65 – 3.74), medications (OR, 1.49; 95% CI, 1.18 – 1.87), and compliance with screening guidelines (OR, 1.36; 95% CI, 1.06 – 1.73) for female patients than for male patients. However, internists were less likely to document smoking history (OR, 0.51; 95% CI, 0.34 – 0.79) and immunizations (OR, 0.61; 95% CI, 0.47–0.79) for female patients than for male patients. Internists were less likely to document smoking history (OR, 0.98; 95% CI, 0.96 – 0.99) but more likely to document immunizations (OR, 1.02; 95% CI, 1.01 – 1.03) with every yearly increase in patient age.

Female pediatricians were less likely than male pediatricians to document drug allergy status (OR, 0.47; 95% CI, 0.28 – 0.79) but were more likely to document smoking history (OR, 2.78; 95% CI, 0.85 – 9.11); interpretation of this latter result is limited by imprecision of the effect size as manifested by the wide confidence interval. For pediatricians, increased clinical time was associated with better documentation for compliance with screening guidelines (OR, 1.22; 95% CI, 1.06 – 1.41) but not for any of the other measures. When compared with pediatricians practicing in the City of Boston, pediatricians practicing outside Boston city limits were more likely to document drug allergies (OR, 3.17; 95% CI, 1.05 – 9.57) and medications (OR, 7.93; 95% CI, 1.87 – 33.68) but less likely to document compliance with screening guidelines (OR, 0.08; 95% CI, 0.01 – 0.58). Pediatricians were more likely to document smoking history with increasing patient age (OR, 1.61; 95% CI, 1.12 – 2.31). Pediatricians were more likely to document drug allergy status for female patients than for male patients (OR, 1.62; 95% CI, 1.19 – 2.20) but less likely to document compliance with age-appropriate screening guidelines for female patients (OR, 0.17; 95% CI, 0.08 – 0.38).