Table 2

Sources of variation and reliability of quality assessments by physician implicit review

HTN

COPD

Diabetes

Acute

Mean rating (1 – 6) *

3.41

3.09

3.39

2.90

Variance ^§

Reviewer

0.15

0.34

0.34

0.33

Patient (between record)

0.75

0.30

0.88

0.26

Noise (within record)

0.72

0.50

0.69

1.03

Total variance

1.62

1.14

1.90

1.62

Proportion of total variance ^†

Reviewer^‡

0.09

0.30

0.18

0.20

Patient (between record)

0.46

0.26

0.46

0.16

Noise (within record)

0.44

0.44

0.36

0.64

Correlation of quality ratings at patient level

COPD

0.382

Diabetes

0.767

0.106

Acute

0.359

0.314

0.262

*Ratings are on a 1–6 scale where 1 = very good care and 6=very poor care. This is the rating for the average patient accounting for differences in reviewer severity. ^† Other than rounding error, the proportion of total variance sums to 100 across the 3 components. ^‡ The proportion of variance at the patient level represents the reliability of structured implicit review for detecting differences between patients when assessed for samples of patients from a similar population and assessed by reviewers drawn from a population of similar reviewers. ^§ Variation at the patient level represents the true score differences in physician ratings of quality of care across patients in regard to the specified condition. Variation at the reviewer level represents the idiosyncratic differences in rating severity the between physician reviewers. Variation at the review occasion level (repeated measurements of the same record) is termed "noise."

Hofer et al. BMC Health Services Research 2004 4:9   doi:10.1186/1472-6963-4-9