Table 2

Inter-rater scoring and agreement by item on the goal-evaluation tool
Goal evaluation tool-diabetes (GET-D) items Frequency of scores* Raw agreement (%) Kappa
0 1 2 3
Goal 1: Relation to Self-Management 24 146 91% 0.62
Goal 2a: Measurable Specificity 128 42 95% 0.87
Goal 2b: Presence of a Deadline 164 6 98% 0.65
Action Plan 3: Relation to the Goal 55 115 87% 0.71
Action Plan 4a: Specific Action 54 116 91% 0.78
Action Plan 4b: Frequency or Schedule 145 25 96% 0.86
Action Plan 4c: Location 168 2 98% −0.01
Action Plan 4d: Intensity or Duration 155 15 94% 0.63
Action Plan 4e: Activity Monitoring 167 3 99% 0.66
Action Plan 5: Feasibility 36 114 13 7 95% 0.77
Average across all 10 Criteria 94.4% 0.66

Legend:

*Criteria 1–4 were scored as either 0,1 or 0,3, to weight some criteria as more critical than others. Criteria 5 was scored as 0,1,2,3. The distribution of scores is based on scores provided by both raters (n=170 total, n=85 for each rater). There were no missing data for any criteria.

Teal et al.

Teal et al. BMC Medical Informatics and Decision Making 2012 12:152   doi:10.1186/1472-6947-12-152

Open Data