Background
Many studies measure a continuous endpoint repeatedly over time. In some cases, this is because researchers wish to judge the time course of a symptom or to evaluate how the effect of a treatment changes over time. For example, in a study of thoracic surgery, patients were evaluated every three months after thoracic surgery to determine the incidence and duration of chronic postoperative pain. The researchers found that the incidence of pain at one year was high and only slightly lower than at three months, showing that post-thoracotomy pain is common and persistent
Measures may also be repeated in order obtain a more precise estimate of an endpoint. In simple terms, measure a patient once and they may be having a particularly good or bad day; measure them several times and you are more likely to get a fair picture of how they are doing in general. Repeat assessment reduces intra-patient variability and thus increases study power. This is of particular relevance to comparative studies. For instance, in a randomized trial of soy and placebo for cancer-related hot flashes, patients recorded the number of hot flashes they experienced each day during a baseline assessment period and then during treatment. In this case, the researchers were interested in the change between baseline and follow-up in each group so as to determine drug effect. The time course of symptoms was not at issue. The researchers therefore took a mean of each patient's hot flash score during the baseline period and subtracted the mean of the final four treatment weeks to create a change score. Change scores were compared between groups using a t-test
There has been little guidance in the methodologic literature as to how researchers should select the number of repeated measures for repeated measures designs. In the few papers that have discussed power and repeat measurement (for example, Frison and Pocock
Consequently, it is not difficult to find studies that appear to have either too few or too many repeat assessments. In a trial of acupuncture for back pain, for example, pain was measured on a visual analog scale (VAS) once at baseline and once following treatment
In this paper I argue that the number of repeat measures should not be seen as a fixed design characteristic, rather it is a design choice that can be informed by statistical considerations. I then outline a method for guiding decisions concerning the number of repeat measures and deduce several rules of thumb that can be applied in trial design.
Methods
To determine an optimal number of repeat measures, I use the premise that the ideal number from a statistical viewpoint is infinity, as this would maximally reduce intra-patient variance. However, it would be best in terms of researcher and patient time and effort if only a single assessment was made. Increasing the number of repeat assessments thus has a benefit in statistical terms that is offset by cost. Whereas cost can be estimated only in general terms by researchers (would patients put up with another questionnaire? how much time would it take for an additional range of motion assessment?) statistical efficiency benefits can be quantified. In the following, I describe the formulae for determining the relative benefit of additional repeat assessments for statistical power and deduce some general design principles.
The key question is the degree to which adding a further measure – for example, assessing pain five times rather than four times – increases statistical power. This is known as the "marginal benefit" of repeat measurement. We will start with the situation where data are recorded only after intervention. This is typical in trials of acute sequelae of a predictable event, for example, post-operative pain, chemotherapy nausea or muscle soreness following exercise. It can be shown (see Figure ).
Derivation of statistical formula
Derivation of statistical formula
Derivation of statistical formula
Derivation of statistical formula
Marginal change in sample size for
This equation does not require that measurements be equally spaced or that correlations between measurements be constant.
It is common that trials investigate an endpoint that can be informatively measured before treatment. In trials of back pain, hypertension or obesity, for example, researchers want to test whether an intervention reduces patients' pain scores, blood pressure or weight from a baseline value. Typically the endpoint in such trials is measured one or more times at baseline and again following treatment. Baseline and post-treatment scores are summarized separately and change analyzed.
Analysis of covariance (ANCOVA) has been repeatedly demonstrated to be the most powerful method of analysis for this type of trial
As is the case for trials without baseline measures, there is no requirement that correlations be equal or that assessments be equally spaced.
Frison and Pocock report that typical figures for
Empirical estimates of correlations from a variety of studies
Endpoint
When assessed
Pre*
Post**
Mix***
Profile of Mood States
Immediately before and after a single treatment
0.83
Profile of Mood States
Every three days during hospitalization
0.56
0.54
Cancer pain+
2, 24 and 48 hrs after single treatment
0.67
Daily scores of migraine headache+
Four weeks at baseline and at one year later
0.53
0.59
0.48
Brief Fatigue Inventory+
Weeks -1, 0, 5 and 6
0.76
0.94
0.65
Hot flashes+
Daily for one week at baseline and six weeks later
0.40
0.8
0.54
Shortness of breath+
Daily for seven days
0.82
Muscle soreness after running
Twice daily for five days
0.58
Chronic neck pain
Baseline and three weeks later
0.39
Neck range of motion
Before and after a single treatment
0.88
Neck pain
Before and after a single treatment
0.9
Constant Murley score of shoulder pain and dysfunction
Baseline and four weeks later
0.57
Back pain by visual analog score
Baseline and four weeks later
0.56
In chronic headache patients+:
SF36 Physical functioning
Baseline and one year later
0.75
SF36 Role limitation physical
Baseline and one year later
0.47
SF36 Role limitation emotional
Baseline and one year later
0.36
SF36 social functioning
Baseline and one year later
0.45
SF36 bodily pain
Baseline and one year later
0.52
SF36 general health
Baseline and one year later
0.74
SF36 health change
Baseline and one year later
0.24
SF36 energy fatigue
Baseline and one year later
0.66
Tumor size in mice
Every three days
0.89
Prostate specific antigen
Four times over 8 to 16 months
0.58
+ unpublished data * Mean correlation between baseline measures **Mean correlation between follow-up measures *** Mean correlation between baseline and follow-up measures
Results
Trials without baseline measures
Table
Marginal decrease in sample size for increasing the number of measures given various correlations between measures. The table refers to the case where no baseline measures are taken.
Correlation
Measures
0
0.15
0.35
0.5
0.65
0.85
1
2
-50.0%
-42.5%
-32.5%
-25.0%
-17.5%
-7.5%
0%
3
-16.7%
-14.2%
-10.8%
-8.3%
-5.8%
-2.5%
0%
4
-8.3%
-7.1%
-5.4%
-4.2%
-2.9%
-1.3%
0%
5
-5.0%
-4.3%
-3.3%
-2.5%
-1.8%
-0.8%
0%
6
-3.3%
-2.8%
-2.2%
-1.7%
-1.2%
-0.5%
0%
7
-2.4%
-2.0%
-1.5%
-1.2%
-0.8%
-0.4%
0%
8
-1.8%
-1.5%
-1.2%
-0.9%
-0.6%
-0.3%
0%
9
-1.4%
-1.2%
-0.9%
-0.7%
-0.5%
-0.2%
0%
10
-1.1%
-0.9%
-0.7%
-0.6%
-0.4%
-0.2%
0%
11
-0.9%
-0.8%
-0.6%
-0.5%
-0.3%
-0.1%
0%
15
-0.5%
-0.4%
-0.3%
-0.2%
-0.2%
-0.1%
0%
Trials with baseline measures
Tables
Sample sizes for various combinations of baseline (p) and follow-up (r) measures. Correlations for
Number of baseline measures (p)
No. of follow-up measures (r)
1
2
3
4
5
6
7
8
9
10
12
15
1
100%
94%
92%
90%
89%
89%
88%
88%
88%
88%
87%
87%
2
80%
74%
72%
70%
69%
69%
68%
68%
68%
68%
67%
67%
3
73%
67%
65%
64%
63%
62%
62%
61%
61%
61%
61%
60%
4
70%
64%
62%
60%
59%
59%
58%
58%
58%
58%
57%
57%
5
68%
62%
60%
58%
57%
57%
56%
56%
56%
56%
55%
55%
6
67%
61%
58%
57%
56%
56%
55%
55%
55%
54%
54%
54%
7
66%
60%
57%
56%
55%
55%
54%
54%
54%
53%
53%
53%
8
65%
59%
57%
55%
54%
54%
53%
53%
53%
53%
52%
52%
9
64%
59%
56%
55%
54%
53%
53%
53%
52%
52%
52%
51%
10
64%
58%
56%
54%
53%
53%
52%
52%
52%
52%
51%
51%
12
63%
57%
55%
54%
53%
52%
52%
51%
51%
51%
51%
50%
15
63%
57%
54%
53%
52%
52%
51%
51%
51%
50%
50%
50%
Sample sizes for various combinations of baseline (p) and follow-up (r) measures. Correlations for
Number of baseline measures (p)
No. of follow-up measures (r)
1
2
3
4
5
6
7
8
9
10
12
15
1
100%
89%
83%
80%
78%
76%
75%
74%
73%
73%
72%
71%
2
67%
56%
50%
47%
44%
43%
42%
41%
40%
39%
38%
38%
3
56%
44%
39%
36%
33%
32%
31%
30%
29%
28%
27%
26%
4
50%
39%
33%
30%
28%
26%
25%
24%
23%
23%
22%
21%
5
47%
36%
30%
27%
24%
23%
22%
21%
20%
19%
18%
18%
6
44%
33%
28%
24%
22%
21%
19%
19%
18%
17%
16%
15%
7
43%
32%
26%
23%
21%
19%
18%
17%
16%
16%
15%
14%
8
42%
31%
25%
22%
19%
18%
17%
16%
15%
14%
13%
13%
9
41%
30%
24%
21%
19%
17%
16%
15%
14%
13%
13%
12%
10
40%
29%
23%
20%
18%
16%
15%
14%
13%
13%
12%
11%
12
39%
28%
22%
19%
17%
15%
14%
13%
12%
12%
11%
10%
15
38%
27%
21%
18%
16%
14%
13%
12%
11%
11%
10%
9%
Sample sizes for various combinations of baseline (p) and follow-up (r) measures. Correlations for
Number of baseline measures (p)
No. of follow-up measures (r)
1
2
3
4
5
6
7
8
9
10
12
15
1
100%
91%
87%
86%
85%
84%
83%
83%
83%
82%
82%
82%
2
86%
77%
73%
72%
71%
70%
69%
69%
69%
69%
68%
68%
3
81%
72%
69%
67%
66%
65%
65%
64%
64%
64%
64%
63%
4
79%
70%
66%
65%
64%
63%
63%
62%
62%
62%
61%
61%
5
78%
68%
65%
63%
62%
62%
61%
61%
60%
60%
60%
59%
6
77%
67%
64%
62%
61%
61%
60%
60%
60%
59%
59%
59%
7
76%
67%
63%
62%
61%
60%
60%
59%
59%
59%
58%
58%
8
76%
66%
63%
61%
60%
60%
59%
59%
58%
58%
58%
57%
9
75%
66%
63%
61%
60%
59%
59%
58%
58%
58%
57%
57%
10
75%
66%
62%
61%
60%
59%
58%
58%
58%
57%
57%
57%
12
75%
65%
62%
60%
59%
58%
58%
57%
57%
57%
57%
56%
15
74%
65%
61%
60%
59%
58%
57%
57%
57%
56%
56%
56%
Table
As an example, given the most common case of
Relative decrease in sample size given various scenarios for increasing the number of baseline (p) or follow-up (r) measures.
pre
post
mix
Change from
Change to
Sample size reduction
No. of baseline measures (p)
No. of follow-up measures (r)
No. of baseline measures (p)
No. of follow-up measures (r)
0.7
0.7
0.5
1
1
1
4
30%
4
1
10%
4
4
40%
4
4
4
7
7%
7
4
3%
7
7
10%
7
7
7
14
5%
14
7
2%
14
14
8%
14
14
28
28
4%
0.5
0.5
0.5
1
1
1
4
50%
4
1
20%
4
4
70%
4
4
4
7
24%
7
4
17%
7
7
40%
7
7
7
14
27%
14
7
22%
14
14
48%
14
14
28
28
49%
0.9
0.9
0.8
1
1
1
4
21%
4
1
14%
4
4
35%
4
4
4
7
5%
7
4
3%
7
7
8%
7
7
7
14
3%
14
7
3%
14
14
6%
14
14
28
28
3%
Some general patterns emerge:
1. Repeating measures can have dramatic effects on power. Increasing the number of follow-up and / or baseline measures from a single one to three or four can reduce sample sizes by 35 – 70%. However, the increases in power for each additional measure rapidly decreases with increasing number of assessments.
2. Under the assumption that
3. The marginal value of additional follow-up assessments is higher where baseline measurements are taken. Take the case where
4. The only situation where it is worthwhile to make more than six or seven assessments is when correlation is moderate and similar between all time periods. This is most likely to be the case for episodic conditions such as headache, where scores at any one time will be poorly correlated with scores at any other time.
Conclusion
Investigators may measure a continuous endpoint repeatedly because they wish to judge the time course of a symptom. In such cases, the number of repeat measures will depend upon the scientific interests of the investigators. Alternatively, investigators may use repeat measurement to increase the precision of an estimate. Though this is a particular concern for randomized or non-randomized comparative studies, it is also pertinent to a variety of other research designs: for example, epidemiologic cohort studies may take a measure such as blood pressure, prostate specific antigen or serum micronutrient levels at baseline and then determine whether this predicts development of disease; repeating baselines will improve the precision of such predictions.
Where measures are repeated to improve precision, decisions about the number of repeated measures, that is, the number of within-patient observations, mirror those of standard power calculation, which concerns observations of separate patients. In both cases, statistical concerns to minimize variance are balanced by logistical concerns to minimize number of assessments. Whilst an extensive literature has developed on various methods for selecting a particular number of patients to study, the number of assessments per patient has received little attention, perhaps because this has tended to be seen as a fixed characteristic of any particular trial design. Here I have shown that simple statistical considerations can be used to guide the number of repeated measures in repeated measures designs. Given the most common correlation structure, taking four baselines and seven follow-up measures dramatically improves power compared to a single baseline and follow-up; where no baseline is taken, four follow-up measures importantly improves power; however, the marginal value of including additional measures rapidly diminishes.
Competing interests
None declared