Table 1 |
|
|
Variables and Definitions Used in Systematic Review |
|
| Variable |
Definition |
|
|
|
| Sample size |
Number of subjects who participated in the research at the initiation of the study |
| Gender |
Percent of sample male/female |
| Age |
Average age of sample |
| Ethnicity |
Percent of identified racial or ethnic categories |
| Response rate |
Sample size divided by the total number of people approached for participation |
| Attrition rate |
Number of subjects finishing study divided by sample size |
| Sampling Frame |
How sample was identified–random, systematic, mixed, none, not indicated |
| Research Design |
Prospective, cross-sectional or retrospective design |
| Statistics |
Based on highest level used: descriptive, univariate, or multivariate |
| Funding Source |
Who paid for research |
| Control Group |
Whether or not a control group was used in design |
| Reliability |
Any indication of the reliability of any measure used. |
| Random assignment |
Use of random assignment to groups |
| Confounding variable |
Any measurement or statistical applications that attempt to identify and control for
variables that might influence outcomes independent of treatment. |
|
|
|
| Threats to validity |
|
|
|
|
| Threats to Statistical Conclusion Validity |
|
| Low statistical power |
Rated as any groups < 10 or correlations with fewer than 30 pairs. |
| Violated assumption |
Evidence of non-normal distributions with parametric statistics |
| Fishing/error rate |
More than 10 statistical tests without a Bonferroni (or similar) correction. |
| Reliability of measures |
Failure to test, note or reference information on measures |
| Reliability of treatment |
Failure to test or note consistently of the application of treatments |
| Random irrelevancies |
Absence of effort to make measurements reasonably consistent. |
| Random heterogeneity representative |
Absence of effort to ensure that sample is reasonably |
| Threats to Internal validity |
|
| History |
Repeated measures without temporal control |
| Maturation |
Repeated measures without temporal control on developmentally sensitive outcomes. |
| Testing |
Use of measures that are sensitive to the testing process. |
| Instrumentation |
Use of poorly or uncalibrated measures. |
| Statistical regression |
Study of cases selected from extremes without control group |
| Mortality |
More than 30% of sample did not complete study |
| Interaction with selection |
Evidence that selection into groups might interact with history, maturation, or testing. |
| Ambiguity of cause |
An association that can be interpreted in either direction vis-à-vis cause and effect. |
| Diffusion of treatment |
Contact between experimental and control subjects. |
| Compensatory work to Equalization |
Evidence that groups are knowledgeable about design and might Equal things out. |
| Compensatory rivalry |
Evidence that groups are knowledge about design and might compete with other group
members. |
| Resentful demoralization |
Evidence that one group feels disadvantaged through group assignment process. |
| Threats to Construct Validity |
|
| Inadequate explication |
Evidence that choice of measurement operations does not represent the construct. |
| Mono-operation bias |
Use of only one question for central outcome |
| Mono-method bias |
Use of only one measurement approach (e.g self report) for central outcome |
| Hypothesis guessing |
Evidence that subjects might attempt to guess what results should be (absence of appropriate
blinding procedures). |
| Evaluation apprehension |
Evidence that subjects might become anxious during assessments. |
| Experimenter bias |
Absence of controls to keep invested parties from participating in the measurement
procedures. |
| Confounding constructs |
Range restriction in measurement |
| With level of constructs |
|
| Threats to External Validity |
|
| Interaction of different Treatment |
Failure to assess additional treatment received during study. |
| Interaction of testing And treatment |
Evidence that testing might be related to the treatment so that subjects complete
tests differently after treatment. |
| Restricted generalizability |
Evidence of a limited measurement approach that may not generalize. |
| Sample bias |
Evidence of a poorly chosen sample that does not represent the population considered. |
| Interaction of selection And treatment |
Evidence that group assignment is influenced by variables related to the likelihood
of response to treatment |
| Interaction of setting And treatment |
Evidence that treatment works only in some settings. |
| Interaction of history And treatment |
Evidence that the time during which the study was done may have an impact on findings. |
|
|
|
|
Jonas et al. BMC Complementary and Alternative Medicine 2001 1:12 doi:10.1186/1472-6882-1-12 |
|
