Research article
A proposed method to investigate reliability throughout a questionnaire
-
* Corresponding author: Are H Pripp are.hugo.pripp@oslo-universitetssykehus.no
1 Unit of Biostatistics and Epidemiology, Oslo University Hospital, Kirkeveien 116, 0450 Oslo, Norway
2 Centre for Child and Adolescent Mental Health, Eastern and Southern Norway, Oslo, Norway
3 Norwegian Centre for Violence and Traumatic Stress Studies, Oslo, Norway
4 Centre for Clinical Research, Haukeland University Hospital, Bergen, Norway
5 Department of Heart Disease, Haukeland University Hospital, Bergen, Norway
6 Faculty of Health and Social Sciences, Bergen University College, Bergen, Norway
7 Institute of Medicine, University of Bergen, Bergen, Norway
BMC Medical Research Methodology 2011, 11:137 doi:10.1186/1471-2288-11-137
Published: 5 October 2011Abstract
Background
Questionnaires are used extensively in medical and health care research and depend on validity and reliability. However, participants may differ in interest and awareness throughout long questionnaires, which can affect reliability of their answers. A method is proposed for "screening" of systematic change in random error, which could assess changed reliability of answers.
Methods
A simulation study was conducted to explore whether systematic change in reliability, expressed as changed random error, could be assessed using unsupervised classification of subjects by cluster analysis (CA) and estimation of intraclass correlation coefficient (ICC). The method was also applied on a clinical dataset from 753 cardiac patients using the Jalowiec Coping Scale.
Results
The simulation study showed a relationship between the systematic change in random error throughout a questionnaire and the slope between the estimated ICC for subjects classified by CA and successive items in a questionnaire. This slope was proposed as an awareness measure - to assessing if respondents provide only a random answer or one based on a substantial cognitive effort. Scales from different factor structures of Jalowiec Coping Scale had different effect on this awareness measure.
Conclusions
Even though assumptions in the simulation study might be limited compared to real datasets, the approach is promising for assessing systematic change in reliability throughout long questionnaires. Results from a clinical dataset indicated that the awareness measure differed between scales.