Email updates

Keep up to date with the latest news and content from BMC Medical Research Methodology and BioMed Central.

Open Access Research article

Power estimation of tests in log-linear non-uniform association models for ordinal agreement

Fabien Valet1* and Jean-Yves Mary2

Author Affiliations

1 Institut Curie, Ecole des Mines de Paris, INSERM U900, Paris, FRANCE

2 INSERM U717, Hôpital Saint-Louis, Paris, FRANCE

For all author emails, please log on.

BMC Medical Research Methodology 2011, 11:70  doi:10.1186/1471-2288-11-70

Published: 17 May 2011

Abstract

Background

Log-linear association models have been extensively used to investigate the pattern of agreement between ordinal ratings. In 2007, log-linear non-uniform association models were introduced to estimate, from a cross-classification of two independent raters using an ordinal scale, varying degrees of distinguishability between distant and adjacent categories of the scale.

Methods

In this paper, a simple method based on simulations was proposed to estimate the power of non-uniform association models to detect heterogeneities across distinguishabilities between adjacent categories of an ordinal scale, illustrating some possible scale defects.

Results

Different scenarios of distinguishability patterns were investigated, as well as different scenarios of marginal heterogeneity within rater. For sample size of N = 50, the probabilities of detecting heterogeneities within the tables are lower than .80, whatever the number of categories. In additition, even for large samples, marginal heterogeneities within raters led to a decrease in power estimates.

Conclusion

This paper provided some issues about how many objects had to be classified by two independent observers (or by the same observer at two different times) to be able to detect a given scale structure defect. Our results also highlighted the importance of marginal homogeneity within raters, to ensure optimal power when using non-uniform association models.