Table 4

Univariate and multivariate multilevel logistic regression to predict incorrect detection of mode effects defined by Robust Z and Bayesian 95% credible interval as a function of study variables
Model/Predictor Univariate Multivariate
OR AUCa OR 95% CI
Robust Z (Model AUC = 0.77)
Size of DIF 1.93** 0.55 2.01** (1.36,2.97)
Percentage of DIF 1.44 0.55 1.48 (0.99,2.20)
2PL IRT Model 1.14 0.52 0.96 (0.63,1.46)
Diff. Mean θ = 1.0 3.31** 0.59 3.95** (2.56,6.08)
CAT Item Usage 1.91** 0.54 4.17** (3.11,5.60)
Item Difficulty 0.28** 0.62 0.12** (0.08,0.19)
Item Discrimination 1.64** 0.56 1.23 (0.96,1.58)
Bayesian 95% Credible Interval (Model AUC = 0.74)
Size of DIF 1.62* 0.55 1.61** (1.20,2.15)
Percentage of DIF 1.33 0.53 1.30 (0.97,1.75)
2PL IRT Model 1.14 0.52 1.08 (0.80,1.47)
Diff. Mean θ = 1.0 1.28E+08** 0.62 4.01** (2.90,5.55)
CAT Item Usage 0.96 0.44 2.36** (1.82,3.06)
Item Difficulty 0.30** 0.65 0.16** (0.11,0.22)
Item Discrimination 1.19 0.51 1.02 (0.82,1.26)

Incorrect detection of mode effects = False positive identification of DIF due to mode among items not simulated with mode DIF; AUC = area under the ROC curve; CI = 95% confidence interval; IRT = item response theory model used to generate response data and parameters used in CAT; CAT item usage = number of times a given item was administered by CAT divided by 100; * p < .05; ** p < .01.

Riley and Carle

Riley and Carle BMC Medical Research Methodology 2012 12:124   doi:10.1186/1471-2288-12-124

Open Data