Table 3

Univariate and multivariate multilevel logistic regression to predict correct detection of mode effects defined by Robust Z and Bayesian 95% credible interval as a function of study variables
Model/Predictor Univariate Multivariate
OR AUCa OR 95% CI
Robust Z (Model AUC = 0.95)
Size of DIF 1.49** 0.55 3.42** (2.58-4.54)
Percentage of DIF 1.17 0.52 1.20 (0.89,1.61)
2PL IRT Modelb 0.76** 0.53 0.47** (0.35,0.64)
Diff. Mean θ = 1.0 0.99 0.50 0.66** (0.50,0.87)
CAT Item Usagec 21133.86** 0.94 3111.68** (1417.85,6829.03)
Absolute Item Difficultyd 0.03** 0.85 0.10** (0.07,0.14)
Item Discriminationd 3.62** 0.60 3.12** (2.34,4.17)
Bayesian 95% Credible Interval (Model AUC = 0.93)
Size of DIF 1.73** 0.56 3.52** (2.73,4.53)
Percentage of DIF 1.17 0.52 1.16 (0.89,1.50)
2PL IRT Modelb 0.74** 0.53 0.50** (0.39,0.65)
Diff. Mean θ = 1.0 0.91 0.49 0.60** (0.47,0.77)
CAT Item Usagec 2468.29** 0.92 505.64** (264.29,967.37)
Absolute Item Difficultyd 0.04** 0.83 0.15** (0.11,0.20)
Item Discriminationd 2.86** 0.58 1.99** (1.54,2.56)

Correct Detection of Mode Effects = true positive detection of mode DIF among items simulated with mode DIF; AUC = area under the ROC curve; CI = 95% confidence interval; IRT = item response theory model used to generate response data and parameters used in CAT; CAT item usage = number of times a given item was administered by CAT divided by 100; * p < .05; ** p < .01.

Riley and Carle

Riley and Carle BMC Medical Research Methodology 2012 12:124   doi:10.1186/1471-2288-12-124

Open Data