Table 1

Summary of the HLA-DR benchmark results.

AUC


Allele

SMM

Gibbs

TEPITOPE

SVRMHC

MHCpred

ARB

SMM-PRF

NetMHCII

N


A

0.730

0.697

0.758

0.749

0.756

14

B

0.740

0.705

0.736

0.762

0.748

0.750

11

C

0.710

0.690

0.714

0.688

0.719

0.719

0.722

5

D

0.737

0.710

0.723

0.606

0.717

0.749

0.754

3


Pearson correlation


Allele

SMM

Gibbs

TEPITOPE

SVRMHC

MHCpred

ARB

SMM-PRF

NetMHCII

N


A

0.420

0.368

0.464

0.436

0.448

14

C

0.408

0.369

0.157

0.431

0.428

0.435

5

D

0.458

0.384

0.218

0.425

0.480

0.487

3


Spearman's rank correlation


Allele

SMM

Gibbs

TEPITOPE

SVRHMM

MHCpred

ARB

SMM-PRF

NetMHCII

N


A

0.430

0.372

0.464

0.445

0.453

14

B

0.443

0.378

0.428

0.479

0.458

0.463

11

C

0.398

0.353

0.430

0.377

0.424

0.422

0.427

5

D

0.450

0.365

0.434

0.210

0.407

0.474

0.481

3


The predictive performance is shown in terms of the average area under the ROC curve (upper panel), the average Pearson's correlation (middle panel), and the average Spearman's rank correlation (lower panel) for the SMM (SMM-align), Gibbs sampler [4], TEPITOPE [3], SVRMHC [7], MHCpred [15], and ARB [12] methods, respectively. The SMM-PRF method refers to the extended SMM align method including penalties for longer peptides and short amino terminal peptide flanking residues, and the NetMHCII method refers to the extended SMM align method including direct encoding of peptide flanking residues and penalties for longer peptides and short amino terminal peptide flanking residues. The last column gives the number of alleles included in each average. In A is shown the average performance for all 14 HLA-DR alleles, in B the average performance for the subset of 11 alleles covered by the TEPITOPE method, in C the average performance for the five alleles covered by the SVRMHC method, and in D the average performance for the three alleles covered by the MHCpred method. For each allele, the performance of the SMM-align, Gibbs sampler, and NetMHCII methods was estimated using five-fold cross-validation as described in Methods.

Nielsen et al. BMC Bioinformatics 2007 8:238   doi:10.1186/1471-2105-8-238

Open Data