An antibody epitope, aka B-cell epitope or antigenic determinant, is a part of an antigen recognized by either a particular antibody molecule or a particular B-cell receptor of the immune system 1. For a protein antigen, an epitope may be either a short peptide from the protein sequence, called a continuous epitope, or a patch of atoms on the protein surface, called a discontinuous epitope. While continuous epitopes can be directly used for the design of vaccines and immunodiagnostics, the objective of discontinuous epitope prediction is to design a molecule that can mimic the structure and immunogenic properties of an epitope and replace it either in the process of antibody production–in this case an epitope mimic can be considered as a prophylactic or therapeutic vaccine–or antibody detection in medical diagnostics or experimental research 23.

If continuous epitopes can be predicted using sequence-dependent methods built on available collections of immunogenic peptides (for review see 4), discontinuous epitopes–that are mostly the case when a whole protein, pathogenic virus, or bacteria is recognized by the immune system–are difficult to predict or identify from functional assays without knowledge of a three-dimensional (3D) structure of a protein 56. The first attempts at epitope prediction based on 3D protein structure began in 1984 when a correlation was established between crystallographic temperature factors and several known continuous epitopes of tobacco mosaic virus protein, myoglobin and lysozyme 7. A correlation between antigenicity, solvent accessibility, and flexibility of antigenic regions in proteins was also found 8. Thornton and colleagues 9 proposed a method for identifying continuous epitopes in the protein regions protruding from the protein's globular surface. Regions with high protrusion index values were shown to correspond to the experimentally determined continuous epitopes in myoglobin, lysozyme and myohaemerythrin 9.

Here we present ElliPro (derived from

Elli

Pro

For evaluation of ElliPro performance and comparison with other methods we used a previously established benchmark approach for discontinuous epitopes 11. We tested ElliPro on a dataset of 39 epitopes present in 39 protein structures where only one discontinuous epitope was known based on 3D structures of two-chain antibody fragments with one-chain protein antigens 11.

Depending on the threshold values for parameters R and S, ElliPro predicted different number of epitopes in each protein; for an R of 6Å and S of 0.5, the average number of predicted epitopes in each protein analyzed was 4, with a variance from 2 to 8. For example, for Plasmodium vivax ookinete surface protein Pvs25 [PDB: 1Z3G, chain A], ElliPro predicted four epitopes with scores of 0.763, 0.701, 0.645, and 0.508, respectively (Fig. 2).

For each predicted epitope in each protein, we calculated the correctly (TP) and incorrectly predicted epitope residues (FN) and non-epitope residues, which were defined as all other protein residues (TN and FN). The statistical significance of a prediction, that is, the difference between observed and expected frequencies of an actual epitope/non-epitope residue in the predicted epitope/non-epitope, was determined using Fisher's exact test (right-tailed). The prediction was considered significant if the P-value was = 0.05. Then, for each prediction the following parameters were calculated:

Sensitivity (recall or true positive rate (TPR)) = TP/(TP + FN) – a proportion of correctly predicted epitope residues (TP) with respect to the total number of epitope residues (TP+FN).

Specificity (or 1 – false positive rate (FPR)) = 1 - FP/(TN + FP) – a proportion of correctly predicted non-epitope residues (TN) with respect to the total number of non-epitope residues (TN+FP).

Positive predictive value (PPV) (precision) = TP/(TP + FP) – a proportion of correctly predicted epitope residues (TP) with respect to the total number of predicted epitope residues (TP+FN).

Accuracy (ACC) = (TP + TN)/(TP + FN + FP + TN) – a proportion of correctly predicted epitope and non-epitope residues with respect to all residues.

Area under the ROC Curve (AUC) – area under a graph representing a dependency of TPR against FPR; that is, sensitivity against 1-specificity. The AUC gives the general performance of the method and is "equivalent to the probability that the classifier will rank a randomly chosen positive instance higher than a randomly chosen negative instance" 22.

For example, for the first predicted epitope in Plasmodium vivax ookinete surface protein Pvs25 [PDB:1Z3G, chain A] (Fig. 2), for an R of 6Å and S of 0.5, TP = 13, FP = 13, TN = 156, FN = 4, P-value = 5.55E-10, giving a sensitivity of 0.76, a specificity of 0.92, an accuracy of 0.91, and an AUC of 0.84. The results and detailed statistics of ElliPro performance for each epitope and other threshold values for R and S are provided in the supplementary materials [see Additional file 1].

The statistics averaged over all epitopes and overall statistics calculated from FP, FN, TP, and TN values summarized for the whole pool of epitope and non-epitope residues are presented in Table 1 and Fig. 3. The results for the methods other than ElliPro have been obtained as described in 11. ElliPro performed best, by AUC values, with the score S set at 0.7 and the distance R set at 6Å when the prediction with the highest score was considered for each protein and with the score S set at 0.5 and the distance R set at 6Å when the best by significance or average prediction was taken into account. Results are described using these thresholds (Table 1, Fig. 3); the results at other threshold values are provided in the supplementary materials [see Additional file 1].

ElliPro's top predictions, that are those with the highest scores, correlated poorly with the discontinuous epitopes known from 3D structures of antibody-protein complexes (Table 1, overall statistics, AUC = 0.523). DiscoTope and the first models from the docking methods performed better, giving AUC values above 0.6, whereas protein-protein binding site predicting methods, ProMate and PPI-PRED, performed worse. However, when the first predictions with the highest score were considered, ElliPro was the best among all the methods based on specificity (1-specificity = 0.047) and comparable with DiscoTope by precision (PPV = 0.158) (Table 1, overall statistics).

In a next set of metrics, we compared the performance between prediction methods when choosing the best hit within the top 10 predictions of each method. This approach takes into account that each antigen harbors multiple distinct binding sites for different antibodies. Therefore it is expected that the top predicted site is not necessarily recognized by the specific antibody used in the dataset. This comparison directly applies only to the docking methods DOT and PatchDock as well as ElliPro. For DiscoTope, only one epitope is predicted, while for CEP no ranking is available to identify the top 10 predictions.

The docking methods DOT and PatchDock have an intrinsic advantage in this comparison over ElliPro, because they use structures of both protein antigen and antibody from the same antibody-protein complex in order to predict binding sites. To our surprise, when the best significant prediction was considered for each protein, ElliPro nevertheless gave the highest AUC value of 0.732, the highest sensitivity of 0.601 and the second highest precision value of 0.29 among all the compared methods (Table 1; Fig. 3, red circle). The docking methods gave the AUC values of 0.693 for DOT and 0.656 for PatchDock, when also the best prediction of the top ten was considered (Table 1, overall statistics; Fig. 3). The average number of predicted epitopes for the analyzed proteins was four, with the rank of the best prediction at most fifth; for more than a half of proteins the rank was first or second, and the rank first, second, or third for more than 70% of all proteins [see Additional file 1].

ElliPro is based on simple concepts. First, regions protruding from the globular surface of the protein are more available for interaction with an antibody 9 and second those protrusions can be determined by treating the protein as a simple ellipsoid 20. Obviously, this is not always the case, especially for multi-domain or large single-domain proteins. However, no correlation between the protein size, which varied from 51 to 429 residues with an average value of 171, or number of domains (8 proteins among the 39 analyzed contained more than one domain) and ElliPro performance was found (data not shown).

ElliPro is a web-based tool for the prediction of antibody epitopes in protein antigens of a given sequence or structure. It implements a previously developed method that represents the protein structure as an ellipsoid and calculates protrusion indexes for protein residues outside of the ellipsoid. ElliPro was tested on a benchmark dataset of discontinuous epitopes inferred from 3D structures of antibody-protein complexes. In comparison with six other structure-based methods that can be used for epitope prediction, ElliPro performed the best (AUC value of 0.732) when the most significant prediction was considered for each protein. Since the rank of the best prediction was at most three in more than 70% of proteins and never exceeded five, ElliPro is considered a potentially useful research tool for identifying antibody epitopes in protein antigens.

While ElliPro was tested on antibody-protein binding sites, it might be interesting to test it on other protein-protein interactions since it implements a method that is based on geometrical properties of protein structure and does not require training.

Comparison with DiscoTope, which is based on training and utilizes epitope features such as amino acid propensities, residue solvent accessibility, spatial distribution, and inter-molecular contacts, suggests that further research on antibody epitopes which considers more features that discriminate epitopes from non-epitopes may improve the prediction of antibody epitopes.

• Project name: ElliPro

• Project home page: http://tools.immuneepitope.org/tools/ElliPro

• Operating system(s): Platform independent

• Programming language: Java

• Other requirements: None

• License: None

• Any restrictions to use by non-academics: None

PI: protrusion index; TP: true positives; FP: false positives; TN: true negatives, FN: false negatives; ROC: Receiver Operating Characteristics; AUC: area under the ROC curve.

HHB conceived, designed and programmed the tool. JVP tested the tool and wrote the manuscript. WL and NF participated in programming the tool. PEB, BP and AS contributed to writing the manuscript. All authors have read and approved the final version of the manuscript.