KnowPred_site predicts PSL based on a knowledge base, which is constructed to capture local sequence similarity of two proteins even when they have sequence identity less than 25%. However, such local similarity is difficult to be detected using the traditional alignment algorithm due to the low sequence similarity. Therefore we adopt the transitivity relationship, which was firstly used in 27 for clustering protein sequences, to capture local similarity between protein sequences. Transitivity refers to deducing a possible similarity between protein A and protein C from the existence of a third protein B, such that A and B as well as B and C are homologues if the sequence identity between A and B as well as that between B and C is above the predefined threshold. Figure 1(a) shows an example of transitivity relationship among protein A, protein B, and protein C. Protein A and protein B share sequence identity of 34%, and protein B and protein C share sequence identity of 27%, whereas protein A and protein C only share sequence identity of 12%. Using the transitivity relationship, remote homologous relationship and local similarity between protein A and protein C can be detected.

In this paper, we apply the transitivity concept to peptide fragments instead of the protein sequences to obtain local similarities between remotely homologues. Protein A and protein C share local similarity if there is a peptide fragment similar (formal definition of peptide similarity will be discussed in next subsection) to subsequences in protein A and protein C. Figure 1(b) illustrates the idea, in which protein A and C are aligned with protein B1 and protein B2 (B1 and B2 can be identical, homologous or non-homologous). If there is a peptide fragment shared by both B1 and B2, the corresponding peptide fragments in protein A and protein C are inferred as locally similar between protein A and protein C. The shared peptide may represent a possible sequence variation in evolution. Moreover, if protein A and protein C are remotely homologous, there is likely more "shared" sequence fragments in different protein B's to characterize their similarity. However, not all such proteins A and C which share local similarity are homologous. Some local similarities may arise without common ancestry. Short sequences may be similar by chance, and sequences may be similar because both are selected to bind to a particular protein. In order to avoid ambiguity, we define such proteins A and C which share local similarity as "related sequences".

Given a dataset of proteins with known localization sites, we construct a knowledge base, called Similar-Peptide Knowledge Base (or SPKB in short). The dataset used to construct SPKB will be described in the Result section. To construct the knowledge base, we first use the native sequence of each protein in the dataset to extract the fixed-length peptide fragments by using a sliding window of length w. Each peptide sequence as well as its protein source and the localization site information are stored in SPKB. Since the performance of knowledge-based methods relies on the size of the knowledge base, we then perform PSI-BLAST search with parameters j = 3, e = 0.001 on each protein in the dataset against the NCBInr database to find similar sequences. Since the NCBInr database contains only the protein sequence information, the localization annotation of peptides generated by similar sequences is determined as follows. Specifically, given a query protein sequence q, PSI-BLAST would generate a large number of significant local pairwise alignments called high-scoring segment pairs (HSPs) between q and its similar proteins. An example of an HSP is shown in Figure 2. Statistically significant BLAST hits usually signify sequence homology. We assumed that in an HSP, the similar peptide sequences in the counterpart sequence (denoted by "Sbjct") represent the possible sequence variations to the corresponding peptide in the query (denoted by "Query"), i.e., the protein q. We use the same sliding window of length w to generate all peptide fragments in each HSP. Two amino acids aligned together in an HSP are said to be interchangeable if they have a positive score in the BLOSUM62 (an interchangeable residue pair is represented as an amino acid letter or a plus symbol in an HSP). The number of amino acid pairs being interchangeable within a sliding window represents the similarity level of the two peptide fragments. A peptide in Sbjct is called a similar peptide if it has at least k residues interchangeable to those of the corresponding peptide in Query. A similar peptide is used to signify local sequence similarity between Sbjct and Query and thus is assigned the localization annotation of the protein q.

Performing PSI-BLAST search for all proteins in the dataset, we can generate a huge number, possibly multi-millions, of similar peptides with localization annotation. Each record in the knowledge base is indexed by a similar peptide, and stores its similar peptide sequences and protein sources (those that are used as query proteins in the PSI-BLAST searches), similarity level and localization site information (inferred from the corresponding protein sources). Note that a similar peptide may occur multiple times in different HSPs of a single PSI-BLAST search result, i.e., derived from different similar proteins found in the PSI-BLAST search. We cluster them together and store the frequency in the peptide record. Table 1 shows a record of the similar peptide MYSKILL (assuming that the window size is 7), which is generated by performing PSI-BLAST search on the three proteins (A, B, and C) with known localization sites, respectively. The frequencies of MYSKILL in the PSI-BLAST search results of proteins A, B, and C are 21, 12, and 17, respectively. The localization site information is inherited from the three protein sources.

The main idea of KnowPred_site is illustrated in Figure 3. Given a target protein t, whose localization annotation is unknown and to be predicted, we perform PSI-BLAST search and use the same procedure as described in the last subsection for knowledge base construction to generate all similar peptides of t and their frequencies from its native sequence and HSPs. Each similar peptide hp is then matched against SPKB, and the peptide record with index hp is called a hit. For each hit, we calculate two types of scores associated with each localization site i: the voting score s_i and the confidence score CS(i). The calculation of the voting score s_i is as follows: Let f denote the frequency of hp found in all t's HSPs. For each record in SPKB, we calculate the score w_i associated with each localization site by summing up the frequencies of the similar peptides that contain the specific site. For example, for the peptide record MYSKILL shown in Table 1, the score of cytoplasm is 38 (21+17; since protein source A and C are both localized into cytoplasm), and those of nuclear and extracellular are 12 and 17, respectively. Then the voting score s_i is defined as f multiplied by (w_i/total frequencies in that record). For example, if MYSKILL is a similar peptide of t and its frequency is 10 in t's HSPs, then the voting scores of cytoplasm, nuclear, and extracellular are 7.6 (=10 × 38/50), 2.4 (=10 × 12/50), and 3.4 (=10 × 17/50), respectively, while those of other localization sites are all 0.

The localization site prediction of the protein t is determined by the confidence score CS(i), which is the total voting score aggregated from all hit records. Finally, each CS(i) is divided by the summation of all frequencies f of all t's hits and then multiplied by 100 to normalize the confidence score in the range of 0 and 100. KnowPred_site predicts t being localized into the site with the highest confidence score for single-localized proteins or into the sites with the two highest confidence scores for multi-localized proteins (All multi-localized proteins in ngLOC dataset have two localization sites).

To differentiate single-localized proteins from those that are multi-localized, we followed King and Guda's method 23 to calculate the multi-localized confidence score (MLCS) associated with a protein t, which gives a relative measure of the likelihood that the protein t is multi-localized. It is derived from the two highest confidence scores (denoted as CS₁ and CS₂) and is defined as follows:

and MLCS(t) is bounded by 100, i.e., when the calculated MLCS(t) is over 100, it is assigned 100.

Since BLAST is the most popular method for sequence comparison, we implemented a simple prediction method based on the BLAST search result. Given a dataset of proteins with known localization site(s), to predict the localization site(s) of a test protein t we first perform the BLAST search against the dataset and then assign the localization annotations of the best BLAST hit to the protein t. If there is no hit at the e-value cutoff 0.001, no annotation will be assigned to the protein t. As reported by Jones and Swindells, the e-value of 0.001 generally produces a safe searching 28. The performance of BLAST-based prediction method is usually treated as the baseline to compare with those of other methods 29.

The performance is estimated using the following measurements. To assess the performance in each localization site, precision, accuracy and Matthew's correlation coefficient (MCC) are calculated by Equations (1) to (3), respectively. The overall accuracy is defined in Equation (4).

where TP_i, TN_i, FP_i, FN_i, and N_i are, respectively, the number of true positives, true negatives, false positives, false negatives, and proteins in localization site i. MCC, which considers both under- and over-predictions, provides a complementary measure of the predictive performance, where MCC = 1 indicates a perfect prediction, MCC = 0 indicates a completely random assignment, and MCC = -1 indicates a perfectly reverse correlation.

KnowPred_site aims to utilize the localization annotations of similar peptides. The determination of similar relations, which depends on the window size w and the threshold of similarity level k, can affect the performance of KnowPred_site. Using a smaller w, similar peptides have a higher probability to be hit against the knowledge base; however, shorter peptide sequences are likely to appear in many unrelated proteins. Given a fixed w, there is also a trade-off in choosing the threshold of similarity level k. A smaller k produces looser similarity relations, which leads to extracting more, but less reliable, similar peptides. To make an appropriate selection of w and k, we conducted a leave-one-out cross validation experiments on only the single-localized proteins in the ngLOC dataset for w ranging from 3 to 11 and k ranging from 0 to w.

Figure 4 shows the overall accuracies of KnowPred_site using different window size w with fixed similarity threshold (k = 0). It shows that the appropriate window size is 7 or 8. Then we further investigate the performance using different thresholds of similarity levels. Table 2 shows the overall accuracies ranging from 90.9% to 92.0% for all combinations of window sizes (w = 7, 8) and similarity thresholds. According to the experiment results, we chose the combination of w = 7 and k = 6 for the following experiments since they provided the best accuracy 92.0%.

After the best parameters have been determined, we conducted a ten-fold cross validation experiment on the entire dataset to compare KnowPred_site with ngLOC and Blast-hit prediction. We used the top N accuracy for evaluation, where N ranges from 1 to 4. A protein is considered to be correctly predicted when the real localization site(s) rank among the top N of the predicted sites. (Top 1 accuracy is simply the Accuracy defined in Equation (4).) Notably, for multi-localized proteins, the accuracy is measured in two ways: first, at least one site correctly predicted and second, both sites correctly predicted. Using the first measurement, a true positive is a multi-localized protein with at least one localization site correctly predicted; whereas a true positive using the second measurement is a multi-localized protein with both sites correctly predicted.

The prediction performance of KnowPred_site, ngLOC, and Blast-hit is summarized in Table 3, in which KnowPred_site performance is reported with ten-fold cross validation and leave-one-out cross validation as denoted by ^#KnowPred_site and *KnowPred_site, respectively. It is observed that KnowPred_site outperforms ngLOC and Blast-hit. (The prediction results of single- and multi-localized proteins by KnowPred_site can be found in Additional file 2 to Additional file 5. Additional file 2 lists the prediction results for single-localized proteins using leave-one-out cross validation; Additional file 3 lists the prediction results for single-localized proteins using ten-fold cross validation; Additional file 4 lists the prediction results for multi-localized proteins using leave-one-out cross validation; Additional file 5 lists the prediction results for multi-localized proteins using ten-fold cross validation.)

For single-localized proteins, the overall accuracies of KnowPred_site are from 91.7 to 98.1 when the correct prediction is considered within the top 1 to top 4 most probable sites. Those of ngLOC are from 88.8% to 96.3%. The accuracy of Blast-hit is 86.0%, which means 86.0% of single-localized proteins could be correctly predicted by BLAST searches. It is noteworthy that 2114 sequences among all single-localized proteins failed to find significant similar proteins by Blast-hit method; however, 58.8% of them were correctly predicted by KnowPred_site. It shows that the local similarity helps identify related sequences for subcellular localization prediction.

The experiment result shows that KnowPred_site has much higher accuracy on multi-localized proteins than the other methods. Using the first accuracy measurement, i.e., at least one site correctly predicted, KnowPred_site achieves more than 90% of the top 1 accuracy, which is higher than ngLOC by 8.2%. Using the tighter second accuracy measurement, KnowPred_site achieves 72.1% of the top 2 accuracy, which is higher than ngLOC by 12.4%. Further observing the top N accuracy, we find that KnowPred_site is more able to narrow down the number of false positives than ngLOC.

The top 1 and top 2 accuracies of the Blast-hit method are 78.8% and 45.7% for the two accuracy measurements. Notably, 318 proteins among all multi-localized proteins failed to find any significant Blast hit; however, 73.3% and 49.7% of them were correctly predicted by KnowPred_site using the two accuracy measurements, respectively.

In contrast to the overall accuracy of the dataset reported in Table 3, we further analyze the prediction performance on each of the 10 distinct localization sites. The results are summarized in Table 4. Among the 10 localization sites, the precision ranges from 75.7% to 98.5% and the accuracy_i ranges from 52.0% to 96.4%. It is observed that higher occurrence of the localization site, e.g., EXC (29.1%) and PLA (25.2%), leads to better prediction, e.g., the precision and accuracy on EXC are 98.5% and 93.9%, respectively. Low occurrence of the localization site can deteriorate prediction, for example, CSK (1%) and GOL (1.1%) have MCC_i of 0.645 and 0.746, respectively. However, if the similar peptide records of a site have higher specificity, prediction performance can be good despite low occurrence. For example, the precision and accuracy on LYS (0.6%) and POX (0.8%) are 87.2% and 81.9%, and 87.3% and 85.1%, respectively. Furthermore, it is noteworthy that although CYT represents 11.1% of the dataset, its MCC_i is 0.774, much lower than other highly occurring sites. Its low MCC_i is due to low precision since KnowPredsite yields more false positives for CYT. High false positives usually occur when the similar peptide records of a site have lower specificity and higher diversity. As a result, proteins of other localization sites are misclassified as CYT.