To provide a valid comparison with our previous system, the same dataset as in 16 was used for evaluation of the new system. The dataset was extracted from the Nuclear Protein Database (NPD) 26 using a Perl script. The NPD is a curated database that stores information on more than 2000 vertebrate proteins, chiefly from human and mouse, which are reported in the literature to be localized in the cell nucleus. Since certain proteins are associated with more than one compartment, a test dataset consisting of proteins with multiple localizations was extracted. These proteins have the same SwissProt or Entrez Protein accession numbers although localized in different compartments. This preparative procedure resulted in 92 proteins that are localized within the six compartments described below. The majority is localized in 2 compartments and the remaining portion is localized in 3 or 4 compartments. After excluding the multi-localization proteins, a non-redundant dataset was further constructed by PROSET 27 to ensure low sequence identity (<50%). In order to have sufficient number of proteins for training and testing, only six localizations were selected for evaluation. These are PML BODY (38), Nuclear Lamina (55), Nuclear Splicing Speckles (56), Chromatin (61), Nucleoplasm (75), and Nucleolus (219). Each of these proteins has a single localization and the total number is 504. The 92 multi-localization proteins are not included in the set of 504 single-localization proteins for the leave-one-out cross-validation (LOOCV). Therefore, the multi-localization dataset is an independent testing set. The summary of the datasets is presented in Table 1.

Given a test protein with GO annotations, the similarity scores between this protein and all the other proteins in the training set are calculated from the similarity scores of GO term pairs (see Methods). The protein with the highest similarity score is designated as the nearest neighbor of the testing protein and its class label will be assigned to the test protein. If multiple proteins in various localizations attain the same highest score or the test protein does not have GO annotation, then the test protein will be assigned as "unpredicted". The unpredicted proteins will be passed on to the SVM module, which uses sequence information 16, for a full coverage of prediction.

Since the numbers of proteins for the six localizations are unbalanced, the Matthew's correlation coefficient (MCC) was employed for the optimization of parameters and evaluation of performance 28. The overall accuracy for the multi-class classification proposed by Rost 29 was also used for the evaluation of our system. Definitions of the MCC and overall accuracy are detailed in Methods section.

Three different similarity measures for GO term pairs were compared: (1) Lord's method 20, (2) SimLP as described in Bioconductor 22, and (3) Exact Match. For Lord's method, the GO term frequencies were extracted based on UniProtKB/Swiss-Prot 30. For a GO term pair, Exact Match defines the similarity score as 1 if the two GO terms are identical, 0 otherwise. SUM_Match was utilized to compute the similarity score between two proteins from similarity scores of GO term pairs. It takes the sum of similarity sores for all matched GO terms from two proteins. Note that the SUM_Match score is equivalent to the inner product of two GO term vectors if Exact Match is used for GO term similarity (see Methods for details). As shown in Table 2, no significant difference in performance can be observed for these three similarity measures of GO term pairs. Surprisingly, the Exact Match method, which does not utilize any DAG structure of GO, achieved competitive performance in comparison with the other two methods.

Very few studies have focused on exploring similarity definition of proteins based on GO terms. Two simple ways are usually employed in defining the similarity between two proteins annotated by GO terms. One is to take the maximum value from the similarity scores of GO term pairs. The other is to take average over all the similarity scores of GO term pairs. However, the above two methods produced poor results especially when the proteins were annotated by many GO terms for the prediction of protein subnuclear localization. Consequently, an extensive investigation on various similarity definitions obtained from similarity scores of GO terms was warranted. As shown in Table 3, similarity definition has profound impact on the quality of prediction. The overall accuracy ranges from 27.0% to 66.5% and overall MCC ranges from 0.141 to 0.519 for proteins with single-location. It seems that the use of the sum of similarity scores over the matched GO term pairs for two proteins as the similarity definition produces the best predictive outcome for this prediction task.

Lord et al. 20 reported a problem that many GO term pairs have identical similarity values. This problem stems from two sources: (1) proteins are represented by relatively small number of GO terms; (2) the similarity measure considers only the information content p_ms (probability of the minimum subsumer) of shared parents of the query terms, meaning that the semantic distances of many different GO term pairs are identical. In order to alleviate this problem, GO terms of homologs retrieved by BLAST were used for the representation of a query protein. The parameter E-value in BLAST is crucial for the quality of homologs, as well as the number of candidate homologs. If E-value is too large, then homologs of low quality may be retrieved. On the other hand, if E-value is too small, then the number of candidate homologs retrieved becomes small. We tested the following E-value parameters: 10⁰, 10^-1, 10^-2, ..., 10^-10, 10^-15, 10^-20, 10^-30, 10^-50, 10^-100, 10^-200, and found that E-value = 10^-9 was a good trade-off value. Even with this threshold the BLAST could retrieve different numbers of hits for different query proteins. We found that up to 5 homologs were suitable to represent the query protein (see Table 4).

As demonstrated before, the predictive outcome is greatly influenced by the ways of combining similarity scores of GO term pairs to give the similarity between two proteins. With the appropriate similarity definition, the performance of the current system can be significantly better than that of the previous SVM system. As seen in Table 5, the overall MCC (accuracy) is elevated from 0.284 to 0.519 (50.0% to 66.5%) for single-localization proteins in the leave-one-out cross-validation; and from 0.420 to 0.541 (65.2%, no change in accuracy) for an independent set of multi-localization proteins. More specifically, 401 (281 true predictions and 120 false predictions) out of 504 proteins were predicted by the GO module in the LOOCV, and the remaining 103 were passed on to the SVM module. For the independent test set of proteins with multi-localizations, 82 (55 true predictions and 27 false predictions) out of 92 proteins were predicted by the GO module, and the remaining 10 were passed on to the SVM module.

It also should be noted that our system currently is designed to predict only one localization. In fact, the results shown for the proteins with multiple localizations is somewhat overestimated, as the prediction is considered correct if any one of localizations of a protein is correctly predicted.

Given a protein sequence, we first BLASTed it against the Swiss-Prot database with a threshold E-value = 10^-9. We selected up to 5 homologs, and submitted the Swiss-Prot accession numbers of the homologs to the QuickGO server 31 for the retrieval of predicted GO terms. The retrieved GO terms were used to represent the given protein.

First we define the depth for each GO term as follows.

Depth(g_i) = the distance of the longest path from GO term g_i to the root of Gene_Ontology, i.e., GO:0003673.

Fig. 1 shows an example of some GO depths, e.g. Depth(GO:0001838) = 7.

The similarity of two GO terms g₁ and g₂can be defined as the depth of their most recent common ancestor (MRCA):

S i m _ G O ( g 1 , g 2 ) = max g c ∈ P ( g 1 , g 2 ) { D e p t h ( g c ) } .       ( 1 ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGtbWucqWGPbqAcqWGTbqBcqGGFbWxcqWGhbWrcqWGpbWtcqGGOaakcqWGNbWzdaWgaaWcbaGaemymaedabeaakiabcYcaSiabdEgaNnaaBaaaleaacqWGYaGmaeqaaOGaeiykaKIaeyypa0ZaaCbeaeaacqqGTbqBcqqGHbqycqqG4baEaSqaaiabbEgaNnaaBaaameaacqqGJbWyaeqaaSGaeyicI4SaemiuaaLaeiikaGIaem4zaC2aaSbaaWqaaiabdgdaXaqabaWccqGGSaalcqWGNbWzdaWgaaadbaGaemOmaidabeaaliabcMcaPaqabaGccqGG7bWEcqWGebarcqWGLbqzcqWGWbaCcqWG0baDcqWGObaAcqGGOaakcqWGNbWzdaWgaaWcbaGaem4yamgabeaakiabcMcaPiabc2ha9jabc6caUiaaxMaacaWLjaWaaeWaaeaacqaIXaqmaiaawIcacaGLPaaaaaa@60E0@

where P(g₁, g₂) is the set of ancestral GO terms shared by both g₁ and g₂ including themselves. When g₁ = g₂, Depth(g_c) = Depth(g₁) = Depth(g₂). For two GO terms from different ontologies (MF, BP, CC), their MRCA is the root GO:0003673, whose depth is zero. That means that there is no similarity between two GO terms from different ontologies.

The GO term similarity described here is the same as the method simLP implemented by Gentleman 22 in Bioconductor.

Consider two proteins that are represented respectively by the sets of GO terms G₁ and G₂. The similarity Sim_Pro between the two proteins can be defined as a function of Sim_GO.

For example, consider protein A (Entrez protein accession number: CAC84554) and protein B (SwissProt accession number:P46055), annotated by 3 GO terms (GO:0005488; GO:0005515; GO:0006412) and 4 GO terms (GO:0005737; GO:0006412; GO:0006415; GO:0016149), respectively. The simLP score for each GO term pair is listed in Table 7. The following 8 functions of combining similarity scores of GO term pairs were examined in this work:

(a) MAX: take the maximum similarity score from the similarity scores of all pairs of GO terms. Sim_Pro = 7.

(b) AVG: take the average similarity score over all pairs of GO terms. Sim_Pro = (7+7+2+2)/12 = 1.5.

(c) SUM: take the sum over all pairs of GO terms, Sim_Pro = 7+7+2+2 = 18.

(d) MAX_Match: same as (a), except that only the matched GO term pairs are considered, e.g. GO:0006412. Sim_Pro = 7.

(e) AVG_Match: same as (b), except that only the matched GO term pairs are considered, e.g. GO:0006412. Sim_Pro = 7/1 = 7.

(f) SUM_Match: same as (c), except that only the matched GO term pairs are considered, e.g. GO:0006412. Sim_Pro = 7.

(g) AVG_BestPairs: Average similarity between the best paired GO terms calculated with the following pseudo codes:

NumofBestPairs ← min {|G₁|, |G₂|}

Sim_Pro ← 0

While (|G₁|>0 and |G₂|>0)

Max_sim_GO ← max{Sim_GO(g_i, g_j)}, g_i ∈ G₁, g_j ∈ G₂

Sim_Pro ← Sim_Pro + Max_sim_GO

Delete g_i from G₁, and g_j from G₂

End while

Sim_Pro ← Sim_Pro/NumofBestPairs

Sim_Pro = (7+2+0)/3 = 3.

(h) SUM_BestPairs: same as (g), except that we do not divide Sim_Pro by NumofBestPairs, i.e., remove the last line in the pseudo codes in (g). Sim_Pro = 7+2+0 = 9.

In this work, the similarity Sim_Pro of two proteins employed in the final system is based on function (f) SUM_Match:

S i m _ P R O ( p 1 , p 2 ) = ∑ g i = g j S i m _ G O ( g i , g j ) , g i ∈ G 1 , g j ∈ G 2       ( 2 ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaafaqabeqacaaabaGaem4uamLaemyAaKMaemyBa0Maei4xa8LaemiuaaLaemOuaiLaem4ta8KaeiikaGIaemiCaa3aaSbaaSqaaiabdgdaXaqabaGccqGGSaalcqWGWbaCdaWgaaWcbaGaemOmaidabeaakiabcMcaPiabg2da9maaqafabaGaem4uamLaemyAaKMaemyBa0Maei4xa8Laem4raCKaem4ta8KaeiikaGIaem4zaC2aaSbaaSqaaiabdMgaPbqabaGccqGGSaalcqWGNbWzdaWgaaWcbaGaemOAaOgabeaakiabcMcaPaWcbaGaem4zaC2aaSbaaWqaaiabdMgaPbqabaWccqGH9aqpcqWGNbWzdaWgaaadbaGaemOAaOgabeaaaSqab0GaeyyeIuoakiabbYcaSaqaaiabdEgaNnaaBaaaleaacqWGPbqAaeqaaOGaeyicI4Saem4raC0aaSbaaSqaaiabdgdaXaqabaGccqWGSaalcqWGGaaicqWGGaaicqWGNbWzdaWgaaWcbaGaemOAaOgabeaakiabgIGiolabdEeahnaaBaaaleaacqWGYaGmaeqaaaaakiaaxMaacaWLjaWaaeWaaeaacqaIYaGmaiaawIcacaGLPaaaaaa@6B52@

where Sim_GO is defined in (1). Alternatively, if Sim_GO is defined as a constant 1, the Sim_Pro is exactly the Inner Product of two GO vectors (see below).

The Inner Product of two GO term vectors has been used in previous study for the prediction of protein subcellular localization 925. A vector with a length equal to the number of all appeared GO terms is prepared for a given protein. An entry is assigned a value 1 if the corresponding GO term is used for the annotation of the protein, 0 otherwise. Then each protein is represented by a binary vector. The similarity between two proteins is defined as the inner product of the two corresponding GO term vectors. Alternatively, Inner Product is the same as the total number of the matched GO terms from the annotation lists of the two proteins.

Our system includes a K-Nearest Neighbor (KNN) model. The best result was achieved with K = 1. A protein is assigned with a localization label of its nearest neighbor that has the highest similarity score Sim_Pro. If the protein does not have associated GO terms or has multiple nearest neighbors in various classes, then the second SVM module built on sequence information 16 will be called to give a prediction.

In our previous work 16, we built an SVM system for prediction of protein subnuclear localizations based solely on protein sequence information. New SVM kernel functions were introduced for the measure of sequence similarity. The k-peptide vectors are first mapped by a matrix of high-scored pairs of k-peptides which are measured by BLOSUM62 scores. The kernels, measuring the similarity for sequences, are then defined on the mapped vectors. By combining these new encoding methods, a multi-class classification system for the prediction of protein subnuclear localizations was established.

Since the numbers of proteins for various localizations are unbalanced, the Matthew's correlation coefficient (MCC) was employed for the optimization of parameters and evaluation of performance 28:

M C C n = p n s n − u n o n ( p n + u n ) ( p n + o n ) ( s n + u n ) ( s n + o n ) , MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGnbqtcqWGdbWqcqWGdbWqdaWgaaWcbaGaemOBa4gabeaakiabg2da9maalaaabaGaemiCaa3aaSbaaSqaaiabd6gaUbqabaGccqWGZbWCdaWgaaWcbaGaemOBa4gabeaakiabgkHiTiabdwha1naaBaaaleaacqWGUbGBaeqaaOGaem4Ba82aaSbaaSqaaiabd6gaUbqabaaakeaadaGcaaqaaiabcIcaOiabdchaWnaaBaaaleaacqWGUbGBaeqaaOGaey4kaSIaemyDau3aaSbaaSqaaiabd6gaUbqabaGccqGGPaqkcqGGOaakcqWGWbaCdaWgaaWcbaGaemOBa4gabeaakiabgUcaRiabd+gaVnaaBaaaleaacqWGUbGBaeqaaOGaeiykaKIaeiikaGIaem4Cam3aaSbaaSqaaiabd6gaUbqabaGccqGHRaWkcqWG1bqDdaWgaaWcbaGaemOBa4gabeaakiabcMcaPiabcIcaOiabdohaZnaaBaaaleaacqWGUbGBaeqaaOGaey4kaSIaem4Ba82aaSbaaSqaaiabd6gaUbqabaGccqGGPaqkaSqabaaaaOGaeiilaWcaaa@633A@

where p_n is the number of correctly predicted proteins of the location n, s_n is the number of correctly predicted proteins not in the location n, u_n is the number of under-predicted proteins, and o_n the number of over-predicted proteins.

Also, the overall accuracy for the multi-class classification proposed by Rost 29 was used for the evaluation of our system. Suppose there are m = m₁ + m₂ + … + m_N test proteins, where m_n is the number of proteins belonging to class n(n = 1,...,N). Suppose further that out of the proteins considered, p_n proteins are predicted to belong to class n. Then p = p₁ + p₂ + … + p_N is the number of correctly predicted proteins. The accuracy for class n is

a c c n = p n m n , MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGHbqycqWGJbWycqWGJbWydaWgaaWcbaGaemOBa4gabeaakiabg2da9maalaaabaGaemiCaa3aaSbaaSqaaiabd6gaUbqabaaakeaacqWGTbqBdaWgaaWcbaGaemOBa4gabeaaaaGccqGGSaalaaa@3A28@

and the overall accuracy, denoted by Q_acc, is defined as

Q a c c = ∑ n = 1 N a c c n × m n m = ∑ n = 1 N p n m = p m . MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacqWGrbqudaWgaaWcbaGaemyyaeMaem4yamMaem4yamgabeaakiabg2da9maaqahabaGaemyyaeMaem4yamMaem4yam2aaSbaaSqaaiabd6gaUbqabaGccqGHxdaTdaWcaaqaaiabd2gaTnaaBaaaleaacqWGUbGBaeqaaaGcbaGaemyBa0gaaaWcbaGaemOBa4Maeyypa0JaeGymaedabaGaemOta4eaniabggHiLdGccqGH9aqpdaaeWbqaamaalaaabaGaemiCaa3aaSbaaSqaaiabd6gaUbqabaaakeaacqWGTbqBaaGaeyypa0ZaaSaaaeaacqWGWbaCaeaacqWGTbqBaaaaleaacqWGUbGBcqGH9aqpcqaIXaqmaeaacqWGobGta0GaeyyeIuoakiabc6caUaaa@56E3@