Before proposing the new method, we briefly describe stem kernels which have been proposed as an extension of the string kernels for measuring the similarities between two RNA sequences from the viewpoint of secondary structures 20. The feature space of the stem kernels is defined by enumerating all possible common base pairs and stem structures of arbitrary lengths. The stem kernel calculates the inner product of common stem structure counts. In other words, the more stem structures two RNA sequences have in common, the more similar they are. However, the time needed for the explicit enumeration of all substructures obviously grows exponentially, which renders this method infeasible for long sequences. We have therefore developed an algorithm for calculating stem kernels which is based on the dynamic programming technique. For an RNA sequence x = x₁x₂ ... x_n (x_k ∈ {A, C, G, U}), we denote a contiguous subsequence x_j ... x_k by x [j, k], and the length of x by |x|. The empty sequence is indicated by ∊. For a base a, the complementary base is denoted as a¯ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmyyaeMbaebaaaa@2D38@. For a string x and a base a, xa denotes the concatenation of x and a. For two RNA sequences x and x', the stem kernel K is defined recursively as follows:

K ( ϵ , x ′ ) = K ( x , ϵ ) = 1 , for  ∀ x , x ′ , K ( x a , x ′ ) = K ( x , x ′ ) + ∑ x k = a ¯ ∑ i < j  s . t .   x ′ i = a ¯ , x ′ j = a K ( x [ k + 1 , | x | ] , x ′ [ i + 1 , j − 1 ] ) . MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeaabiqaaaqaauaabeqabiaaaeaacqWGlbWscqGGOaaktqvzynutnfgDOLeDHXwAJbqegmwBTLwmWaaceiGae8x9diRaeiilaWIafCiEaGNbauaacqGGPaqkcqGH9aqpcqWGlbWscqGGOaakcqWH4baEcqGGSaalcqWF1pGScqGGPaqkcqGH9aqpcqaIXaqmcqGGSaalaeaacqqGMbGzcqqGVbWBcqqGYbGCcqqGGaaicqGHaiIicqWH4baEcqGGSaalcuWH4baEgaqbaaaacqGGSaalaeaacqWGlbWscqGGOaakcqWH4baEcqWGHbqycqGGSaalcuWH4baEgaqbaiabcMcaPiabg2da9iabdUealjabcIcaOiabhIha4jabcYcaSiqbhIha4zaafaGaeiykaKIaey4kaSYaaabuaeaadaaeqbqaaiabdUealjabcIcaOiabhIha4jabcUfaBjabdUgaRjabgUcaRiabigdaXiabcYcaSiabcYha8jabhIha4jabcYha8jabc2faDjabcYcaSiqbhIha4zaafaGaei4waSLaemyAaKMaey4kaSIaeGymaeJaeiilaWIaemOAaOMaeyOeI0IaeGymaeJaeiyxa0LaeiykaKIaeiOla4caleaacqWGPbqAcqGH8aapcqWGQbGAcqqGGaaicqqGZbWCcqGGUaGlcqqG0baDcqGGUaGlcqqGGaaicuWG4baEgaqbamaaBaaameaacqWGPbqAaeqaaSGaeyypa0JafmyyaeMbaebacqGGSaalcuWG4baEgaqbamaaBaaameaacqWGQbGAaeqaaSGaeyypa0JaemyyaegabeqdcqGHris5aaWcbaGaemiEaG3aaSbaaWqaaiabdUgaRbqabaWccqGH9aqpcuWGHbqygaqeaaqab0GaeyyeIuoaaaaaaa@9DA8@

Both the time and the memory required for the calculation K(x, x') are of the order of O(|x|²|x'|²), which renders this method impractical for applying to large data sets of ncRNAs.

Here, we develop a new technique based on directed acyclic graphs (DAGs) derived from base-pairing probability matrices of RNA sequences, which significantly reduces the time needed for computing stem kernels. Figure 1 contains a diagram illustrating the calculation of the new kernels.

First, for each RNA sequence x = x₁x₂ ... x_n, we calculate a base-pairing probability matrix P^x using the McCaskill algorithm 21. We denote the base-pairing probability of (x_i, x_j) by Pijx MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemiuaa1aa0baaSqaaiabdMgaPjabdQgaQbqaaiabhIha4baaaaa@3160@, which is defined as:

P i j x = E [ I i j | x ] = ∑ y ∈ Y ( x ) p ( y | x ) I i j ( y ) , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemiuaa1aa0baaSqaaiabdMgaPjabdQgaQbqaaiabhIha4baakiabg2da9mrr1ngBPrwtHrhAYaqeguuDJXwAKbstHrhAGq1DVbaceaGae8hHWxKaei4waSLaemysaK0aaSbaaSqaaiabdMgaPjabdQgaQbqabaGccqGG8baFcqWH4baEcqGGDbqxcqGH9aqpdaaeqbqaaiabdchaWjabcIcaOiabdMha5jabcYha8jabhIha4jabcMcaPiabdMeajnaaBaaaleaacqWGPbqAcqWGQbGAaeqaaOGaeiikaGIaemyEaKNaeiykaKcaleaacqWG5bqEcqGHiiIZt0uy0HwzTfgDPnwy1egarCqtHrhAL1wy0L2yHvdaiuaacqGFyeFwcqGGOaakcqWH4baEcqGGPaqkaeqaniabggHiLdGccqGGSaalaaa@6CD3@

where Y MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae8hgXNfaaa@3779@(x) is an ensemble of all possible secondary structures of x, p(y|x) is the posterior probability of y given x, and I_ij(y) is an indicator function, which equals 1 if the i-th and the j-th nucleotides form a base-pair in y or 0 otherwise. We employ the Vienna RNA package 22 for computing these expected counts (2) using the McCaskill algorithm.

Subsequently, we build a DAG for the base-pairing probability matrix, where each vertex corresponds to a base pair whose probability is above a predefined threshold p*. Let G_x = (V_x, E_x) be the DAG for an RNA sequence x, where V_x and E_x are vertices and edges in the DAG G_x, respectively. For each v_i = (k, l) ∈ V_x, (x_k, x_l) is a likely base pair, in other words, Pklx≥p∗ MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemiuaa1aa0baaSqaaiabdUgaRjabdYgaSbqaaiabhIha4baakiabgwMiZkabdchaWnaaCaaaleqabaGaey4fIOcaaaaa@35BD@. Each e_ij ∈ E_x is an edge from vertex v_i to vertex v_j.

For vertices v_i = (k, l) and v_i' = (k', l'), we can define a partial order, v_i ≺ v_i' if and only if k <k' and l > l'. An edge e_ii' connects vertices v_i and v_i' if and only if v_i ≺ v_i' and there exists no v_j ∈ V_x such that v_i ≺ v_j ≺ v_i'.

Finally, we calculate a kernel value between two DAGs representing RNA structure information through the DAG kernel using a dynamic programming technique. The vertices in the DAG can be numbered in a topological order such that for every edge e_ij, i <j is satisfied, in other words, there are no directed paths from v_j to v_i if i <j. Thus, we can apply the dynamic programming technique as follows:

K ( G x , G x ′ ) = ∑ v i ∈ r o o t ( G x ) , v i ′ ∈ r o o t ( G x ′ ) r ( i , i ′ ) r ( i , i ′ ) = { K v ( v i , v i ′ ) + g v ( v i ) + g v ( v i ′ ) ( ∄ j , j ′  s . t .   j > i , j ′ > i ′ ) K v ( v i , v i ′ ) + g v ( v i ) ∑ j > i g e ( e i j ) r ( j , i ′ ) + g v ( v i ′ ) ( ∄ j ′  s . t .   j ′ > i ′ ) K v ( v i , v i ′ ) + g v ( v i ) + g v ( v i ′ ) ∑ j ′ > i ′ g e ( e i ′ , j ′ ) r ( i , j ′ ) ( ∄ j  s . t .   j > i ) K v ( v i , v i ′ ) ∑ j > i , j ′ > i ′ K e ( e i j , e i ′ j ′ ) r ( j , j ′ ) + g v ( v i ) ∑ j > i g e ( e i j ) r ( j , i ′ ) + g v ( v i ′ ) ∑ j ′ > i ′ g e ( e i ′ j ′ ) r ( i , j ′ ) ( otherwise ) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGceiqabeaaOiqaaiabdUealjabcIcaOiabdEeahnaaBaaaleaacqWH4baEaeqaaOGaeiilaWIaem4raC0aaSbaaSqaaiqbhIha4zaafaaabeaakiabcMcaPiabg2da9maaqafabaGaemOCaiNaeiikaGIaemyAaKMaeiilaWIafmyAaKMbauaacqGGPaqkaSqaaiabdAha2naaBaaameaacqWGPbqAaeqaaSGaeyicI4SaemOCaiNaem4Ba8Maem4Ba8MaemiDaqNaeiikaGIaem4raC0aaSbaaWqaaiabhIha4bqabaWccqGGPaqkcqGGSaalcqWG2bGDdaWgaaadbaGafmyAaKMbauaaaeqaaSGaeyicI4SaemOCaiNaem4Ba8Maem4Ba8MaemiDaqNaeiikaGIaem4raC0aaSbaaWqaaiqbhIha4zaafaaabeaaliabcMcaPaqab0GaeyyeIuoaaOqaaiaaxMaacqWGYbGCcqGGOaakcqWGPbqAcqGGSaalcuWGPbqAgaqbaiabcMcaPiabg2da9maaceaabaqbaeaabuGaaaaabaGaem4saS0aaSbaaSqaaiabdAha2bqabaGccqGGOaakcqWG2bGDdaWgaaWcbaGaemyAaKgabeaakiabcYcaSiabdAha2naaBaaaleaacuWGPbqAgaqbaaqabaGccqGGPaqkcqGHRaWkcqWGNbWzdaWgaaWcbaGaemODayhabeaakiabcIcaOiabdAha2naaBaaaleaacqWGPbqAaeqaaOGaeiykaKIaey4kaSIaem4zaC2aaSbaaSqaaiabdAha2bqabaGccqGGOaakcqWG2bGDdaWgaaWcbaGafmyAaKMbauaaaeqaaOGaeiykaKcabaGaeiikaGYenfgDOvwBHrxAJf2maeHbnfgDOvwBHrxAJf2maGabaiab=rGiXkabdQgaQjabcYcaSiqbdQgaQzaafaGaeeiiaaIaee4CamNaeiOla4IaeeiDaqNaeiOla4IaeeiiaaIaemOAaOMaeyOpa4JaemyAaKMaeiilaWIafmOAaOMbauaacqGH+aGpcuWGPbqAgaqbaiabcMcaPaqaaiabdUealnaaBaaaleaacqWG2bGDaeqaaOGaeiikaGIaemODay3aaSbaaSqaaiabdMgaPbqabaGccqGGSaalcqWG2bGDdaWgaaWcbaGafmyAaKMbauaaaeqaaOGaeiykaKIaey4kaSIaem4zaC2aaSbaaSqaaiabdAha2bqabaGccqGGOaakcqWG2bGDdaWgaaWcbaGaemyAaKgabeaakiabcMcaPmaaqafabaGaem4zaC2aaSbaaSqaaiabdwgaLbqabaGccqGGOaakcqWGLbqzdaWgaaWcbaGaemyAaKMaemOAaOgabeaakiabcMcaPiabdkhaYjabcIcaOiabdQgaQjabcYcaSiqbdMgaPzaafaGaeiykaKIaey4kaSIaem4zaC2aaSbaaSqaaiabdAha2bqabaGccqGGOaakcqWG2bGDdaWgaaWcbaGafmyAaKMbauaaaeqaaOGaeiykaKcaleaacqWGQbGAcqGH+aGpcqWGPbqAaeqaniabggHiLdaakeaacqGGOaakcqWFeisScuWGQbGAgaqbaiabbccaGiabbohaZjabc6caUiabbsha0jabc6caUiabbccaGiqbdQgaQzaafaGaeyOpa4JafmyAaKMbauaacqGGPaqkaeaacqWGlbWsdaWgaaWcbaGaemODayhabeaakiabcIcaOiabdAha2naaBaaaleaacqWGPbqAaeqaaOGaeiilaWIaemODay3aaSbaaSqaaiqbdMgaPzaafaaabeaakiabcMcaPiabgUcaRiabdEgaNnaaBaaaleaacqWG2bGDaeqaaOGaeiikaGIaemODay3aaSbaaSqaaiabdMgaPbqabaGccqGGPaqkcqGHRaWkcqWGNbWzdaWgaaWcbaGaemODayhabeaakiabcIcaOiabdAha2naaBaaaleaacuWGPbqAgaqbaaqabaGccqGGPaqkdaaeqbqaaiabdEgaNnaaBaaaleaacqWGLbqzaeqaaOGaeiikaGIaemyzau2aaSbaaSqaaiqbdMgaPzaafaGaeiilaWIafmOAaOMbauaaaeqaaOGaeiykaKIaemOCaiNaeiikaGIaemyAaKMaeiilaWIafmOAaOMbauaacqGGPaqkaSqaaiqbdQgaQzaafaGaeyOpa4JafmyAaKMbauaaaeqaniabggHiLdaakeaacqGGOaakcqWFeisScqWGQbGAcqqGGaaicqqGZbWCcqGGUaGlcqqG0baDcqGGUaGlcqqGGaaicqWGQbGAcqGH+aGpcqWGPbqAcqGGPaqkaeaacqWGlbWsdaWgaaWcbaGaemODayhabeaakiabcIcaOiabdAha2naaBaaaleaacqWGPbqAaeqaaOGaeiilaWIaemODay3aaSbaaSqaaiqbdMgaPzaafaaabeaakiabcMcaPmaaqafabaGaem4saS0aaSbaaSqaaiabdwgaLbqabaGccqGGOaakcqWGLbqzdaWgaaWcbaGaemyAaKMaemOAaOgabeaakiabcYcaSiabdwgaLnaaBaaaleaacuWGPbqAgaqbaiqbdQgaQzaafaaabeaakiabcMcaPiabdkhaYjabcIcaOiabdQgaQjabcYcaSiqbdQgaQzaafaGaeiykaKcaleaacqWGQbGAcqGH+aGpcqWGPbqAcqGGSaalcuWGQbGAgaqbaiabg6da+iqbdMgaPzaafaaabeqdcqGHris5aaGcbaaabiqaaOpacaWLjaGaey4kaSIaem4zaC2aaSbaaSqaaiabdAha2bqabaGccqGGOaakcqWG2bGDdaWgaaWcbaGaemyAaKgabeaakiabcMcaPmaaqafabaGaem4zaC2aaSbaaSqaaiabdwgaLbqabaGccqGGOaakcqWGLbqzdaWgaaWcbaGaemyAaKMaemOAaOgabeaakiabcMcaPiabdkhaYjabcIcaOiabdQgaQjabcYcaSiqbdMgaPzaafaGaeiykaKIaey4kaSIaem4zaC2aaSbaaSqaaiabdAha2bqabaGccqGGOaakcqWG2bGDdaWgaaWcbaGafmyAaKMbauaaaeqaaOGaeiykaKYaaabuaeaacqWGNbWzdaWgaaWcbaGaemyzaugabeaakiabcIcaOiabdwgaLnaaBaaaleaacuWGPbqAgaqbaiqbdQgaQzaafaaabeaakiabcMcaPiabdkhaYjabcIcaOiabdMgaPjabcYcaSiqbdQgaQzaafaGaeiykaKcaleaacuWGQbGAgaqbaiabg6da+iqbdMgaPzaafaaabeqdcqGHris5aaWcbaGaemOAaOMaeyOpa4JaemyAaKgabeqdcqGHris5aaGcbaGaeiikaGIaee4Ba8MaeeiDaqNaeeiAaGMaeeyzauMaeeOCaiNaee4DaCNaeeyAaKMaee4CamNaeeyzauMaeiykaKcaaaGaay5Eaaaaaaa@9C53@

where root(G) is a set of vertices which have no incoming edges, K_v and K_e are kernel functions for vertices and edges, respectively, and g_v and g_e are gap penalties for vertices and edges, respectively. K calculates the sum of kernel values for all pairs of possible substructures of G_x and G_x'. Each of these kernel values is composed of the product of the subkernels K_v, K_e, g_v and g_e. Therefore, K is a convolution kernel and is positive semi-definite if K_v and K_e are also positive semi-definite 23.

The time and the memory required for the computation of K are of the order of O(c²|V_x||V_x'|) and O(|V_x||V_x'|), respectively, where c is the maximum out-degree of G_x and G_x'. We can control |V_x| using the predefined threshold for base pairs, p*. When p* = 0, V_x contains all possible base pairs, i.e., |V_x| = n(n - 1)/2. When p* > 0, since each base can take part in V_x at most 1/p* times, |V_x| is proportional to n of the length of the RNA sequence x. Since in many cases c ≪ |V_x|, the time and the memory required for this algorithm are approximately of the order of O(n²) for sufficiently large values of p*.

Several choices of sub-kernels K_v, K_e, g_v and g_e in Eq. (3) are available. In order to connect the DAG-based stem kernels to the naive stem kernels calculated from Eq. (1), we first define simple sub-kernels as follows:

K v ( v , v ′ ) = { 1 ( x ¯ k = x l  and  ( x k , x l ) = ( x ′ k ′ , x ′ l ′ ) for  v = ( k , l ) ∈ V x  and  v ′ = ( k ′ , l ′ ) ∈ V x ′ ) 0 ( otherwise ) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4saS0aaSbaaSqaaiabdAha2bqabaGccqGGOaakcqWG2bGDcqGGSaalcuWG2bGDgaqbaiabcMcaPiabg2da9maaceaabaqbaeaabiGaaaqaaiabigdaXaqaamaabmaabaqbaeaabiqaaaqaaiqbdIha4zaaraWaaSbaaSqaaiabdUgaRbqabaGccqGH9aqpcqWG4baEdaWgaaWcbaGaemiBaWgabeaakiabbccaGiabbggaHjabb6gaUjabbsgaKjabbccaGiabcIcaOiabdIha4naaBaaaleaacqWGRbWAaeqaaOGaeiilaWIaemiEaG3aaSbaaSqaaiabdYgaSbqabaGccqGGPaqkcqGH9aqpcqGGOaakcuWG4baEgaqbamaaBaaaleaacuWGRbWAgaqbaaqabaGccqGGSaalcuWG4baEgaqbamaaBaaaleaacuWGSbaBgaqbaaqabaGccqGGPaqkaeaacqqGMbGzcqqGVbWBcqqGYbGCcqqGGaaicqWG2bGDcqGH9aqpcqGGOaakcqWGRbWAcqGGSaalcqWGSbaBcqGGPaqkcqGHiiIZcqWGwbGvdaWgaaWcbaGaeCiEaGhabeaakiabbccaGiabbggaHjabb6gaUjabbsgaKjabbccaGiqbdAha2zaafaGaeyypa0JaeiikaGIafm4AaSMbauaacqGGSaalcuWGSbaBgaqbaiabcMcaPiabgIGiolabdAfawnaaBaaaleaacuWH4baEgaqbaaqabaaaaaGccaGLOaGaayzkaaaabaGaeGimaadabaGaeiikaGIaee4Ba8MaeeiDaqNaeeiAaGMaeeyzauMaeeOCaiNaee4DaCNaeeyAaKMaee4CamNaeeyzauMaeiykaKcaaaGaay5Eaaaaaa@8AAC@

K e ( e , e ′ ) = { 1 ( e ∈ E x  and  e ′ ∈ E x ′ ) 0 ( otherwise ) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4saS0aaSbaaSqaaiabdwgaLbqabaGccqGGOaakcqWGLbqzcqGGSaalcuWGLbqzgaqbaiabcMcaPiabg2da9maaceaabaqbaeaabiGaaaqaaiabigdaXaqaaiabcIcaOiabdwgaLjabgIGiolabdweafnaaBaaaleaacqWH4baEaeqaaOGaeeiiaaIaeeyyaeMaeeOBa4MaeeizaqMaeeiiaaIafmyzauMbauaacqGHiiIZcqWGfbqrdaWgaaWcbaGafCiEaGNbauaaaeqaaOGaeiykaKcabaGaeGimaadabaGaeiikaGIaee4Ba8MaeeiDaqNaeeiAaGMaeeyzauMaeeOCaiNaee4DaCNaeeyAaKMaee4CamNaeeyzauMaeiykaKcaaaGaay5Eaaaaaa@58CE@

g_v(v) = 1,   ∀v ∈ V_x ∪ V_x'

g_e(e) = 1,   ∀e ∈ E_x ∪ E_x'.

When p* → 0, the DAG-based stem kernels calculated form Eq. (3) with the above sub-kernels approach the naive stem kernels calculated from Eq. (1) since both Eqs. (1) and (3) designate recursive traversal to all substructures of x and x' in the sense of the partial order ≺, and when p* = 0, the substructures of x and x' for both kernels which contribute kernel values are identical to each other due to these sub-kernels. More sophisticated kernels can be constructed using substitution scoring matrices, as well as local alignment kernels 24:

K v ( v , v ′ ) = exp ⁡ ( P k l x P k ′ l ′ x ′ ⋅ α ⋅ S ( x k , x l , x ′ k ′ , x ′ l ′ ) )       ( for  v = ( k , l ) ∈ V x  and  v ′ = ( k ′ , l ′ ) ∈ V x ′ ) , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGceiqabiaayvaa0AqaaiabdUealnaaBaaaleaacqWG2bGDaeqaaOGaeiikaGIaemODayNaeiilaWIafmODayNbauaacqGGPaqkcqGH9aqpcyGGLbqzcqGG4baEcqGGWbaCdaqadaqaaiabdcfaqnaaDaaaleaacqWGRbWAcqWGSbaBaeaacqWH4baEaaGccqWGqbaudaqhaaWcbaGafm4AaSMbauaacuWGSbaBgaqbaaqaaiqbhIha4zaafaaaaOGaeyyXICTaeqySdeMaeyyXICTaem4uamLaeiikaGIaemiEaG3aaSbaaSqaaiabdUgaRbqabaGccqGGSaalcqWG4baEdaWgaaWcbaGaemiBaWgabeaakiabcYcaSiqbdIha4zaafaWaaSbaaSqaaiqbdUgaRzaafaaabeaakiabcYcaSiqbdIha4zaafaWaaSbaaSqaaiqbdYgaSzaafaaabeaakiabcMcaPaGaayjkaiaawMcaaaqaaiaaxMaacaWLjaGaeiikaGIaeeOzayMaee4Ba8MaeeOCaiNaeeiiaaIaemODayNaeyypa0JaeiikaGIaem4AaSMaeiilaWIaemiBaWMaeiykaKIaeyicI4SaemOvay1aaSbaaSqaaiabhIha4bqabaGccqqGGaaicqqGHbqycqqGUbGBcqqGKbazcqqGGaaicuWG2bGDgaqbaiabg2da9iabcIcaOiqbdUgaRzaafaGaeiilaWIafmiBaWMbauaacqGGPaqkcqGHiiIZcqWGwbGvdaWgaaWcbaGafCiEaGNbauaaaeqaaOGaeiykaKIaeiilaWcaaaa@86C3@

K e ( e , e ′ ) = { γ n ( e ) + n ( e ′ ) ( e ∈ E x  and  e ′ ∈ E x ′ ) 0 ( otherwise ) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4saS0aaSbaaSqaaiabdwgaLbqabaGccqGGOaakcqWGLbqzcqGGSaalcuWGLbqzgaqbaiabcMcaPiabg2da9maaceaabaqbaeaabiGaaaqaaiabeo7aNnaaCaaaleqabaGaemOBa4MaeiikaGIaemyzauMaeiykaKIaey4kaSIaemOBa4MaeiikaGIafmyzauMbauaacqGGPaqkaaaakeaacqGGOaakcqWGLbqzcqGHiiIZcqWGfbqrdaWgaaWcbaGaeCiEaGhabeaakiabbccaGiabbggaHjabb6gaUjabbsgaKjabbccaGiqbdwgaLzaafaGaeyicI4Saemyrau0aaSbaaSqaaiqbhIha4zaafaaabeaakiabcMcaPaqaaiabicdaWaqaaiabcIcaOiabb+gaVjabbsha0jabbIgaOjabbwgaLjabbkhaYjabbEha3jabbMgaPjabbohaZjabbwgaLjabcMcaPaaaaiaawUhaaaaa@637E@

g_v(v) = γ²,   ∀v ∈ V_x ∪ V_x'

g_e(e) = γ^n(e),   ∀e ∈ E_x ∪ E_x',

where S(xl,xk,x′k′,x′l′) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4uamLaeiikaGIaemiEaG3aaSbaaSqaaiabdYgaSbqabaGccqGGSaalcqWG4baEdaWgaaWcbaGaem4AaSgabeaakiabcYcaSiqbdIha4zaafaWaaSbaaSqaaiqbdUgaRzaafaaabeaakiabcYcaSiqbdIha4zaafaWaaSbaaSqaaiqbdYgaSzaafaaabeaakiabcMcaPaaa@3DC2@ is a substitution scoring function from a base pair (x_l, x_k) to a base pair (x′k′,x′l′) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeiikaGIafmiEaGNbauaadaWgaaWcbaGafm4AaSMbauaaaeqaaOGaeiilaWIafmiEaGNbauaadaWgaaWcbaGafmiBaWMbauaaaeqaaOGaeiykaKcaaa@34B5@, α > 0 is a weight parameter for base pairs, γ > 0 is the decoy factor for loop regions, and n(e) is the number of nucleotides in the loop region enclosed by base pairs at both ends of an edge e.

In our experiments, we employed the RIBOSUM 80-65 9 for S, and p* = 0.01, α = 0.1, γ = 0.4, which were optimized by cross-validation tests. In order to prevent sequence length bias, we normalize our kernels K as follows:

K ′ ( G x , G x ′ ) = K ( G x , G x ′ ) K ( G x , G x ) K ( G x ′ , G x ′ ) . MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafm4saSKbauaacqGGOaakcqWGhbWrdaWgaaWcbaGaeCiEaGhabeaakiabcYcaSiabdEeahnaaBaaaleaacuWH4baEgaqbaaqabaGccqGGPaqkcqGH9aqpjuaGdaWcaaqaaiabdUealjabcIcaOiabdEeahnaaBaaabaGaeCiEaGhabeaacqGGSaalcqWGhbWrdaWgaaqaaiqbhIha4zaafaaabeaacqGGPaqkaeaadaGcaaqaaiabdUealjabcIcaOiabdEeahnaaBaaabaGaeCiEaGhabeaacqGGSaalcqWGhbWrdaWgaaqaaiabhIha4bqabaGaeiykaKIaem4saSKaeiikaGIaem4raC0aaSbaaeaacuWH4baEgaqbaaqabaGaeiilaWIaem4raC0aaSbaaeaacuWH4baEgaqbaaqabaGaeiykaKcabeaaaaGaeiOla4caaa@538D@

Stem kernels can be applied only to RNA secondary structures. However, primary sequences are still important for calculating the similarities between a pair of RNA sequences. Therefore, in order to take into account both primary sequences and secondary structures, we combine our stem kernels with the local alignment kernels by adding them.

If multiple alignments of homologous RNA sequences are available, we can calculate their base-paring probability matrices more precisely by taking the averaged sum of individual base-pairing probability matrices in accordance with the given multiple alignment 25. The algorithm of the DAG-based stem kernels for a pair of RNA sequences can be extended to that for a pair of multiple alignments of RNA sequences using averaged base-pairing probability matrices. Since the method of the averaged base-paring probability matrices has been proven to be accurate and robust by Kiryu et al. 25, we can expect this method to improve the proposed stem kernel method. We call these profile-profile stem kernels.

We denote the i-th column of a multiple alignment A by A_i, a nucleotide in A_i of the j-th sequence by a_ij, and the number of aligned sequences in A by num(A). We can calculate the averaged base-pairing probability matrix of a given multiple alignment A as follows:

P k l A = 1 n u m ( A ) ∑ x ∈ A P ′ k l x , P ′ k l x = { P ρ ( k ) ρ ( l ) x ′ ( for either of  x k  and  x l  are not gaps ) 0 ( otherwise ) , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGceaqabeaacqWGqbaudaqhaaWcbaGaem4AaSMaemiBaWgabaGaeCyqaeeaaOGaeyypa0tcfa4aaSaaaeaacqaIXaqmaeaacqWGUbGBcqWG1bqDcqWGTbqBcqGGOaakcqWHbbqqcqGGPaqkaaGcdaaeqbqaaiqbdcfaqzaafaWaa0baaSqaaiabdUgaRjabdYgaSbqaaiabhIha4baaaeaacqWH4baEcqGHiiIZcqWHbbqqaeqaniabggHiLdGccqGGSaalaeaacuWGqbaugaqbamaaDaaaleaacqWGRbWAcqWGSbaBaeaacqWH4baEaaGccqGH9aqpdaGabaqaauaabaqaciaaaeaacqWGqbaudaqhaaWcbaGaeqyWdiNaeiikaGIaem4AaSMaeiykaKIaeqyWdiNaeiikaGIaemiBaWMaeiykaKcabaGafCiEaGNbauaaaaaakeaacqGGOaakcqqGMbGzcqqGVbWBcqqGYbGCcqqGGaaicqqGLbqzcqqGPbqAcqqG0baDcqqGObaAcqqGLbqzcqqGYbGCcqqGGaaicqqGVbWBcqqGMbGzcqqGGaaicqWG4baEdaWgaaWcbaGaem4AaSgabeaakiabbccaGiabbggaHjabb6gaUjabbsgaKjabbccaGiabdIha4naaBaaaleaacqWGSbaBaeqaaOGaeeiiaaIaeeyyaeMaeeOCaiNaeeyzauMaeeiiaaIaeeOBa4Maee4Ba8MaeeiDaqNaeeiiaaIaee4zaCMaeeyyaeMaeeiCaaNaee4CamNaeiykaKcabaGaeGimaadabaGaeiikaGIaee4Ba8MaeeiDaqNaeeiAaGMaeeyzauMaeeOCaiNaee4DaCNaeeyAaKMaee4CamNaeeyzauMaeiykaKIaeiilaWcaaaGaay5Eaaaaaaa@9AB4@

where x' is the sequence x with all gaps removed and ρ(k) is an index on x' of the k-th column of A. After constructing PklA MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemiuaa1aa0baaSqaaiabdUgaRjabdYgaSbqaaiabhgeabbaaaaa@30FA@, we can build DAGs, and the kernel K_v for columns can be calculated by replacing the substitution function S in Eq. (9) with

S ( A k , A l , A ′ k ′ , A ′ l ′ ) = 1 n u m ( A ) n u m ( A ′ ) ∑ i = 1 n u m ( A ) ∑ i ′ = 1 n u m ( A ′ ) S ′ ( a k i , a l i , a ′ k ′ i ′ , a ′ l ′ i ′ ) S ′ ( a k i , a l i , a ′ k ′ i ′ , a ′ l ′ i ′ ) = { S ( a k i , a l i , a ′ k ′ i ′ , a ′ l ′ i ′ ) ( any of  a k i , a l i , a ′ k ′ i ′ ,  and  a ′ l ′ i ′  are not gaps ) 0 ( otherewise ) . MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGceaqabeaacqWGtbWucqGGOaakcqWHbbqqdaWgaaWcbaGaem4AaSgabeaakiabcYcaSiabhgeabnaaBaaaleaacqWGSbaBaeqaaOGaeiilaWIafCyqaeKbauaadaWgaaWcbaGafm4AaSMbauaaaeqaaOGaeiilaWIafCyqaeKbauaadaWgaaWcbaGafmiBaWMbauaaaeqaaOGaeiykaKIaeyypa0tcfa4aaSaaaeaacqaIXaqmaeaacqWGUbGBcqWG1bqDcqWGTbqBcqGGOaakcqWHbbqqcqGGPaqkcqWGUbGBcqWG1bqDcqWGTbqBcqGGOaakcuWHbbqqgaqbaiabcMcaPaaakmaaqahabaWaaabCaeaacuWGtbWugaqbaiabcIcaOiabdggaHnaaBaaaleaacqWGRbWAcqWGPbqAaeqaaOGaeiilaWIaemyyae2aaSbaaSqaaiabdYgaSjabdMgaPbqabaGccqGGSaalcuWGHbqygaqbamaaBaaaleaacuWGRbWAgaqbaiqbdMgaPzaafaaabeaakiabcYcaSiqbdggaHzaafaWaaSbaaSqaaiqbdYgaSzaafaGafmyAaKMbauaaaeqaaOGaeiykaKcaleaacuWGPbqAgaqbaiabg2da9iabigdaXaqaaiabd6gaUjabdwha1jabd2gaTjabcIcaOiqbhgeabzaafaGaeiykaKcaniabggHiLdaaleaacqWGPbqAcqGH9aqpcqaIXaqmaeaacqWGUbGBcqWG1bqDcqWGTbqBcqGGOaakcqWHbbqqcqGGPaqka0GaeyyeIuoaaOqaaiqbdofatzaafaGaeiikaGIaemyyae2aaSbaaSqaaiabdUgaRjabdMgaPbqabaGccqGGSaalcqWGHbqydaWgaaWcbaGaemiBaWMaemyAaKgabeaakiabcYcaSiqbdggaHzaafaWaaSbaaSqaaiqbdUgaRzaafaGafmyAaKMbauaaaeqaaOGaeiilaWIafmyyaeMbauaadaWgaaWcbaGafmiBaWMbauaacuWGPbqAgaqbaaqabaGccqGGPaqkcqGH9aqpdaGabaqaauaabaqaciaaaeaacqWGtbWucqGGOaakcqWGHbqydaWgaaWcbaGaem4AaSMaemyAaKgabeaakiabcYcaSiabdggaHnaaBaaaleaacqWGSbaBcqWGPbqAaeqaaOGaeiilaWIafmyyaeMbauaadaWgaaWcbaGafm4AaSMbauaacuWGPbqAgaqbaaqabaGccqGGSaalcuWGHbqygaqbamaaBaaaleaacuWGSbaBgaqbaiqbdMgaPzaafaaabeaakiabcMcaPaqaaiabcIcaOiabbggaHjabb6gaUjabbMha5jabbccaGiabb+gaVjabbAgaMjabbccaGiabdggaHnaaBaaaleaacqWGRbWAcqWGPbqAaeqaaOGaeiilaWIaemyyae2aaSbaaSqaaiabdYgaSjabdMgaPbqabaGccqGGSaalcuWGHbqygaqbamaaBaaaleaacuWGRbWAgaqbaiqbdMgaPzaafaaabeaakiabcYcaSiabbccaGiabbggaHjabb6gaUjabbsgaKjabbccaGiqbdggaHzaafaWaaSbaaSqaaiqbdYgaSzaafaGafmyAaKMbauaaaeqaaOGaeeiiaaIaeeyyaeMaeeOCaiNaeeyzauMaeeiiaaIaeeOBa4Maee4Ba8MaeeiDaqNaeeiiaaIaee4zaCMaeeyyaeMaeeiCaaNaee4CamNaeiykaKcabaGaeGimaadabaGaeiikaGIaee4Ba8MaeeiDaqNaeeiAaGMaeeyzauMaeeOCaiNaeeyzauMaee4DaCNaeeyAaKMaee4CamNaeeyzauMaeiykaKIaeiOla4caaaGaay5Eaaaaaaa@F18C@

We performed several experiments in which SVMs based on our kernel attempted to detect known ncRNA families. The accuracy was assessed using the specificity (SP) and the sensitivity (SN), which are defined as follows:

S P = T N T N + F P , S N = T P T P + F N , MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeqabeGaaaqaaiabdofatjabdcfaqjabg2da9KqbaoaalaaabaGaemivaqLaemOta4eabaGaemivaqLaemOta4Kaey4kaSIaemOrayKaemiuaafaaOGaeiilaWcabaGaem4uamLaemOta4Kaeyypa0tcfa4aaSaaaeaacqWGubavcqWGqbauaeaacqWGubavcqWGqbaucqGHRaWkcqWGgbGrcqWGobGtaaGccqGGSaalaaaaaa@4594@

where TP is the number of correctly predicted positives, FP is the number of incorrectly predicted positives, TN is the number of correctly predicted negatives, and FN is the number of incorrectly predicted negatives. Furthermore, the area under the receiver operating characteristic (ROC) curve, i.e., the ROC score, was also used for evaluation. The ROC curve plots the true positive rates (= SN) as a function of the false positive rates (= 1 - SP) for varying decision thresholds of a classifier.

In our first experiment, the discrimination ability and the execution time of the stem kernels were tested on our previous dataset used in 20, which includes five RNA families: tRNAs, miRNAs (precursor), 5S rRNAs, H/ACA snoRNAs, and C/D snoRNAs. We chose 100 sequences in each RNA family from the Rfam database 26 as positive samples such that the pairwise identity was not above 80% for any pair of sequences, and 100 randomly shuffled sequences with the same dinucleotide composition as the positives were generated as negative samples for each family. The discrimination performance was evaluated using 10-fold cross validation. In order to determine an appropriate cutoff threshold for the base-pairing probabilities p*, we performed the experiments for various values of p* ∈ {0.1, 0.01, 0.001, 0.0001}. Figure 2 shows the accuracy and the calculation time for each threshold. Since the accuracy for p* = 0.01 was slightly better than that for the other values, and the calculation time in this case was acceptable for practical use, we fixed p* = 0.01 as the default cutoff threshold of the base-pairing probabilities. Then, we compared the DAG-based stem kernels with the naive stem kernels. The experimental results shown in Table 1 indicate that the DAG-based kernels are significantly faster than the naive kernels owing to the approximation by a predefined threshold of the base-pairing probability. Furthermore, in spite of using an approximation, the DAG-based kernels are slightly more accurate than the naive kernels due to the convolution with the local alignment kernels and the removal of low-likelihood base pairs which may create noise.

Next, we performed the experiment on a large dataset including multiple alignments, which was used to train RNAz 17. This dataset includes 12 ncRNA families of 7,169 original alignments, extracted from the Rfam database 26, with the exception of the single-recognition particle (SRP) RNA and RNAseP, which were extracted from 2728. Each alignment consists of two to ten sequences aligned by CLUSTAL-W 29, and the mean pairwise identities are between 50% and 100%. The dataset also includes 7,169 negatives, which were generated from the original alignments by shuffling the columns, where the conservation rate on each column was preserved 30. In this experiment, for each RNA family, SVMs trained the model which distinguishes the original alignments of a target RNA family from all other original and shuffled alignments in the dataset. We compared the profile-profile stem kernels with the local alignment kernels 24, which only consider primary sequences of RNAs. Subsequently, we extended the local alignment kernels using the same technique as in the case of the profile-profile stem kernels in order to account for multiple alignments.

The discrimination performance of both kernels was evaluated with 10-fold cross-validation. Table 2 presents the experimental results for this dataset. The stem kernels attained nearly perfect discrimination for all families in this dataset, while the local alignment kernels failed to discriminate some families. The performance with respect to tmRNA and RNAse P in terms of sensitivity was especially low. Furthermore, the stem kernels collected a smaller number of support vectors in comparison with the local alignment kernels due to the robustness of the stem kernels with respect to secondary structures. This is a desirable feature since the prediction process of SVMs requires only support vectors for the calculation of kernel values against an input sequence.

In addition, we employed another kernel machine instead of SVM, called support vector data description (SVDD) 31, which calculates a spherically shaped boundary around a dataset so as to increase the robustness against outliers without the need for negative examples. In other words, SVDD does not need to generate artificial negative examples. Many applications of SVMs to biological problems require the artificial generation of negative examples such as shuffled positive sequences. However, since most artificial negatives can be easily distinguished from positives in many cases, the generation of artificial negative examples is a crucial problem to attaining practical prediction performance 32. In this regard, SVDD can avoid this problem by using only positive examples. We applied SVDD instead of SVMs to the above dataset. Table 3 shows the surprising discovery that there is little difference in the accuracy of SVMs and SVDD. This result indicates that negative examples produced by shuffling the alignments make a very small contribution to learning the classifiers with our kernels. Furthermore, the number of support vectors in SVDD decreased significantly in comparison to SVMs.