Bioinformatics Research Center, Aarhus University, Aarhus, Denmark

Department of Molecular Biology and Genetics, Aarhus University, Aarhus, Denmark

Interdisciplinary Nanoscience Center, Aarhus University, Aarhus, Denmark

CLC bio, Finlandsgade 10-12, Aarhus N, DK-8000, Denmark

Department of Statistics, University of Oxford, OX1 3TG, UK

Oxford Centre for Integrative Systems Biology, University of Oxford, OX1 3QU, UK

Department of Computer Science, Aarhus University, Aarhus, Denmark

Abstract

Comparative methods for RNA secondary structure prediction use evolutionary information from RNA alignments to increase prediction accuracy. The model is often described in terms of stochastic context-free grammars (SCFGs), which generate a probability distribution over secondary structures. It is, however, unclear how this probability distribution changes as a function of the input alignment. As prediction programs typically only return a single secondary structure, better characterisation of the underlying probability space of RNA secondary structures is of great interest. In this work, we show how to efficiently compute the information entropy of the probability distribution over RNA secondary structures produced for RNA alignments by a phylo-SCFG, and implement it for the PPfold model. We also discuss interpretations and applications of this quantity, including how it can clarify reasons for low prediction reliability scores. PPfold and its source code are available from

Background

The function of RNA molecules is known to depend on their three-dimensional structure, which is stabilized by a secondary structure scaffold of basepairing. The secondary structure is defined by hydrogen bonds between nucleotides, which form across the structure for thermodynamic stability and molecular function. Despite its importance, the accurate prediction of RNA secondary structure remains an unsolved challenge in computational biology.

With the advent of next-generation sequencing technologies and new methods in transcriptomics, an explosively growing amount of biological RNA data is available in public databases such as Rfam

Several methods have been established to predict RNA secondary structures from nucleotide sequences. In this paper, we focus entirely on non-pseudoknotted secondary structure prediction. Thermodynamic optimisation based on minimising free-energy functions has been used to great effect in algorithms such as mfold

Common to these methods is that they produce a probability distribution over all possible nested secondary structures for the input sequences, but usually only a single, optimal secondary structure is reported to the user. A particularly interesting question is how the underlying distribution changes as a function of input data. Due to the large space of possible secondary structures, however, it is difficult to report useful quantities to describe this. Information entropy is one such measure.

Entropy computations in the context of RNA secondary structure prediction have been considered previously from a thermodynamic perspective, calculating the thermodynamic entropy over both secondary structure space

The information entropy

The information entropy is a measure for the "spread" of the probability distribution, and has well-defined lower and upper bounds. The minimum entropy of 0 occurs when there is only one outcome with probability 1. For _{2 }(

In the case of RNA secondary structure prediction based on a semantically unambiguous SCFG, the information entropy of the probability distribution over RNA secondary structures can be computed as the derivational entropy of the SCFG that generates the distribution. We restrict ourselves to semantically unambiguous SCFGs, in order to maintain a one-to-one correspondence between SCFG derivations and secondary structures. Thus, throughout this paper we use "information entropy" and "derivational entropy" interchangeably.

Notation

Consider RNA alignments of _{i }^{k }^{k }^{k}^{k}, N, S, R

• Σ^{k }

•

•

• ^{k }

•

In the case of a phylo-grammar, _{π = (A → α) }

Let

where _{d }

The grammar is consistent if ∑_{d }p

The expected frequency (count) of a rule

The expected frequency of each rule can be computed in practice using a dynamic programming algorithm known as the ^{k}^{k }^{k}

with

The _{l }^{3}) time.

The derivational entropy of an SCFG is the information entropy of the probability distribution of all derivations under the SCFG (c.f. equation 1):

where the sum is over all possible derivations of the grammar. This quantity can be computed efficiently using expected rule frequencies

We assume that the phylo-SCFG describes RNA secondary structure, and while it may be

for ^{' }^{k}

Type 1 rules correspond to the production of a single column of the alignment, and their probability can be expressed as

where _{G}

Type 2 rules correspond to the production of two basepaired columns of the alignment, and their probability can be expressed as

where _{G}^{' }

Type 3 rules express bifurcation and correspond to dividing the alignment into two parts. As these rules do not depend on alignment columns, we have:

where _{G}

It is now clear that the probability of any particular derivation under a phylo-SCFG this structure can be expressed as a product of two probabilities: a probability _{G }_{T }_{G}_{T}

for _{a }_{b }_{c }

Given a single RNA alignment, there are typically a large number of possible derivations, each corresponding to a possible secondary structure for the alignment. In the rest of this work, we restrict ourselves to this space of derivations, which we characterize by its derivational entropy as described below.

Results and discussion

Algorithm

Let the set of all derivations for the input alignment be Φ. The total probability of the grammar producing the input string is:

The computation of

Note that Equations 4 and 8 still hold when the set of derivations is restricted to a subset, as opposed to the entire space, so we can write the entropy as:

We now show how to express the entropy in terms of expected rule frequencies. Note that:

which can be computed using the expected rule frequencies obtained from the inside-outside algorithm (cf. equations 7 and 8). This can be seen by noting that Equations 4 and 7 still hold when the set of derivations is restricted to a subset, as opposed to the entire space.

Furthermore, if

We observe that _{d∈Φ }_{2}(_{T}

Once the values of the inside-outside variables have been calculated for an input string of length ^{2}) time for rules of Type 2, and ^{3}) time for rules of Type 3. As the time complexity of the inside-outside algorithm is also ^{3}), the computation of the entropy over the possible derivations of the input string does not increase the time complexity of RNA secondary structure prediction.

Interpretation of the derivational entropy

The derivational entropy provides a measure for the "spread" of the probability distribution on possible secondary structures. For equiprobable events, information entropy increases logarithmically with the number of possible outcomes. It is clear, therefore, that the maximum derivational entropy increases with sequence length. It has been shown

The maximum derivational entropy is therefore expected to increase logarithmically with _{l}

_{max}

In practice, however, _{max }_{max}_{max}_{max}

The entropy of the structure probability distributions computed by PPfold, for random sequences of various lengths and nucleotide compositions

**The entropy of the structure probability distributions computed by PPfold, for random sequences of various lengths and nucleotide compositions**. The nucleotide composition of each dataset is given in the legend. Each point represents the mean of the entropy values for 100 random sequences, with the error bars indicating the standard deviations.

Using information entropy to interpret low reliability scores

Derivational entropy is related to various reliability measures already reported by prediction programs. PPfold in particular computes the probability of a pair between two columns as the sum of the (normalized) expected frequencies of rules of Type 2 emitting that column pair:

The probability of column

Importantly, while the reliability scores depend both on the structures and their probabilities, the derivational entropy is only a function of probabilities, and does not depend on the similarity of the structures to each other. Derivational entropy therefore provides complementary information to reliability scores. For example, if the reliability scores predict a low accuracy, the entropy can help reveal the underlying reasons. A low reliability score can be observed in different situations, for example (a) if there is insufficient structure signal, so there are no structures of high probabilities and the probability distribution is "spread", or (b) if there are two or more possible (topologically different) structures of high probabilities, so the probability distribution has several "peaks". Entropy will be high in the first case, but low in the second case, and can therefore be used to distinguish the two situations from each other.

To illustrate this with a practical example, a PP-fold prediction of the secondary structure of the random nucleotide sequence:

reveals that the underlying probability distribution has an entropy of 28.55 bits, with an average reliability score 0.65 for the final predicted structure. In comparison, the following combinatorial sequence of the same length:

shows a significantly lower entropy of 18.82 bits, at the same time as a low average reliability score of 0.56. Despite the low reliability scores in both cases, the entropy value correctly reveals that the combinatorial sequence has a relatively small number of very different secondary structures dominating the probability space, whereas the probabilities are more uniformly distributed over a large number of possible structures in the case of the random sequence.

To investigate if similar patterns could also be observed for alignments of longer biological sequences, we applied the same technique on a range of bacterial and eukaryotic 16S/18S rRNA alignments. The generation of the alignments is described in the Methods section. We plotted the entropy against the PPfold reliability scores, and colour-coded the data points after the accuracy of predictions. As Figure

Use of entropy in conjunction with reliability scores to characterise probability distributions on RNA structure space

**Use of entropy in conjunction with reliability scores to characterise probability distributions on RNA structure space**. (a) Prokaryotic alignments (_{max }_{max }

Relationship to prediction accuracy

An important question with respect to comparative RNA secondary structure prediction is how the accuracy of predictions varies with the quality of input alignment, and to what extent the accuracy of a predicted structure can be predicted. Reliability scores and entropy both measure variation in the secondary structure space, so both are expected to be correlated with prediction accuracy. We note, however, that a natural limitation of both entropy and reliability scores is that they are computed under the model, which effectively assumes that the model itself is an accurate description of the biological folding processes. If this is not the case, high confidence values computed under the model can still correspond to low prediction accuracies in reality.

Nevertheless, we compared how the accuracy of predictions correlates with both the average structure reliability scores and the information entropy, for all alignments in our dataset consisting of bacterial and eukaryotic 16S/18S rRNA alignments. The results are shown in Figure

Variation in accuracy (F-measure) correlated with (a) derivational entropy (as fraction of _{max}

**Variation in accuracy (F-measure) correlated with (a) derivational entropy (as fraction of H**.

We observe that despite the PPfold reliability scores generally suggest somewhat higher prediction accuracies than what was actually observed, they convey

Lastly, we note that there is large variation in the reliability measures reported by different RNA structure prediction programs. By contrast, information entropy is a

Future perspectives include the computation of measures similar to entropy, such as the self-information of particular outcomes, or the Kullback-Leibner divergence of distributions to compare constrained and unconstrained models for RNA folding. We also expect that entropy may provide a number of other possible future applications in RNA secondary structure prediction.

Conclusions

The information entropy of the probability distribution generated by phylo-grammars can be computed efficiently from the inside-outside variables, and has been implemented as part of PPfold. Information entropy is a well-defined characteristic of the underlying probability distribution, which complements the reliability values already reported by algorithms for an increased understanding of RNA structure variability. It is also a method-independent measure of prediction certainty, providing theoretical advantages over existing methods.

Methods

Implementation

We have implemented the algorithm for the Knudsen-Hein (KH99) grammar

We note that the KH99 grammar is not in double emission normal form, as it includes the rule

In the case of the KH99 grammar, there is a maximum of one bifurcating rule originating from each nonterminal symbol. It is also known that the expected frequency of a nonterminal symbol can be computed in ^{2}) time for any SCFG, and the expected frequencies of rules from the same nonterminal symbol sum to the expected frequency of the nonterminal symbol. Hence, in the case of the KH99 grammar, the time complexity of the computation of the derivational entropy (given the inside-outside variables) could further be reduced to ^{2}).

As described in the Results and Discussion section, the value of the entropy depends on the length of the alignments. Hence, small adaptations were made in the algorithm to be able to compare the entropies of alignments that include a particular sequence. The default option in PPfold is to remove columns where fewer than 75% of the sequences have nucleotides. In the case of entropy computation, this is replaced with removing only the columns where the selected sequence has gaps. This ensures that all alignments that include the selected sequence have an equal prediction length.

Test sequences and alignments

Random sequences of different nucleotide compositions were generated using the online FaBox tool

For the generation of alignments, among families with an experimentally verified secondary structure in the Rfam database, we chose two families (RF01960 - Eukaryotic small subunit ribosomal RNA, and RF00177 - Bacterial small subunit ribosomal RNA), for which our initial tests indicated that a particularly wide range of structure prediction accuracies may be achieved depending on the choice on the sequences included in the alignment. Starting with a sequence of interest in both families, we constructed our datasets by randomly adding sequences from the family alignment one by one, up to a final size of maximum 15 sequences. For each alignment size (between 1 and 15 sequences in the alignment), the process was repeated 50 times. This way we obtained 1+14 × 50, not necessarily distinct cases per family, with 1-15 sequences each (including the starting sequence as a standalone case). The alignments of the selected sequences and the reference secondary structures thereof were adapted from those in Rfam, by deleting gap-only columns and any base pairs involved with those columns.

Comparing accuracies

Accuracies are reported in terms of the

where

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

ZS designed the algorithm with the help of BK and CNSP, implemented it and wrote the manuscript. JWJA and ÁN provided testing data, participated in the testing, and helped with writing the manuscript. CNSP and JK supervised the project, critically revised the manuscript and provided funding. All authors read and approved the final manuscript.

Declarations

This research has been funded by the Danish Council for Strategic Research under the project #09-061856. JWJA would like to thank the EPSRC for funding. ÁN would like to thank BBSRC for continued funding through OCISB. CNSP is supported by Centre for Membrane Pumps in Cells and Disease (PUMPKIN), a Centre of Excellence funded by the Danish National Research Foundation. The publication costs for this article were also funded by PUMPKIN.

This article has been published as part of