Integrated siRNA design based on surveying of features associated with high RNAi effectiveness

1471-2105-7-516 1471-2105 Research article Integrated siRNA design based on surveying of features associated with high RNAi effectiveness Gong Wuming wuming@biocompute.umn.edu Ren Yongliang yongliang@biocompute.umn.edu Xu Qiqi qiqi@biocompute.umn.edu Wang Yejun yejun@biocompute.umn.edu Lin Dong lindong@biocompute.umn.edu Zhou Haiyan haiyan@biocompute.umn.edu Li Tongbin toli@biocompute.umn.edu

Department of Neuroscience, University of Minnesota, Minneapolis, MN 55455, USA

BMC Bioinformatics 1471-2105 2006 7 1 516 http://www.biomedcentral.com/1471-2105/7/516 17129386 10.1186/1471-2105-7-516 10 6 2006 27 11 2006 27 11 2006 2006 Gong et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

Short interfering RNAs have allowed the development of clean and easily regulated methods for disruption of gene expression. However, while these methods continue to grow in popularity, designing effective siRNA experiments can be challenging. The various existing siRNA design guidelines suffer from two problems: they differ considerably from each other, and they produce high levels of false-positive predictions when tested on data of independent origins.

Results

Using a distinctly large set of siRNA efficacy data assembled from a vast diversity of origins (the siRecords data, containing records of 3,277 siRNA experiments targeting 1,518 genes, derived from 1,417 independent studies), we conducted extensive analyses of all known features that have been implicated in increasing RNAi effectiveness. A number of features having positive impacts on siRNA efficacy were identified. By performing quantitative analyses on cooperative effects among these features, then applying a disjunctive rule merging (DRM) algorithm, we developed a bundle of siRNA design rule sets with the false positive problem well curbed. A comparison with 15 online siRNA design tools indicated that some of the rule sets we developed surpassed all of these design tools commonly used in siRNA design practice in positive predictive values (PPVs).

Conclusion

The availability of the large and diverse siRNA dataset from siRecords and the approach we describe in this report have allowed the development of highly effective and generally applicable siRNA design rule sets. Together with ever improving RNAi lab techniques, these design rule sets are expected to make siRNAs a more useful tool for molecular genetics, functional genomics, and drug discovery studies.

Background

Short interfering RNAs (siRNAs) are double-stranded RNAs typically of length between 19 and 25 with 2 nucleotide overhangs on the 3' ends, and they are capable of inducing sequence-specific, post-transcriptional deletion of gene products, leading to the silencing of the gene activity. Naturally occurring siRNAs are cleavage products from long double-stranded RNAs (dsRNAs) by Dicer, a ribonuclease III enzyme 12. The siRNA-induced mRNA degradation is a complicated process involving multiple steps, initiated by the binding of siRNA with RISC (RNA induced silencing complex), followed by RISC's activation, resulting in the recognition of the target mRNA and the degradation of the latter 134. As a gene knock-down tool used in labs, siRNAs can also be chemically synthesized and introduced into the cells by direct transfection 56 or delivered into the cells in forms of hairpin precursors through plasmid or viral vectors 78. The siRNA-based gene knock-down techniques are preferred by many because of their ability to disrupt individual gene's function without affecting related genes 9. These techniques are particularly attractive for gene silencing studies in mammalian cells, because, unlike longer double-stranded RNAs, siRNAs are not likely to trigger interferon responses which lead to non-specific mRNA degradation 5.

The efficacy issue represents a major challenge in siRNA design. This issue concerns the question of how to choose from the large number of candidate siRNAs the ones that give rise of the highest levels of knock-down activity. It is well known that only a fraction of these candidate siRNAs are highly effective in silencing the target genes. Two siRNAs targeting the mRNA sites that are separated by only a few nucleotides could exhibit very different knock-down efficacies 1011. What are the properties some siRNAs possess that render them more effective in knocking down the target genes than others? This is an issue of heated debate. Several sets of rules for designing high-efficacy siRNAs have been proposed (e.g., 11121314). In addition, a long list of factors have been claimed to influence siRNA knock-down efficacy and thus should be considered in siRNA design 151617181920212223242526.

There are significant disagreements among these design rules and considerable controversies over these claims. This situation has been discussed extensively in several recent review articles 2728, therefore we only list some examples of these disagreements here: 20 suggested that the sequence information alone was sufficient in determining the efficacy of a siRNA; however, 152224 advocated the need to incorporate thermodynamic properties (calculated using tools such as Mfold 29) in assisting siRNA design; while 1725 emphasized the importance of the accessibility to the mRNA sites by the siRNAs, and endorsed methods of filtering candidate siRNAs based on mRNA secondary structure properties. On factors determined by siRNA sequences, 1230 recommended choosing of sequences of intermediate G/C contents (around 50%) for effective siRNAs, while 1118243132 endorsed the choosing of sequences of lower G/C contents (< 60%) to increase the chance of making high-efficacy siRNAs. On position-specific properties, 11 suggested that the nucleotides on positions 3, 10, 13 and 19 on the sense strand played a critical role in determining the knock-down efficacy; while 14 claimed that positions 19 and 11, and perhaps 6, 13 and 16 on the sense strand were important in determining the knock-down efficacy of the siRNAs.

The debates over siRNA efficacy go beyond the disagreements among these design rules. In fact, the effectiveness of these rules per se is in question. 17 showed that most published siRNA design tools output large numbers of ineffective siRNAs, and had a similar performance to (or even worse than) a random selector when tested on data of an independent origin. 20 made similar observations, and alleged that several published efficacy predicting algorithms gave close to random classification on unseen data.

At least two groups of researchers pointed out that many existing studies on siRNA design criteria suffered from the "overfitting" problem 2024. This term describes scenarios where rules are extracted from datasets that have small sample sizes, low signal-to-noise ratios, and unique experimental settings. Rules obtained under these conditions are prone to spurious effects caused by noise in the data samples or specific aspects of the experimental settings or both; rules obtained in this manner are likely to perform unsatisfactorily when used on data obtained under different experimental settings.

The key to countering the overfitting problem and developing truly effective and generally applicable siRNA design rules is the availability of a large collection of siRNA efficacy data from diverse origins. We recently undertook the effort to document all siRNA experiments in published studies and provide sensible efficacy ratings of these experiments. This effort resulted in siRecords, the largest known curated database of mammalian siRNA experiments with consistent efficacy ratings 33. The availability of the siRecords data makes it possible to better analyze factors responsible for achieving effective RNAi experiments.

In this study, we first conducted a survey on the siRecords data of all known "features" previously implicated to influence siRNA knock-down efficacy. This survey resulted in a list of features that significantly boosted the chance of achieving higher siRNA efficacies. Then, we examined quantitatively how these significant features interact with one another in their joint effects on achieving higher efficacies. The combinations of features that give rise to the highest levels of boosting to siRNA efficacies were picked and reorganized using a disjunctive rule merging (DRM) procedure, which led to a bundle of non-redundant rule sets with controlled stringency level. The performance of these rule sets (termed the DRM rule sets) was then assessed using a reserved dataset and compared with existing design tools commonly used in current siRNA design practice.

An implementation of the DRM rule sets developed in this study is available for testing as an online siRNA design server 34.

Results

Overview of siRecords data

siRecords is a continuing effort aimed to document all mammalian siRNA experiments reported in literature, and provide systematically rated efficacies for these experiments 33. Currently, about 9000 records of siRNA experiments targeting more than 3000 genes are hosted in the siRecords database. For each siRNA experiment, we document the siRNA sequence, the target gene, key information about experimental conditions (cell line used; the method of producing the siRNA – chemically synthesized or vector-based; the method of testing the siRNA efficacy – western blot or real-time PCR or others), and an efficacy rating (elaborated below).

For this investigation, we picked all complete records of 19-mer siRNA experiments (21-mers if the two overhanging nucleotides on the 3' ends are counted) from the siRecords collection (dated 12/12/2005). The distribution of number of records per study is highly skewed – about 17.5% of the records (657 siRNA experiments) originated from 0.4% of the studies (6 studies, each reporting ≥ 30 siRNA experiments, Figure 1). To prevent our analyses from being biased by this small number of studies, we limited the number of siRNA experiments originated from a single study to be ≤ 30. For these studies where more than 30 siRNA experiments were reported, we randomly picked 30 to include in our analyses and discarded the rest. The resulting dataset includes the records of 3277 siRNA experiments targeting 1518 genes originated from 1417 independent studies. We randomly divided the dataset into two subsets at a 2:1 ratio. The larger subset – termed Set A – included 2184 records, and was used to survey features significantly associated with high efficacies and analyze the combinatorial effects of these features. The other subset (termed Set T, 1093 records) was reserved to test the conclusions obtained through the analyses of Set A.

Figure 1

The distribution of the number of siRNA experiments per study is highly skewed in the siRecords collection

The distribution of the number of siRNA experiments per study is highly skewed in the siRecords collection. A. Studies were categorized based on the number of siRNA experiments reported. Only 6 out of the 1,417 studies (0.4%) reported > 30 siRNA experiments per study. B. The distribution of the total number of records in each category. Six hundred and fifty-seven records (representing 17.5% of the entire dataset) originated from the 6 studies with > 30 records per study.

Survey of features significantly boosting siRNA efficacy

We set out to determine, using the Set A data, what "features" of the siRNA experiments are associated with elevated RNAi efficacies. A feature is a binary property of a siRNA experiment concerning a factor potentially relevant to siRNA efficacy, for example, the 6th nucleotide of the siRNA sequence = A. Each feature has a "complementary feature". A feature and its complementary feature constitute a "feature pair". More discussions about the definition of feature and related terms can be found in

Methods.

In siRecords, the effectiveness of any siRNA experiment is rated on a four-level scale: very high (if the gene product was reduced by ≥ 90%), high (if the gene product was reduced by 70–90%), medium (if 50–70% knock-down was achieved); and low (if < 50% knock-down was observed). In Set A, the percentages of records receiving very high, high, medium and low efficacy ratings are 34.1%, 34.6%, 16.3% and 14.9% respectively (Figure 2). The decision of using this four-level rating scheme was made based on balanced considerations about the usefulness and the reliability of the ratings 33. One consequence of this decision is that that the conventional t-test type of analysis 11 can not be performed on this dataset, because the dependent variable (efficacy rating) is not a continuous variable, but rather a categorical, ordinal variable. Proper categorical analysis techniques need to be adopted to analyze this type of data 35.

Figure 2

Survey of features associated with the achievement of higher efficacies

Survey of features associated with the achievement of higher efficacies. The efficacy of a siRNA experiment is rated on a four-level scale. In Set A, the percentages of records achieving these ratings are 34.1%, 34.6%, 16.3% and 14.9%, respectively. The distribution of the efficacy ratings across the four levels changes when certain feature is present in the siRNA experiments. For 14 selected features (they constitute 7 pairs of "complementary features"), the efficacy rating distributions of the subpopulations of siRNA experiments carrying these features are presented. Dotted vertical lines extend from the distribution of the general population.

We chose to use the Wald test of monotone trend to assess the evidence that the presence of a feature is associated with a significant up-shift (or down-shift) of the efficacy distribution. In addition, we conducted odds ratio permutation tests for two efficacy levels: > 90% and > 70% efficacies, because in siRNA design practice, we are interested in assessing whether a feature leads to increased chances of achieving higher efficacies (see

Methods). For instance, a Wald test of monotone trend indicated that the presence of the feature the 6th nucleotide of the siRNA sequence = A is associated with significant up-shift of the efficacy distribution (P = 0.0058); odds ratio permutation tests showed that the presence of this feature led to significant increase in the probabilities of achieving both > 90% (P = 0.043) and > 70% (P = 0.0024) efficacies (see Supplementary Figure 1 in Additional file 1).

Additional File 1

Supplementary results and discussions. Discussions of cooperativity between features in their joint effects, performance of DRM rule sets in subsets divided by confounding factors, utility of online siRNA design tools, rationale of DRM procedure, and survey results of features significant associated with high siRNA efficacy.

Click here for file

We examined 276 features (they constitute 138 "feature pairs") for their association with higher RNAi efficacies, using the Wald test of monotone trend and the odds ratio permutation tests. The features we examined include, to our knowledge, all that have been implicated in previous studies to improve siRNA effectiveness. Each of these features can be placed into one of five categories. The first category is based on nucleotide identities at specific positions on the 19-mer siRNA sequence, e.g. the 6th nucleotide = A; there are 76 feature pairs in this category. The second category includes 19 feature pairs that are either composite sequence features, e.g. there are at least three (A/U)'s in the seven nucleotides at the 3' end of the siRNA, or features that are defined based on the G/C content of the siRNA. The third category consists of 13 feature pairs that are based on the thermodynamics of the siRNAs as measured by the melting temperature, or binding energy. The fourth category, consisting of 16 feature pairs, includes features based on target mRNA sites, such as the relative positions of the target sites on the mRNA, and the local secondary structures of the target regions. Finally, the fifth category includes 14 feature pairs that are based on experimental settings, such as the cell lines used in the experiments (HeLa cells, HEK293 cells, and others), the methods used for making and delivering the siRNAs, and the methods used to evaluate the efficacy of the siRNA (Western blot, PCR-based, and others). The complete list of these tested features, and references to the studies that implicated them in enhancing siRNA efficacies, are provided in Supplementary Tables 1-5 in Additional file 1.

Table 1

Non-redundant significant features meeting the criteria (P_wald< 0.01) and (P₇₀< 0.01 or P₉₀< 0.01)

Feature name

% Low

% Medium

% High

% Very high

P ₇₀

P ₉₀

P _wald

2nd nucleotide = A

12.1

16.0

33.8

38.1

0.01

0.0026

0.0019

4th nucleotide = C

14.1

15.4

31.5

39.0

0.098

0.00036

0.0075

6th nucleotide ≠ C

14.3

15.6

35.0

35.1

0.00066

0.0089

0.0052

7th nucleotide ≠ U

14.4

15.9

34.5

35.2

0.01

0.0043

0.0091

9th nucleotide = C

11.1

16.6

32.6

39.6

0.008

0.00021

0.00053

17th nucleotide = A

11.4

15.5

37.1

35.9

0.00049

0.1

0.0049

18th nucleotide ≠ C

14.4

15.9

34.3

35.4

0.01

0.00071

0.0048

19th nucleotide = (A/U)

12.0

16.0

35.3

36.7

0.00029

0.0043

0.000058

At least three (A/U)s in the seven nucleotides at the 3' end

13.4

16.4

33.7

36.5

0.00001

2.5E-09

No occurrences of four or more identical nucleotides in a row

14.2

15.9

35.4

34.5

0.00001

0.012

0.0014

No occurrences of G/C stretches of length 7 or longer

14.3

16.4

34.9

34.4

0.00001

0.000015

G/C content is between 35% and 60%

13.3

16.7

35.1

35.0

0.00001

0.0019

0.00018

T_mis between 20 and 60°C

13.2

16.5

35.0

35.3

0.0045

0.023

0.003

Binding energy of N16–N19 > -9 KCal/Mol

11.8

17.1

34.0

37.1

0.01

0.0026

0.00025

Binding energy of N16–N19 – binding energy of N1–N4 is between 0 and 1 KCal/Mol

12.6

14.9

32.5

39.9

0.01

0.00036

0.0078

Local folding potential (mean) ≥ -22.72 KCal/Mol

12.0

14.6

34.7

38.7

0.00001

9.3E-09

Target site is on CDS

14.4

16.2

34.3

35.2

0.00001

0.000055

Cell line = HeLa

7.9

10.6

41.2

40.3

0.00001

0.00016

4.0E-09

Test method = Western blot

10.8

15.5

34.8

36.9

0.00001

3.8E-14

Test object ≠ mRNA

13.1

14.5

34.8

37.6

0.00001

9.3E-10

At P < 0.01, the FDR for the three tests: Wald test of monotone trend, permutation test of odds ratios (> 70%) and permutation test of odds ratios (> 90%) were controlled at the levels of 0.056, 0.044 and 0.038 respectively.

Table 2

Non-redundant DRM rule set for the highest α level:RS_0.951.

Feature

F ₁

F ₂

F ₃

F ₄

F ₅

F ₆

F ₇

F ₈

F ₉

F ₁₀

F ₁₁

F ₁₂

F ₁₃

F ₁₄

F ₁₅

F ₁₆

F ₁₇

Rule 1

√

Rule 2

√

Rule 3

√

Rule 4

√

Rule 5

√

Rule 6

√

Rule 7

√

List of features:

Feature Index

Feature Names

F₁

2nd nucleotide = A

F₂

4th nucleotide = C

F₃

6th nucleotide ≠ C

F₄

7th nucleotide ≠ U

F₅

9th nucleotide = C

F₆

17th nucleotide = A

F₇

18th nucleotide ≠ C

F₈

19th nucleotide = (A/U)

F₉

At least three (A/U)s in the seven nucleotides at the 3' end

F₁₀

No occurrences of four or more identical nucleotides in a row

F₁₁

No occurrences of G/C stretches of length 7 or longer

F₁₂

G/C content is between 35 and 60%

F₁₃

T_mis between 20 and 60°C

F₁₄

Binding energy of N16–N19 > -9 KCal/Mol

F₁₅

Binding energy of N16–N19 – binding energy of N1–N4 is between 0 and 1 KCal/Mol

F₁₆

Local folding potential (mean) ≥ -22.72 KCal/Mol

F₁₇

Target site is on CDS

Table 3

Comparison in performance between 15 online siRNA design tools and DRM rule sets with four different stringency levels (α = 0.951, 0.895, 0.845 and 0.827).

Design Program

Institution/Company

URL

Avg. # Effective siRNAs

Predicted per Gene

Sensitivity

(> 90%)

Specificity

(> 90%)

PPV

(> 90%)

Sensitivity

(> 70%)

Specificity

(> 70%)

PPV (> 70%)

Ambion siRNA Target Finder

Ambion, Inc.

[64]

190.0

0.603

0.456

37.0%

0.574

0.457

70.3%

Jack Lin's siRNA Sequence Finder

Cold Spring Harbor Laboratory

[65]

207.5

0.204

0.759

30.9%

0.221

0.757

67.1%

siDESIGN Center

Dharmacon, Inc.

[66]

9.8

0.042

0.976

48.5%

0.036

0.982

81.8%

siRNA Target Finder

GenScript Corp.

[67]

22.4

0.032

0.979

44.4%

0.030

0.988

85.2%

Imgenex sirna Designer

Imgenex Corp.

[68]

22.8

0.116

0.913

41.5%

0.108

0.929

77.4%

EMBOSS sirna

Institute Pasteur

[69]

639.4

0.778

0.250

35.4%

0.767

0.258

69.9%

IDT RNAi Design (SciTools)

Integrated DNA Technologies, Inc.

[70]

5.8

0.032

0.975

40.0%

0.030

0.979

76.7%

BLOCK-iT RNAi Designer

Invitrogen Corp.

[71]

11.4

0.026

0.985

47.6%

0.020

0.982

71.4%

siSearch

Karolinska Institutet

[72]

19.6

0.016

0.986

37.5%

0.017

0.991

81.3%

SiMAX

MWG-Biotech, Inc.

[73]

35.1

0.161

0.843

35.3%

0.172

0.872

75.1%

BIOPREDsi

Novartis Institutes for BioMedical Research

[74]

10.0

0.794

0.908

31.3%

0.820

0.899

64.6%

Promega siRNA Target Designer

Promega Corp.

[75]

38.0

0.093

0.941

45.5%

0.083

0.958

81.8%

QIAGEN siRNA Design Tool

QIAGEN, Inc.

[76]

29.6

0.167

0.862

38.9%

0.161

0.881

75.3%

SDS/MPI

University of Hong Kong

[77]

432.8

0.656

0.380

35.9%

0.632

0.368

69.2%

WI siRNA Selection Program

Whitehead Institute

[78]

9.5

0.019

0.992

53.8%

0.015

0.994

84.6%

DRM RS_0.951

18.9

0.021

0.996

72.7%

0.013

0.997

90.9%

DRM RS_0.895

20.7

0.032

0.992

66.7%

0.021

0.994

88.9%

DRM RS_0.845

38.1

0.032

0.986

54.5%

0.026

0.984

90.9%

DRM RS_0.827

51.8

0.037

0.973

42.4%

0.038

0.988

87.9%

Comparison made based on Set T (1,093 siRNA experiments targeting 744 genes). Default settings were used for the 15 online predicting tools. Two sets of parameters (sensitivity, specificity and PPV) were calculated for each predicting tool or rule set. One was for > 90% efficacy (that is, a siRNA experiment was considered as truly effective if it achieved > 90% efficacy), the other one was for > 70% (considered truly effective if > 70% efficacy was reached in the experiment).

Table 4

Comparison in performance between 15 online siRNA design tools and 4 DRM rule sets based on independent subset of Set T.

Design Program

Institution/ Company

# Predicted effective siRNAs

# Predicted ineffective siRNAs

Sensitivity

Specificity

PPV (%)

Ambion siRNA Target Finder

Ambion, Inc.

144

0.645

0.362

74.3

Jack Lin's siRNA Sequence Finder

Cold Spring Harbor Laboratory

180

0.229

0.897

86.4

siDESIGN Center

Dharmacon, Inc.

217

0.036

0.983

85.7

siRNA Target Finder

GenScript Corp.

218

0.024

0.966

66.7

Imgenex sirna Designer

Imgenex Corp.

200

0.114

0.914

79.2

EMBOSS sirna

Institute Pasteur

180

0.801

0.190

73.8

IDT RNAi Design (SciTools)

Integrated DNA Technologies, Inc.

220

0.012

0.966

50.0

BLOCK-iT RNAi Designer

Invitrogen Corp.

222

0.012

1.000

100

siSearch

Karolinska Institutet

224

N/A

SiMAX

MWG-Biotech, Inc.

176

0.235

0.845

81.3

BIOPREDsi

Novartis Institutes for BioMedical Research

220

0.018

0.983

75.0

Promega siRNA Target Designer

Promega Corp.

198

0.127

0.914

80.8

QIAGEN siRNA Design Tool

QIAGEN, Inc.

191

0.151

0.862

75.8

SDS/MPI

University of Hong Kong

151

0.663

0.293

72.8

WI siRNA Selection Program

Whitehead Institute

212

0.072

1.000

100

DRM RS_0.951

223

0.006

100

DRM RS_0.895

220

0.024

100

DRM RS_0.845

219

0.030

100

DRM RS_0.827

215

0.048

0.983

88.9

Comparison made based on the independent subset of Set T (224 siRNA experiments targeting 197 genes). Default settings were used for the 15 online predicting tools. A siRNA experiment was considered effective if it achieved > 70% efficacy (was rated "high" or "very high" efficacy).

Table 5

Contingency table for the outcome of prediction tasks.

Truly Effective

Truly Ineffective

Predicted Effective

N _A

N _B

Predicted Ineffective

N _C

N _D

Of the features examined, we found 34 that were associated with a significant improvement in the efficacy distribution (P < 0.01, Wald test of monotone trend; FDR controlled at 0.056 by the q-value technique 36); among which, 26 significantly elevated the chance of achieving > 90% efficacies (P < 0.01, odds ratio permutation test, FDR controlled at 0.038), and 27 significantly enhanced the probability of achieving > 70% efficacies (P < 0.01, odds ratio permutation tests, FDR controlled at 0.044; see Supplementary Tables 1-5 in Additional file 1). There are several cases of sub-feature – super-feature relationships among these significant features. For example, the features the 6th nucleotide = A, and the 6th nucleotide ≠ C were both significant features, however, the former is a sub-feature of the latter since when the former feature is present, the latter must also be present. In each occurrence of sub-feature – super-feature relationship, we eliminated all but the one feature determined to be the most significant by the Wald test. The feature the 6th nucleotide = A was thus eliminated because the Wald test P value of this feature was higher than that of the feature the 6th nucleotide ≠ C. G/C content related features were treated as a special case. Several different G/C content ranges were suggested in previous studies as being possibly associated with high RNAi effectiveness (32–79%, 30–70%, 30–52%, 35–60%, 20–50% and 31.6–57.9%) 11121824303132. All these features were tested. Although they do not constitute sub-feature – super-feature relationships, we treated these features as redundant features, and retained only one of them (G/C content is between 35 and 60%) because it yielded the lowest P value (0.00018) in the Wald test. The resulting list of non-redundant significant features is shown in Table 1. Detailed discussions about these significant features, and comparisons of our analyses with previous findings can be found in the Additional file 1.

Combined effects of multiple significant features

The presence of any single significant feature was not sufficient to improve the efficacy distribution substantially. When present alone, the significant features listed in Table 1 increased the probability of achieving > 90% efficacies by an average of only 2.5% (from 34.1% to 36.6%), and they increased the chance of achieving > 70% efficacies by an average of merely 2.2% (from 68.7% to 70.9%). To achieve substantially improved efficacies, the concurrent presence of several significant features is required.

When multiple features are co-present, we cannot assume that their contributions to the effectiveness of the RNAi experiments are additive, since features are not always independent of one another. For instance, the presence of the feature the 19th nucleotide = (A/U), clearly increases the probability that the feature there are at least three (A/U)'s in the seven nucleotides on the 3' end of the siRNA to be true. Indeed, these two features exhibited negative cooperativity: when present alone, they increased the chances of achieving > 90% efficacies by 2.6% and 2.4%, respectively; when co-present, these two features resulted in merely a 2.7% increase in the chance of achieving > 90% efficacy, much smaller than the sum of the effects of the two features (see Additional file 1 for discussions about cooperativity and additive effects of multiple features).

In seeking effective siRNA design rules, we should try to identify combinations of features that exhibit positive cooperativity. The large size and diverse origins of the records in the siRecords dataset allowed us to systematically analyze how features jointly influence siRNA efficacies. Three significant features: Cell line = HeLa, Test method = Western blot and Test object ≠ mRNA were excluded from joint effect analyses because they are based on experimental settings, which are typically chosen independent of siRNA design. For the remaining 17 significant features, we looked at all possible combinations of a fixed number (l = 2,3,4,5 and 6) of features. For each combination of l features, we examined the number of records in Set A that concurrently carry all l features, and the percentages of these records that achieved > 90% and > 70% efficacies. For every given l, we focused on the top-10 feature combinations, i.e., the 10 combinations that exhibited the highest percentage of records achieving > 90% or > 70% efficacies. When there was a tie of more than 10 feature combinations, all tied combinations were considered. As we expected, as l – the number of features in the combinations increased, the number of records concurrently carrying all l features declined sharply (Figure 3C). Meanwhile, the percentage of experiments achieving > 90% and > 70% efficacies increased steadily as l, the number of features included in the feature combinations, increased (Figure 3A and 3B).

Figure 3

Highly effective siRNA design rules were obtained by selecting the top l-feature combinations, i.e., the combination of l non-redundant significant features that exhibited the highest percentages of records achieving > 70% or > 90% efficacies on Set A. A. For l = 2 through 6, the subpopulations of Set A records that carry all combinations of l features were examined, and the 10 feature combinations (FCs) that resulted in the highest percentages of records achieving > 70% efficacies were selected. When there was a tie of more than 10 FCs, all of them were considered (marked in the graph). The mean percentages of the top FCs are presented in black filled circles. These FCs were used to select siRNA experiments in the Set T, and the results are shown in grey filled circles. Error bars indicate standard errors. The first two data points in the graphs represent the base line levels (the percentage of records achieving > 70% efficacies for the entire Set A or Set T), and the mean levels for top-10 individual features (the 10 individual features that led to highest percentages of records achieving > 70% efficacies), respectively. B. Similarly to A, the top FCs selected with > 90% efficacies are plotted, together with the baseline levels and the mean levels for top individual features. C. The numbers of records selected in the top l-feature combinations dropped sharply as l increased. The mean numbers of selected records for Set A (with error bars indicating standard errors) are presented in black filled circles and black open circles for > 70% and > 90% efficacies, respectively. The numbers of selected records for Set T are presented in corresponding grey symbols. Again, the first two data points represent the baseline levels (numbers of records in entire Set A and Set T), and the numbers of records selected with the top-10 individual features, respectively.

The sigmoid shape of the two ascending curves is an indication of positive cooperativity (see discussion in Additional file 1). This suggests that by simply retaining the feature combinations that led to the highest percentages of records achieving efficacies of > 90% or > 70%, we were, in effect, exploiting the positive cooperativity, or favorable interaction, among these features. At l = 5, 24 feature combinations had a 100% chance of having efficacies > 70%, that is, every experiment in which the siRNA used had all features contained in any one of the 24 feature combinations exhibited efficacies of > 70%. Similarly, 14 feature combinations had 100% probabilities of having efficacies > 90% at l = 5, meaning that all siRNA experiments having these feature combinations demonstrated efficacies > 90%. At l = 6, 188 feature combinations had 100% probabilities of having efficacies of > 70%, and 94 feature combinations had 100% probabilities of achieving efficacies of > 90%.

Integrated rule sets for effective siRNA design

A disjunction of the top feature combinations described above (across l = 2 through 6; a feature combination is also called a rule thereafter) defines a rule set for designing effective siRNA experiments. Rule sets defined in this way are likely to contain redundancies, because if a rule consisting of l^ MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaacuWGSbaBgaqcaaaa@2E1D@