Versteeg et al. 4 identified clusters where the median expression level of the genes involved is significantly higher than expected (RIDGEs), using a moving median false discovery rate (mmFDR) procedure (Figure 1). The mmFDR procedure identifies RIDGEs by testing the input gene-expression against the null hypothesis that the position of the genes on the chromosomes does not affect their expression levels. This same procedure can be used to identify significant windows (i.e., windows in the input sequence that have a median value that deviates significantly from expected, if assumed that the ordering of the numbers in the input sequence is random) related to any genomic feature mapped to DNA sequences. In an even wider scope, it can also be used to identify significant windows in any sequence of numbers.

Computationally, the most expensive step in the original mmFDR procedure is the repeated determination of medians over sliding windows of permutations of the input data to estimate the probability function corresponding to the null hypothesis. Our first improvement to the original method was to derive an exact formula for this distribution (see definitions and derivation in Additional file 1):

f ˆ ( r ) = ( r − 1 K ) ( N − r S − K − 1 ) ( N S ) = ( r − 1 ( S − 1 ) / 2 ) ( N − r ( S − 1 ) / 2 ) ( N S ) = S N ( r − 1 ( S − 1 ) / 2 ) ( N − r ( S − 1 ) / 2 ) ( N − 1 S − 1 ) . MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGafmOzayMbaKaacqGGOaakcqWGYbGCcqGGPaqkcqGH9aqpjuaGdaWcaaqaamaabmaabaqbaeqabiqaaaqaaiabdkhaYjabgkHiTiabigdaXaqaaiabdUealbaaaiaawIcacaGLPaaadaqadaqaauaabeqaceaaaeaacqWGobGtcqGHsislcqWGYbGCaeaacqWGtbWucqGHsislcqWGlbWscqGHsislcqaIXaqmaaaacaGLOaGaayzkaaaabaWaaeWaaeaafaqabeGabaaabaGaemOta4eabaGaem4uamfaaaGaayjkaiaawMcaaaaakiabg2da9KqbaoaalaaabaWaaeWaaeaafaqabeGabaaabaGaemOCaiNaeyOeI0IaeGymaedabaGaeiikaGIaem4uamLaeyOeI0IaeGymaeJaeiykaKIaei4la8IaeGOmaidaaaGaayjkaiaawMcaamaabmaabaqbaeqabiqaaaqaaiabd6eaojabgkHiTiabdkhaYbqaaiabcIcaOiabdofatjabgkHiTiabigdaXiabcMcaPiabc+caViabikdaYaaaaiaawIcacaGLPaaaaeaadaqadaqaauaabeqaceaaaeaacqWGobGtaeaacqWGtbWuaaaacaGLOaGaayzkaaaaaOGaeyypa0tcfa4aaSaaaeaacqWGtbWuaeaacqWGobGtaaWaaSaaaeaadaqadaqaauaabeqaceaaaeaacqWGYbGCcqGHsislcqaIXaqmaeaacqGGOaakcqWGtbWucqGHsislcqaIXaqmcqGGPaqkcqGGVaWlcqaIYaGmaaaacaGLOaGaayzkaaWaaeWaaeaafaqabeGabaaabaGaemOta4KaeyOeI0IaemOCaihabaGaeiikaGIaem4uamLaeyOeI0IaeGymaeJaeiykaKIaei4la8IaeGOmaidaaaGaayjkaiaawMcaaaqaamaabmaabaqbaeqabiqaaaqaaiabd6eaojabgkHiTiabigdaXaqaaiabdofatjabgkHiTiabigdaXaaaaiaawIcacaGLPaaaaaGccqGGUaGlaaa@86DF@

This exact formula reduces the number of cycles of computing moving medians of an input sequence of approximately 25,000 entries from at least 5,000 to 1, giving SigWin-detector the efficiency it needs to be used routinely and for processing and comparing multiple datasets within minutes to hours, instead of days. This efficiency could not be if f(m) was estimated by sampling the permutation space E_π, and counting the number of times m was the median value in any sliding window of size S.

Although we removed the need for computing moving medians over permutations of the input sequence, we still need to compute medians of windows sliding over the input sequence. We developed a new algorithm to compute those moving medians efficiently by exploiting the fact that moving medians for many window sizes must be computed simultaneously (Figure 2). This new algorithm is also suitable for computing any other order-statistics.

Additional Figure A1 (Additional file 2) shows a graph comparing our moving medians algorithm with the commonly used Hardle and Steiger's algorithm 22. While the execution time of their algorithm increases with window size (for a fixed sequence size), the execution time of our algorithm decreases with window size (Figure A1, upper panel). Because SigWin-detector needs to compute moving medians for many window sizes, our algorithm has a clear advantage over Hardle and Steiger's algorithm. In Figure A1, the break-even point of the cumulative computation is for S_max around 400. The efficiency of our method can be further improved by using a mixed algorithm that uses Hardle and Steiger's algorithm for small window sizes and our algorithm for large window sizes, or by employing a divide-and-conquer approach. For example, a two-phase algorithm would start by dividing the input sequence into chunks of size 2M, with M ≥ 2S_max, and applying the original algorithm to each chunk separately. Similarly, the second phase computes the medians for the missing sliding windows by dividing the sequence into chunks of the same size, but now using an offset M. This two-phase algorithm is also suitable for parallelization.

To broaden the applicability of the mmFDR procedure, we implemented SigWin-detector using an e-Science approach by implementing a general, reusable, and adaptable tool with access to Grid resources using the WS-VLAM workflow management system2021.

First we split the procedure into a collection of workflow components (called modules), each module performing a specific task that may be fine-tuned using parameters. The modules exchange data with each other by means of input and output ports. We then can choose the appropriate modules and compose a workflow suited to our specific needs 16. Figure 3 describes a basic workflow configuration of SigWin-detector.

The SigWin-detector Config-Basic1 workflow was tested on a Grid computer cluster composed of geographically distributed computational nodes: Distributed ASCI Supercomputer 3 (DAS-3, 23). Additional Figure A2 (Additional file 2) presents wall clock execution times of the SigWin-detector Config-Basic1 workflow (Figure 3) for input sequences of various sizes.

The basic workflow can be altered by substituting, deleting, or adding modules. For example, we can extend the workflow to get the input sequence from a remote uniform resource identifier (URI)and then put the resulting SigWin-map back into it. We can modify the workflow to generate one SigWin-map per logical subsequence of the input sequence, instead of a single SigWin-map for the complete sequence 16. We can also expand our workflow by computing significant windows for high median values (e.g., RIDGEs) and significant windows for low median values (e.g., anti-RIDGEs) simultaneously. The SigWin-detector workflow itself can be made into a "composite module" for more complex workflows. Furthermore, interconnection of WS-VLAM with the TAVERNA workbench 19 will permit the use of the existing TAVERNA components in connection with SigWin-detector. At the moment, Grid authentication prevents WS-VLAM workflows being used outside the Grid without the extra step of Grid certification. However, we are working on a Taverna workflow that encapsulates the SigWin detector, to be made available through the myExperiment webpage 24.

Once we finished our basic SigWin-detector, we modified it (Additional file 3) for application in our biological use case that aims to find (anti-)RIDGES in transcriptome maps. Figures 4 and 5 show a series of RIDGEOGRAMS for gene expression data for a recent version of the human transcriptome map (HTM) based on the UCSC release hg18 4, and Table 1 summarizes some RIDGE statistics. Each RIDGEOGRAM displays both RIDGEs (red-shades) and anti-RIDGEs (blue-shades), the different color shades representing different mmFDR threshold levels. The size of the resulting RIDGEOGRAMS is proportional to the number of genes on a chromosome. We determined i) genome-wide (anti-)RIDGEs, i.e., windows for which the median expression is significantly higher (lower) than expected by considering the whole genome gene expression profile in the mmFDR procedure (Figure 4), and ii) chromosome specific (anti-)RIDGEs, i.e., the same analysis, but considering only the specific chromosome gene expression profile (Figure 5). This distinction has a major effect on the outcome. If the expression values of the genes on a certain chromosome are typically significantly higher than the genome-wide values, then there are less chromosome specific than genome-wide RIDGEs (e.g., chromosome 19 in Figures 4 and 5 and Table 1). Conversely, if the expression values of the genes on a chromosome are typically significantly smaller than the genome-wide values, then there are more chromosome specific RIDGEs (e.g., chromosome 6 in Table 1 and Figures 4 and 5). In the case of anti-RIDGEs the opposite holds (e.g., chromosomes 17 in Table 1 and Figures 4 and 5). This example shows the importance of choosing the right sequence to compute the null hypothesis distribution. Based on the fact that chromosomes are separate molecules in a cell, one may favor the results from the individual chromosome SigWin-detector analysis to investigate potential higher-order gene expression regulatory mechanisms.

The RIDGEOGRAMS shown in Figures 4 and 5 only take the ordering of the genes into account, and not their actual physical position in the chromosome. However, from a biological perspective it is likely that the higher order gene-expression mechanisms that underlie RIDGEs relate to an actual section of the chromosome rather than a cluster of genes just ordered by their chromosome location. So we used our SigWin-detector to take the physical gene position into account by subdividing the chromosomes in stretches of constant value (250 kb). If a stretch contains the beginning of one or more genes, their average expression value is assigned to that stretch of DNA. For this analysis we used the SigWin-detector Config-Sub2 with preprocessed HTM data and adapted parameters. The resulting RIDGEOGRAMS are proportional to the chromosome's size (Additional Figure A3, Additional file 2). The anti-RIDGEs show a lower cut-off caused by the many 0 values in the HTM. The results from the SigWin-detector analysis using chromosome position are substantially different to those using chromosome ordering. This application demonstrated that SigWin-detector is an e-Science tool that allows convenient in-silico experimentation. To prove that this tool is generic, we used our workflow to examine a simple sequential data set: an extended time series of hourly ground level ozone concentration measurements (Additional file 4).