Computer and Information Sciences and Engineering, University of Florida, Gainesville, FL 32611, USA

Abstract

Background

Transcription factors regulate numerous cellular processes by controlling the rate of production of each gene. The regulatory relations are modeled using transcriptional regulatory networks. Recent studies have shown that such networks have an underlying hierarchical organization. We consider the problem of discovering the underlying hierarchy in transcriptional regulatory networks.

Results

We first transform this problem to a mixed integer programming problem. We then use existing tools to solve the resulting problem. For larger networks this strategy does not work due to rapid increase in running time and space usage. We use divide and conquer strategy for such networks. We use our method to analyze the transcriptional regulatory networks of

Conclusions

Our experiments demonstrate that: (i) Our method gives statistically better results than three existing state of the art methods; (ii) Our method is robust against errors in the data and (iii) Our method’s performance is not affected by the different topologies in the data.

Background

Genes are the smallest functional units of an organism. They carry out vital functions in cells by interacting with each other and with other molecules. Biological networks model such interactions among genes. Using biological networks, researchers are able to take a holistic approach on the analysis of cellular functions. Such analysis has shown that biological networks have a number of global properties. One of these properties is their hierarchical organization. Hierarchical organization defines a partial ordering of the underlying genes. Recent studies have shown that directed interactions between transcription factors (TFs) in transcriptional regulatory networks (TRNs) impose a hierarchy on TRNs

Hierarchical decomposition of a sample network with seven nodes denoted by _{1}, _{2}, _{7} to three levels

**Hierarchical decomposition of a sample network with seven nodes denoted by **_{1}**, **_{2}**, ****, **_{7}**to three levels.** Directed edges represent the interactions. Dashed line splits the nodes into different levels. Each of the seven nodes are assigned one of the three existing. levels

Problem definition

Let us denote a TRN with _{
i
}. We represent an edge from the node _{
i
}to _{
j
} with (_{
i
},_{
j
}). Also, we denote the maximum possible number of levels in _{
i
}with _{
i
} where _{
i
} is an integer in {1,2,,3,…,_{
i
},_{
j
})→{0,1} be a binary function that describes the key topological relationship between _{
i
} and _{
j
}. (We elaborate on the _{
ij
}for each pair of nodes as follows:

Our aim is to find an assignment of hierarchies to the nodes of

In this paper, we use two different

1. **Adjacency.** We define _{
i
},_{
j
}) = 1 if (_{
i
},_{
j
}) ∈_{
i
},_{
j
}) = 0 otherwise.

2. **Reachability.** We define _{
i
},_{
j
}) = 1 if there exists a path from _{
i
}to _{
j
}in _{
i
},_{
j
}) = 0 otherwise.

Depending on the choice of the two

There has been attempts to devise methods to reveal the underlying hierarchies of TRNs. Yu and Gerstein developed BFS-level method to carry out this task

Contributions

In this paper, we develop a novel approach to tackle the problem of discovering underlying network hierarchy. We first consider the topology of the network as a set of constraints. Then, we define two different objective functions using adjacency and reachability penalty functions. We define the minimization of total penalty as the objective of the problem. Using the above explanations, we transform this problem to a mixed integer programming problem(MIPP) **HI**erarchical **DE**composition of regulatory **N**etworks (HIDEN). The main advantage of HIDEN is it introduces a sound mathematical formulation to the network hierarchy problem. Our formulation can work with any objective function that is a linear combination of the edges. One drawback of HIDEN is that it does not scale well to very large networks due to the growing size of the MIPP with increasing number of TFs. In order to address this issue we develop a divide and conquer approach.

The rest of this article is organized as follows: In Section Algorithm, we describe the methods we developed in this paper. In Section Results and discussion, we discuss the results of HIDEN in detail. Finally, in Section Conclusion, we briefly conclude the paper.

Method

In this section, we describe the hierarchical decomposition method we developed. Section HIDEN describes our method. Section Example demonstrates HIDEN on a simple example. Section Divide and Conquer method describes divide and conquer method we employ to scale HIDEN to larger networks.

HIDEN

HIDEN transforms the hierarchical network decomposition problem to a MIPP

Let us denote the given network that will be decomposed with _{1}, _{2}, …, _{
m
}, where _{
i
} with _{
i
} for all _{
i
}∈ {1,2,3,…,_{1},_{2},…,_{
m
}} that minimizes the total penalty resulting from this level assignment. Therefore, the objective of our problem is the sum of individual penalty scores for each pair of nodes:

Next, we set a limit on the number of levels in the hierarchy. We do this by limiting the variables _{
i
} as follows:

We, then, represent each _{
ij
} as a linear constraint. Remember that _{
ij
}is a binary function in the following form:

We can rewrite this function as follows:

Let us only consider the cases where _{
i
},_{
j
}) = 1. We can represent the rest of this function using two linear inequalities. The following set of constraints represent the function _{
ij
}:

In order to prove that these inequalities model the function _{
ij
}correctly, we need to inspect all possible scenarios:

1. if _{
i
}>_{
j
}and _{
ij
}= 0, then −1 ≥_{
j
}−_{
i
}≥ −(_{
ij
}= 0. Therefore both (4) and (5) holds.

2. if _{
i
}≤_{
j
}and _{
ij
}= 0, then _{
j
}−_{
i
}≥ 0 and _{
ij
}= 0. Therefore, (4) holds, however, (5) does not hold.

3. if _{
i
}>_{
j
}and _{
ij
}= 1, then −1 ≥_{
j
}−_{
i
}≥ −(_{
ij
}=_{
j
}−_{
i
}−_{
ij
}is smaller than or equal to −

4. if _{
i
}≤_{
j
}and _{
ij
}= 1, then (_{
j
}−_{
i
}≥ 0 and _{
ij
}=

Therefore, enforcing the constraints (3), (4) and (5) implies:

This corresponds to the latter definition of the function _{
ij
} except the condition of _{
i
},_{
j
}) = 1. Since we choose the function _{
i
},_{
j
}) for every pair (_{
i
},_{
j
}). Therefore, we can manually ensure this property, by only considering _{
ij
} where _{
i
},_{
j
}) = 1 and excluding _{
ij
} completely from our calculations where _{
i
},_{
j
}) = 0.

Based on the constraints above, the MIPP we construct to solve the network hierarchy problem is as follows:

Example

In this section, we show the application of HIDEN on the network in Figure

Using this

Now we go over to the constraints. First set of constraints limit _{
i
}:

Then, we write the remaining functions as follows:

In the resulting problem,

Figure

Result of the hierarchical decomposition of the network in Figure

**Result of the hierarchical decomposition of the network in Figure****using HIDEN.** Note that the decomposition differs from the decomposition in Figure

Divide and Conquer method

HIDEN works well for networks that have up to 100 nodes. For larger networks, however, it becomes difficult to solve the resulting MIPP using current hardware. This is mainly because the number of integer variables of the MIPP that describe the problem for the given network increases. This increases the memory consumption and the running time significantly.

In order to solve our problem for networks that have more than 100 nodes we adopt a divide and conquer approach. Given a large TRN, we randomly divide this network into fixed size partitions. We do this by first randomly selecting a node from the given network. This node is the seed of the first partition, and thus it is a member of that partition. We then chose the remaining nodes in that partition iteratively by randomly growing the partition one node at a time. More specifically, at each iteration, we randomly select a node that is not selected so far and that is interacting with at least one of the nodes in the partition. We repeat these iterations until the number of nodes in the partition reaches to a predefined threshold or all the nodes in the TRN are assigned to a partition. Then, we use HIDEN to decompose the subnetwork defined by the nodes and the edges in this partition into hierarchical levels. Once we determine the levels of all the nodes in the current partition, we store those values as they will remain unchanged in the rest of our solution. Next we randomly pick another node from the given TRN among those that have not been considered yet as the seed of the next partition. We grow the next partition similarly and use HIDEN to decompose it into hierarchical levels. We repeat these steps until we exhaust all the nodes in the given TRN.

This method greatly reduces the running time of HIDEN on large networks. Since MIPP is NP-hard, depending on the size and the connectivity of the given TRN, the divide and conquer strategy can be orders of magnitude faster than the unpartitioned HIDEN. However, due to random selection of the nodes, it is possible for us to not achieve the optimal result. This is possible if the partition of the network we start with does not intersect with one or more of the levels in its underlying hierarchy. It is worth mentioning that this probability is usually very low. We can explain this as follows. Consider an N node network which contains n nodes belonging to a specific level x. If we select k nodes among these N nodes randomly, the probability that none of the k nodes belong to level x is (N-n choose k)/(N choose k). As k or n increases, this expression quickly converges to zero. In order to reduce this probability further, we repeat the divide and conquer strategy multiple times, each time starting from a randomly selected node. In our implementation, we repeat this process 1000 times for real TRNs. After 1000 iterations, the probability of all the trials starting with an undesired (i.e. does not intersect with all the final levels) partition becomes very small (i.e. if for 1 iteration, the probability is as high as 0.9, after 1000 iterations, the probability becomes 0.9^{1000}∼10^{−46}). Since the running time of partitioned HIDEN is orders of magnitude less than that of the unpartitioned HIDEN, 1000 repetitions remains to be practical. It took less than 10 minutes for the largest dataset (

Results and discussion

In this section, we evaluate HIDEN using a number of computational tests. In our tests, we let the underlying MIPP solver to handle the case of multiple optimal results. We only consider the unique result reported by the solver in our discussions. In the rest of this paper, we will use the term

Dataset In our experiments, we used TRNs of

In the rest of this section, we first compare HIDEN with other existing hierarchical decomposition methods in Section Comparison with existing hierarchical decomposition methods. In Section Biological evaluation of network hierarchies we evaluate the results our method using a number of biological properties of TFs. Finally in Section Effects of input on HIDEN, we analyze the behavior of our algorithm with respect to different quantitative properties of the data.

Comparison with existing hierarchical decomposition methods

The objective of hierarchical decomposition is to arrange the TFs of a given network to levels so that the gene that alter the activity of the other appears at a higher level than the other throughout the network as frequently as possible. The two

The penalty is a quantitative value that can be used to compare different methods on the same dataset. However, since the size (number of genes and interactions) and the topology of these networks deviate significantly, the resulting penalties will differ significantly across datasets. In order to report a statistically sound value that describes the success of a method independent of the network size and topology, we also compute the Z-scores of the resulting penalty values.

Let us denote the level assignment obtained by a specific method for an _{1},_{2},⋯,_{
m
}}. Let

A higher Z-score implies a better level assignment. Typically, a Z-score of four or higher is very significant as they indicate a result which is 4 or more standard deviations more extreme than the mean Table

**Organism**

**Method**

**Num.**

**Adjacency**

**Reachability**

**Level**

**Penalty**

**Z-score**

**Penalty**

**Z-score**

^{a}For HIDEN we vary the maximum allowable level from three to eight. We report the adjacency and reachability penalties as well as the Z-scores for these penalties for each experiment. “Num. Level” denotes the maximum number of allowed levels. The results for HiNO on human are omitted, because of problems in execution.

Yeast

HIDEN

3

140

10.8

3600

13.9

4

103

10.8

3027

14.6

5

88

9.5

2774

14.1

6

91

10.2

2573

13.4

7

79

9.8

2469

12.8

8

79

11.6

2365

13.7

vertex sort

9

179

7.3

3920

10.2

BFS-level

4

245

6.0

5734

9.9

HiNO

3

279

6.8

6205

10.5

E. coli

HIDEN

3

15

6.3

19

7.1

4

8

6.5

10

6.8

5

5

6.4

7

6.7

6

5

6.2

6

6.6

7

5

6.2

5

6.6

8

5

6.2

5

6.6

vertex sort

6

10

5.7

11

6.5

BFS-level

4

44

3.7

65

5.1

HiNO

4

41

4.2

59

5.3

Human

HIDEN

3

101

7.4

1950

9.4

4

84

7.4

1608

10.6

5

75

7.9

1435

10.8

6

66

7.9

1347

10.2

7

72

7.5

1287

9.7

8

72

7.3

1248

9.9

vertex sort

5

207

1.2

2162

5.8

BFS-level

3

210

0.72

2163

6.0

S. cerevisiae

We compared HIDEN with all the three competing methods for this dataset. Our method outperformed all the three methods in terms of both adjacency and reachability penalty values as well as the Z-scores regardless of the number of levels. As the number of levels allowed increases, the penalty incurred by HIDEN monotonically decreases. This, however, is not true for the Z-score as it depends on the distribution of nodes to levels. For instance HIDEN attains the highest Z-score for adjacency penalty at level eight whereas it attains that using only six levels for the reachability penalty. The biggest drop of penalty takes place when the number of allowed levels increases from three to four. We observe further, yet, smaller improvement in the penalty as the number of allowed levels increases beyond four.

Among the competing methods, the vertex sort method of Jothi

E. coli

For this dataset, we compared HIDEN with all three existing methods. The penalty values of all the methods for

H. sapiens

We compared HIDEN with vertex sort and BFS-level methods for this dataset. We omitted HiNO in this experiment because we could not run it on this dataset. The results follow a similar pattern as those of the two other datasets. HIDEN outperformed vertex sort and BFS-level even when it used fewer levels. The gap between the Z-scores of HIDEN and the other methods was even more significant than the previous datasets. HIDEN led to the highest drop of penalty of from three to four levels and continued to improve with increased number of levels.

Biological evaluation of network hierarchies

In this section, we analyze HIDEN using biological evidence. First, we check functional properties of genes across different levels. Then, we evaluate the locations of essential genes in the hierarchy.

Functions of genes

TRNs regulate the expression of genes that take part in many processes in an organism

The p-values for the observed number of genes annotated with the wound healing process at each level for the

**The p-values for the observed number of genes annotated with the wound healing process at each level for the ****TRN.** The network is divided into six levels using HIDEN with reachability as penalty scheme.

Each gene in an organism takes part in at least one metabolic function. A gene participating in a large number of reactions is a common phenomena in many organisms. In this experiment, we compare the level of each gene with the number of functions they participate in. By doing so, we aim to discover any existing relation between the two. In order to do this, we use the gene ontology database

Illustration of the distribution of the number of functions that each gene participates in for the TRN of

**Illustration of the distribution of the number of functions that each gene participates in for the TRN of ****.** Each circle represents a TF. The network is divided into six levels using the reachability as the penalty function and placed in relevant levels. The horizontal lines separate the TFs to different levels. The genes are colored according to the number of Gene ontology terms they are annotated with in gray scale. The least number of functions is assigned the color black, where the largest number of functions is assigned the color white.

Illustration of the distribution of the number of functions that each gene participates in for the TRN of

**Illustration of the distribution of the number of functions that each gene participates in for the TRN of ****.** Each circle represents a TF. The network is divided into six levels using the reachability as the penalty function and placed in relevant levels. The horizontal lines separate the TFs to different levels. The genes are colored according to the number of Gene ontology terms they are annotated with in gray scale. The least number of functions is assigned the color black, where the largest number of functions is assigned the color white.

Illustration of the distribution of the number of functions that each gene participates in for the TRN of

**Illustration of the distribution of the number of functions that each gene participates in for the TRN of ****.** Each circle represents a TF. The network is divided into six levels using the reachability as the penalty function and placed in relevant levels. The horizontal lines separate the TFs to different levels. The genes are colored according to the number of Gene ontology terms they are annotated with in gray scale. The least number of functions is assigned the color black, where the largest number of functions is assigned the color white.

Gene Essentiality

The genes which an organism cannot survive without are called essential genes

The ratio of essential genes(solid boxes) and the P-values(dashed line) for the number of essential genes observed in

**The ratio of essential genes(solid boxes) and the P-values(dashed line) for the number of essential genes observed in ****TRN in each level of the hierarchy.** The network is divided into five different levels using the reachability penalty. The P-values are calculated based on the hypergeometric distribution.

Figure

The TRN of

**The TRN of ****with a subnetwork related to cancer highlighted.** In this subnetwork, external signals (i.e. Growth factors, other proteins and molecules) regulate or affect the proteins

Effects of input on HIDEN

In this section, we analyze HIDEN by changing the input of the algorithm. In order to do this, we first change the number of layers we decompose the network into. Then, we assume errors and uncertainties in input networks. Using our results, we explain how reliable our method is under different conditions. Finally, we discuss the quality of our results for different subnetworks.

Navigation of genes across levels in varying hierarchies

The location of a gene in the hierarchy depends highly on the total number of levels. This leads to the following important question: How much can we rely on the relative levels of genes? One key feature of our method is that it allows the user to specify the number of levels in the hierarchical decomposition of the given network. By exploiting this feature, next, we answer this question. Particularly, we show how the change the number of levels affect the locations of the nodes in the hierarchy. In order to do this, we first calculate the levels of every node for

Illustration of the navigation of genes across levels for the TRN of

**Illustration of the navigation of genes across levels for the TRN of ****.** Each circle represents a gene. The locations represent the levels of the genes in a 6-level decomposition, whereas colors of the genes represent their locations in a 5-level decomposition. The color red represents the bottom level in the hierarchy, green represents the topmost level and the gradient of colors in between is used to color the nodes in between.

Illustration of the navigation of genes across levels for the TRN of

**Illustration of the navigation of genes across levels for the TRN of ****.** Each circle represents a gene. The locations represent the levels of the genes in a 6-level decomposition, whereas colors of the genes represent their locations in a 5-level decomposition. The color red represents the bottom level in the hierarchy, green represents the topmost level and the gradient of colors in between is used to color the nodes in between.

Illustration of the navigation of genes across levels for the TRN of

**Illustration of the navigation of genes across levels for the TRN of ****.** Each circle represents a gene. The locations represent the levels of the genes in a 6-level decomposition, whereas colors of the genes represent their locations in a 5-level decomposition. The color red represents the bottom level in the hierarchy, green represents the topmost level and the gradient of colors in between is used to color the nodes in between.

Robustness of HIDEN

One weakness of all hierarchical decomposition methods arises from the nature of the biological network datasets that they are incomplete and imprecise. As a result, the actual network topology observed can be slightly different than what is given in existing network databases

This section evaluates the

Briefly, we mutate a given network

We conducted the experiments on

(Evaluation of the robustness of HIDEN) The Z-score of HIDEN’s hierarchical decomposition of the

**(Evaluation of the robustness of HIDEN) The Z-score of HIDEN’s hierarchical decomposition of the ****, ****and ****network using adjacency and reachability penalties.** Level assignment is done on the original network. The Z-score is computed on the mutant network where the network is mutated at increasing mutation percentages. The results are reported for three different highest allowed levels, namely three, six and eight.

The most important observation that follows from our results is that the Z-score remains high even after we mutate the network by 20%. We observe a slight drop as the mutation rate increases, yet the results remain statistically significant. This observation holds for small (3), medium (6) and large (8) number of allowed hierarchical levels. This result has two major implications. First, HIDEN is extremely robust with respect to network mutations. It was able to identify hierarchical structure using the clues that remain in the topology of the given network after all mutations take place. Thus, even if the original network may be imprecise, the decomposition found by HIDEN will be a true decomposition with a high probability. Second, the degree preserving edge shuffling does not affect the decomposability of the network. The fact that even the original level assignment

Stability of HIDEN to network mutations

So far, we have observed that HIDEN was able to decompose the networks of the given three organisms successfully. This observation along with our last conclusion from the previous section begs the following question: Can HIDEN decompose the mutant networks or was there a bias in topology of these three networks in favor of HIDEN? In other words, how stable is HIDEN with respect to alterations in the network topology?

In order to evaluate the

Tables

**Organism**

**Level**

**Mutation [%]**

**0**

**5**

**10**

**20**

**40**

^{a}The numbers above them is the corresponding Z-score. Level indicates the maximum number of allowed levels.

Yeast

3

9.10

11.63

10.17

12.12

10.10

(118)

(137)

(127)

(127)

(130)

4

9.31

11.20

11.30

11.37

10.91

(99)

(117)

(103)

(114)

(104)

5

9.26

10.96

10.97

11.62

10.67

(84)

(108)

(92)

(103)

(88)

E. coli

3

5.76

4.98

5.26

4.98

5.34

(17)

(16)

(22)

(16)

(15)

4

5.43

4.77

5.67

4.77

5.57

(11)

(15)

(14)

(15)

(10)

5

5.46

4.72

5.52

4.72

5.34

(9)

(15)

(12)

(15)

(8)

Human

3

7.44

9.24

8.66

7.79

9.14

(101)

(105)

(95)

(106)

(107)

4

7.37

9.09

8.14

8.22

8.98

(84)

(90)

(83)

(92)

(92)

5

7.90

8.83

7.70

8.53

9.33

(75)

(93)

(81)

(73)

(86)

**Organism**

**Level**

**Mutation [%]**

**0**

**5**

**10**

**20**

**40**

^{a}The numbers above them is the corresponding Z-score. Level indicates the maximum number of allowed levels.

Yeast

3

12.35

15.21

14.62

15.20

15.31

(3674)

(3600)

(3483)

(3599)

3598

4

12.33

14.74

14.47

14.73

14.66

(3027)

(3026)

(2923)

(3025)

(3024)

5

12.27

14.18

14.29

14.18

14.18

(2754)

(2773)

(2644)

(2772)

(2771)

E. coli

3

7.73

6.58

6.90

6.58

6.37

(21)

(26)

(21)

(26)

(27)

4

7.29

6.16

6.21

6.16

5.93

(15)

(20)

(15)

(20)

(26)

5

6.95

6.00

6.22

6.00

5.81

(14)

(20)

(11)

(20)

(26)

Human

3

8.25

11.17

11.17

8.66

11.17

(1950)

(1951)

(1951)

(1944)

(1951)

4

8.88

12.02

12.02

10.45

12.02

(1628)

(1613)

(1613)

(1608)

(16.13)

5

12.40

11.71

11.71

10.45

11.71

(1431)

(1441)

(1441)

(1431)

(1441)

Local versus global hierarchy of subnetworks

The entire biological network of an organism can be considered as a (possibly overlapping) collection of smaller subnetworks where each subnetwork corresponds to a coherent functional group. For instance, cell cycle network describes the interactions that take place during the division and replication of a cell to produce new cells. Similarly, meiosis network describes a special type of cell division only observed in reproductive cells. These smaller subnetworks may follow a hierarchical structure as well within their local topologies. Clearly, we can use HIDEN on each of these subnetworks to find their hierarchical structure by isolating them from the rest of the network one by one. We call such hierarchical decomposition as

Let us denote the entire network with

**Subnetwork**

**Num.**

**Global**

**Local**

**Level**

**Penalty**

**Z-score**

**Penalty**

**Z-score**

^{a}The experiment is conducted on the two subnetworks of

Cell Cycle

3

4

3.2

3

4.2

4

2

3.3

1

4.0

5

2

3.2

0

3.7

6

2

3.1

0

3.7

7

2

3.0

0

3.7

8

2

2.9

0

3.7

Meiosis

3

8

0.7

2

3.5

4

6

1.2

1

3.8

5

6

1.2

1

3.8

6

5

1.6

1

3.8

7

5

1.5

1

3.8

8

5

1.5

1

3.8

The results demonstrate that the local hierarchy is better than the global one. This is not surprising as the global hierarchy is determined based on the entire network. Thus, the levels are determined with the goal of optimizing all the interactions in the network. On the other hand, local hierarchy is determined only based on the restrictions asserted by the corresponding subnetwork. We observe that the gap between the local and the global hierarchy is small for the cell cycle network. It is, however, significant for the meiosis network. In order to understand the factors that contribute to this gap, we performed a detailed analysis of the topology of the entire

These results demonstrate that the local and global hierarchies can deviate significantly depending on the topological relationship between the subnetwork and the rest of the network. Thus, detailed analysis of both decompositions can yield useful information regarding how the functions of a given subnetwork is depends on the other genes. HIDEN is capable of revealing such information.

Conclusion

In this paper, we took a novel approach to the problem of discovering underlying network hierarchy. We first transformed our problem to a MIPP. Then, we solved this problem using existing optimizers. We named this method **HI**erarchical **DE**composition of gene regulatory **N**etworks. However, due to the growing size of the MIPP with increasing number of genes, we encountered scalability issues. We proposed a divide and conquer approach to tackle such problems. Later, we experimentally showed that our algorithm outperformed existing solutions in terms of minimizing conflicting edges in hierarchy. We also evaluated our method using biological and statistical tools. Then, we discussed the relation between the hierarchy of a gene in a TRN and its location in cell, essentiality and function, based on our experimental results and biological evidence.

Availability and requirements

The source code for HIDEN can be found in Additional file

**A c++ implementation of the algorithm developed in this paper.**

Click here for file

**The resulting level assignments for the transcriptional regulatory networks of** **
S. Cerevisiae
**

Click here for file

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

GG and TK designed the method. GG implemented the method. GG and NB gathered experimental results. GG and TK wrote the paper. All authors read and approved the final manuscript.

Acknowledgements

This work was supported partially by NSF under grants CCF-0829867 and IIS-0845439.