An image database of 226 gene expression pattern images was initially generated using data from the literature 17181920212223242526272829. All were lateral images and exhibited early stage (1–8) expression patterns. These images were selected because they had some commonality of gene expression (as seen by the human eye), which allowed us to evaluate the performance of the BESTi in finding correct as well as false matches under controlled conditions. BESTi was also tested for scalability on a larger dataset containing 1819 (1593 plus the 226) lateral views of early stage embryos. These 1593 images were obtained from 262 articles.

In order to present comprehensible result sets in this paper, we have primarily discussed the findings from the dataset of 226 and provided information on how those queries scaled when they were conducted for the larger dataset. In general, our focus was to show the retrieval of biologically significant matches based on both the visual overlap of the spatial gene expression pattern and the genes associated with the pattern retrieved.

Each image was standardized and the binary expression pattern extracted following the procedures described previously 6. These extracted patterns, their invariant moments (φ₁ through φ₇), and binary feature representations were stored in a database. We also calculated and stored the expression area (the count of the number of 1's in the binary feature represented image), the X and Y coordinates of the centroid (, ), and the principal angle (θ) for each extracted pattern.

To quantify the similarity of gene expressions in two images, we computed two measures (S_S, S_C) based on the BFV representation (See equations 2 and 3 in Methods). S_S is designed to find gene expression patterns with overall similarity to the query image, whereas S_C is for finding images that contain as much of the query image expression pattern as possible without penalizing for the presence of any expression outside the overlap region in the target image. For a given pair of gene expression patterns (A and B), S_S is the same irrespective of which image in the pair is the query image. That is, S_S(A,B) = S_S (B,A). This is not so for S_C, because S_C measures how much of the query gene expression pattern is contained in the image. Therefore, S_C (A,B) ≠ S_C(B,A).

For IMV representation, we computed one dissimilarity measure (D_φ, equation 13 in Methods). Results from D_φ should be compared to that from S_S, as both of these measurements do not depend on the reference image, i.e., D_φ (A,B) = D_φ(B,A) and, also they capture overall similarity or dissimilarity.

The effectiveness of the BESTi in finding biologically similar expression patterns was geared towards determining the biological validity of the results obtained from the image matching procedure. All results were based solely on quantitative similarities between images without using any textual descriptions. All images were lateral views from the early stages of fruit fly embryogenesis and were oriented anterior end to the left and dorsal to the top. We refer to the images retrieved as the BESTi-matches.

Due to the presence of multiple areas of expression, some patterns in the database that appeared to contain much better matches (by eye and biologically) to the query image were not found or ranked very high. Hence, we also analyzed multi-domain expression patterns separately for the smaller dataset. Developmental biologists are also interested in finding such patterns as they contain overlaps with the expression domains in the query image. In fact, a large number of the expression patterns available today contain multiple isolated domains of expressions since more than one topologically distinct region of expression may be produced by many genes, transgenic constructs, probes or experimental techniques (multiple staining). In such cases, we need to consider each of these regions individually as well as in the context of the composite pattern. Biologically, it is important to consider them separately because different regions of expression may be under the control of distinct cis-regulatory sequences [e.g., 2838] or may represent the expression of different genes in a multiply-stained embryo.

Separating multi-domain gene expression patterns into individual components was straightforward; we simply generated multiple images from the same initial image and included them in the target dataset. This resulted in 192 additional images (418 total) in the database all of which were components of the initial gene expression patterns. The images were separated into expression regions horizontally and/or vertically depending on the gene expression. For this new set of images, the IMV as well as BFV representations were re-calculated and the BESTi query constructed as above.

Results from BFV-S_S and IMV queries for this data set are given in Figures 1D and 1E, respectively. Now, many images with multiple regions of expression are retrieved in the result set (Figure 1D: D1–D8) and many of them show an even better match with the query pattern than those in Figure 1A for the BFV-based BESTi search. For instance, gene expression patterns are now retrieved (with more than 55% pattern similarity) from embryos with the expression of tailless (tll), which is known to interact with slp1 in defining the embryonic head 22, and with a composite expression of race (related to angiotensin converting enzyme), sog (short gastrulation) and eve (even-skipped) due to enhanced race expression in the anterior domain caused by a transgenic construct causing ectopic expression of sog 19. Therefore, the strategy of dividing multi-domain expression data into individual domains provides additional flexibility to query individual components or sub-sets of complex expression patterns. Results also improved for IMV (Figure 1E), but again the outcome reinforced the need to use the difference in centroid to limit the result set.

Next we examine the performance of S_S, S_C and D_φ in finding BESTi matches for a query pattern with multiple regions of expression (Figure 5A). This complex expression pattern consists of anterior and posterior domains caused by enhanced race expression resulting from dosage alteration of dpp in a gastrulation defective (gd) mutant background, and a middle stripe due to misexpressed sog using an eve stripe-2 enhancer [Figure 2d in 19]. The results from this query are shown in Figure 5A1–A8 (only the original image set (226) was used as the target database in this case). We again find that S_S finds many images from the same paper as well as some images from other research articles with similar expression patterns. The results correctly include expression pattern of eve (Figure 5A4), of another pair-rule gene (ftz: fushi tarazu; Figure 5A6), and of two other developmentally related genes 3940.

When D_φ is used as a search criterion, it produces some correct matches in the result set (Figure 5B1–B8). However, it generally fails to rank biologically meaningful matches as the best matches. Use of the centroid in this case is also not productive, as most of the matches show very close centroids. The principal angle (θ) value calculated does not show a significant difference in the early stage embryos used in this study. The results using the S_C based search are given in Figure 5C1–C8. They show a number of images in common with the S_S results. However, as expected, there are significant differences between the two searches.

The results in Figures 5D and 5E demonstrate the power of the BESTi-search when the multi-domain expression data are represented in their component patterns (domain database). In this case, all the BESTi searches are based on the use of S_S as the search criterion. These searches are based on the complete expression (Figure 5D) and on one of its components (bottom-left domain, Figure 5E). All, but one, BESTi-matches in Figure 5D contain both domains of expression. In contrast, the use of only the left, anterior, domain (Figure 5E) in the BESTi search produces many other images in which the gene expression pattern is similar to only the anterior-ventral query pattern. Therefore, the use of individual expression components as search arguments increases the potential of directly identifying different overlapping expression patterns.

The binary coded bit stream pattern, in which the two possible states indicate staining over or under a threshold value, is called as Binary Feature Vector (BFV). This is referred to as the Binary Sequence Vector (BSV) in 6. In other words, we represent each image as a sequence of 1's and 0's, where the black pixels (stained areas) are denoted by a value of 1 and the white pixels (unstained and background) are denoted by a value of 0. This BFV holds the gene expression and localization pattern information of each image.

The expression patterns are ordered by evaluating a set of difference values, D_E, between the binary feature vectors of every possible pair of images in the dataset. D_E was introduced in 6 and is formally given as,

D_E = Count(A XOR B)/Count(A OR B)     (1)

The term Count(A XOR B) corresponds to the number of pixels

not

S_S = (1 - D_E).     (2)

S_S quantifies the amount of similarity based on the overlap between two expression patterns. S_S is equal to 1 when the two expression patterns are identical (D_E = 0).

We introduce a new similarity measure in this paper that does not penalize for any non-overlapping region. The measure S_C quantifies the amount of similarity based on the containment of one expression pattern in the other given by

S_C = Count(A AND B)/Count (A)     (3)

If the entire query image is contained within the result set images found in the database, i.e., there is complete overlap (with respect to the query image) S_C is equal to 1. Note that, S_C(A,B) ≠ S_C(B,A), because the denominator corresponds to the gene expression area of the query image.

Some methodologies of image analysis produce numeric descriptors that compensate for variations of scale, translation and rotation. In the following section, we describe the invariant moment analysis of gene expression data. Invariant moment calculations have been used in optical character recognition and other applications for many years 15.

To calculate these invariant moment descriptors the standardized binary image 6 is converted to a binary representation of the same pattern (BFV). From this binary sequence of the image, the invariant moments and other descriptors are extracted using the following method 1441. The continuous scale equation used is

M_pq = ∬x^p y^q f(x, y)dxdy,     (4)

where M_pq is the two-dimensional moment of the function of the gene expression pattern, f(x, y). The order of the moment is defined as (p + q), where both p and q are positive natural numbers. When implemented in a digital or discrete form this equation becomes

We then normalize for image translation using and which are the coordinates of the center of gravity, centroid, of the area showing expression. They are calculated as

Discrete representations of the central moments are then defined as follows:

A further normalization for variations in scale can be implemented using the formula,

and is the normalization factor. From the central moments, the following values are calculated:

where φ₇ is a skew invariant to distinguish mirror images. In the above, φ₁ and φ₂ are second order moments and φ₃ through φ₇ are third order moments. φ₁ (the sum of the second order moments) may be thought of as the "spread" of the gene expression pattern; whereas the square root of φ₂ (the difference of the second order moments) may be interpreted as the "slenderness" of the pattern. Moments φ₃ through φ₇ do not have any direct physical meaning, but include the spatial frequencies and ranges of the image.

In order to provide a discriminator for image inversion (and rotation), sometimes called the "6", "9" problem, it has been suggested 1442 that the principal angle be used to determine "which way is up". This is extremely important in embryo images because gene expression at the anterior and posterior regions may simply appear to be mirror images of each other to the invariant moments, but biologically they are completely distinct. The principal axis of the gene expression pattern f(x, y) is the angular displacement of the minimum rotational inertia line that passes through the centroid (, ) and is given as:

The slope of the principal axis is called the principal angle θ. It is calculated knowing that the moment of inertia of f around the line is a line through (, ) with slope θ. We can find the θ value at which the momentum is minimum by differentiating this equation with respect to θ and setting the results equal to zero. This produces the following equation:

Using the condition |θ| < 45° one can distinguish the "6" from the "9" and rotationally similar gene expression patterns.

In invariant moment analysis, our initial method of image comparison calculates the Euclidean distance between the images using all moments (φ₁ through φ₇) and combinations of these moments. For example, if the first two invariant moments are used, then

and the distance D_ij, between a pair of images i and j where i, j = 1, 2,...n is given by

This can be expanded to use all of the moment variables. Here, the Euclidean distance, D_φ, between any two images is calculated as

where i and q designate images whose distance is being calculated and j designates the parameters used in the distance calculation and j = 1, 2, ..., 7. This assumes that all moments have the same dimensions or that they are dimensionless.

Using this method, it is possible to rank each of the images in order of their similarity based on, for example, the first two invariant moments that have clear-cut physical meanings. Expansion to include additional moments or parameters can be performed in a number of ways. It is possible to add additional parameters to the distance calculation making sure that each of the parameters has the same dimension. For example, φ₁ has the dimension of distance squared, while φ₂ has the dimension of the fourth power of distance, thus requiring the square root function to equalize dimensions for comparable distance calculation purposes. In general, the greater number of invariant moments used in the distance calculation, the more selective the ranking. We have also allowed for the use of the centroids and principal angle as a means of list limiting.