Chair of Computational Science, ETH Zürich, Universitätstrasse 6, CAB H69.2, ETH Zürich, CH-8092 Zürich, Switzerland

Abstract

Background

Despite significant progress in imaging technologies, the efficient detection of edges and elongated features in images of intracellular and multicellular structures acquired using light or electron microscopy is a challenging and time consuming task in many laboratories.

Results

We present a novel method, based on the discrete curvelet transform, to extract a directional field from the image that indicates the location and direction of the edges. This directional field is then processed using the non-maximal suppression and thresholding steps of the Canny algorithm to trace along the edges and mark them. Optionally, the edges may then be extended along the directions given by the curvelets to provide a more connected edge map. We compare our scheme to the Canny edge detector and an edge detector based on Gabor filters, and show that our scheme performs better in detecting larger, elongated structures possibly composed of several step or ridge edges.

Conclusion

The proposed curvelet based edge detection is a novel and competitive approach for imaging problems. We expect that the methodology and the accompanying software will facilitate and improve edge detection in images available using light or electron microscopy.

Background

The analysis of microscopy images is a time consuming task in many microbiological and biomedical laboratories. There is an ever-increasing need for analyzing large numbers of images acquired with microscopes in connection with different assays, where one wishes to measure the number of cells, the size of certain objects, the area occupied by cells, etc

The problem of edge detection has a long history in computer vision (see e.g.

A number of edge detection methods employ 2D Gabor filters. These filters are characterized by frequency, width and direction and have been mainly applied to object recognition problems.

Other powerful edge detection methods are also available, such as 'snakes' or 'active contours', which also use gradient information from the image to evolve a connected contour that minimizes its 'energy' in the landscape defined by the image

The curvelet transform was developed by ^{2}-edges, considering the number of curvelet coefficients needed to represent the edge to a given accuracy

The present edge detection scheme uses the discrete curvelet transform to extract information about directionality and magnitude of features in the image at selected levels of detail. The edges are then extracted using the 'non-maximal suppression' and 'hysteresis thresholding' steps of the Canny algorithm

The paper is organised as follows: in the next section, we give a brief introduction to the curvelet transform, as well as a description of the Canny edge detector and the Gabor filter edge detector of

The discrete curvelet transform

The discrete curvelet transform was introduced by

We introduce the discrete curvelet transform applied to an image with intensity values given by the function _{1}, _{2}), _{1 }= 0, 1,..., _{1 }- 1, _{2 }= 0, 1,..., _{2 }- 1, whose discrete Fourier transform (DFT) is

The discrete curvelet transform is now a decomposition of the image _{jlk}, such that

where _{1}, _{2}) and _{jlk }is the curvelet on level ^{2}-norms, i.e.

The discrete curvelet transform thus provides a decomposition of the image _{j }directions on each level, and _{jl,1 }× _{jl,2 }spatial shifts for each of these directions. The curvelet _{jlk }is defined through its discrete Fourier transform as

and

Here, _{l}. The slopes defined by the angles _{l }are equispaced. _{j }is a frequency window function with compact support, which is approximately 1 inside a wedge around the _{1}-axis with 2^{j }≤ _{1 }< 2^{j+1}, and decreases quickly to zero outside this area. The _{j }are defined so that

Thus, the discrete curvelet transform provides a decomposition of the frequency space into dyadic rectangular coronae, each of which is divided into wedges, the number of which doubles with every second level.

In figure _{jlk}. The imaginary part is odd with respect to its center, while the real part is even.

Curvelet and Gabor filter profiles

**Curvelet and Gabor filter profiles**. Profiles along the oscillatory direction of curvelets and Gabor filters. From top to bottom: real part of a curvelet; imaginary part of a curvelet; an odd Gabor filter with

The curvelet coefficients _{jlk }are computed as

This scalar product is computed as a sum over a parallelogram containing the support of _{jlk}, which in the wrapping version is wrapped by periodicity onto a rectangular area near the origin, namely 0 ≤ _{1 }<_{jl,1}, 0 ≤ _{2 }<_{jl,2}. The size of this rectangle also determines the ranges for the spatial shifts _{1 }and _{2}, and therefore the resolution at curvelet level

The curvelet transform can be seen as an efficient computation of convolutions with filters with profiles as in figure _{j}, the curvelet profile has the same shape at all scales.

On the coarsest level, _{1 }= 1) and are similar to the Meyer wavelet scaling function (see e.g.

The Canny edge detector

Since the convolution with the gradient of a function is equal to the gradient of the convolution, the filtering can be efficiently performed by first convolving with a Gaussian to smooth the image and then computing the gradient. The extraction of ridges of maxima is performed by looking for local maxima in the gradient direction. Additionally, the edge pixels are thresholded using two thresholds in order to reduce 'streaking', that is the subdivision of edges into short segments, while simultaneously reducing the probability to extract isolated edge points.

An implementation of the Canny edge detector thus amounts to the following steps (see also [

**Algorithm 1 (The Canny edge detector)**

^{2}

_{1}_{2}, _{1 }<_{2}, _{2}_{1}_{2}

It is interesting to note that

Gabor filters

A 2D odd Gabor filter, characterized by the angular frequency

The 1D profile of this filter (along the direction given by

Another significant difference is that the even Gabor filter does not have zero integral, and it is thus sensitive to absolute intensity values, not only variations in intensity. This makes it less suitable for edge detection applications. For curvelets, however, both the real and imaginary parts can be used to detect different types of edges, which we make use of in our scheme.

Methods

In this paper we combine the curvelet transform with a Canny edge detector algorithm leading to an edge detection scheme consisting of the following steps:

**Algorithm 2 (Edge detection using curvelets)**

We implemented the scheme in MATLAB (The MathWorks) using CurveLab

**CurveletUtils.** Matlab source code for a GUI implementing the edge detection method.

Click here for file

Applying the fast discrete curvelet transform

We apply the fast discrete curvelet transform, in the 'wrapping' version, with the default values for number of levels and directions. We use wavelets at the finest level since it makes the transform faster, and since the objects we are interested in live on coarser levels, so directional information at the finest level is of no importance. If edges on the finest level are of interest, however, curvelets can be used on the finest level and the edge detection scheme may be applied to this level as well.

Extracting the directional field

As each curvelet coefficient _{jlk }is associated with a particular location (the index

We first select a number of curvelet levels {_{1},..., _{P}}, depending on the size of the image features we are interested in, typically the width of the edges. The selected levels are usually determined by trial and error, but it is generally better to include more than one level, as edges may vary in width and leave traces on several levels. Each selected level, _{i}, is associated with a grid _{1}, _{2})|0 ≤ _{1 }<_{2 }<_{P}, that we compute the directional field. Note that this grid contains fewer grid points than the original image has pixels, so the directional field is not computed for each pixel in the image.

Now, each curvelet coefficient _{i }is mapped onto a subset _{i, P }(

Similarly, each curvelet coefficient _{jlk }is associated with a direction determined by the index _{i, P}(

_{i, P }(_{P }- 1}|_{P}·_{i }<

if _{i }is the number of directions on level _{i}.

Now, for each direction _{1}, _{2}) on the finest level _{P}, we sum up the magnitudes of the curvelet coefficients as

Note that we need only compute _{lk }for _{P}/2-1, since the directions _{P}/2 are separated by 180 degrees and thus represent the same direction. Furthermore, if we apply the curvelet transform to a real-valued image, then _{jlk}, so |_{jlk}|. It should be noted that by using the absolute values of the curvelet coefficients, we combine the even and odd filters shown in figure

Having computed _{lk}, we can now compute the major direction _{0}(

and define the field Ψ(_{1}(_{2}(

where _{0}, by the definition of the discrete curvelet transform. The angle _{0 }is taken along the valleys of the curvelets, meaning that the direction of the field Ψ(

Alternatively, we may compute the total curvelet magnitude at each position by

which gives a non-directional measure of the intensity variations in the image to compare with the directional measure Ψ(

Before picking the maximal direction, we also have the possibility to perform some smoothing, for example by averaging neighboring values of _{lk}, in both indices. This often leads to better results in the next step of our scheme. In particular, averaging over neighboring directions tends to give more distinct edges in the curvelet magnitude image, since curvelets that are not perfectly aligned with the edge also account to some extent for the edge.

Extracting the edges using non-maximal suppression

The directional field Ψ(

The last three steps of the Canny algorithm will then trace along the ridges of high magnitude of Ψ(

Extending the edges along the directional field

Even if the thresholding in the Canny algorithm is designed to reduce 'streaking' (the subdivision of edges into short segments), the edges that are extracted from steps 1 to 3 of algorithm 2 are not always connected to the desired extent. The reason for this may be that the influence of nearby edges prevent some pixels from being local maxima, so that they are ruled out by the non-maximal suppression, or that some pixels happen to have a value smaller than the low threshold, even though they are actually part of the edge.

As a remedy, the edges may be extended as follows: starting at the end points of the already selected edge segments, we take a step in the direction given by the directional field, away from the edge segment we start at. We then continue moving along the directional field until a specified number of steps have been taken, or we end up on a different edge segment. If we end up on a different edge segment, the entire path we moved along is included as an edge. A threshold is also employed to exclude pixels with small magnitude to be included in the extension.

Post-processing

The output from the scheme described so far is not always what one would like to present as a final output. In particular, one might want to make sure that the edges form a connected loop, remove isolated edge segments, and thin the edges. Furthermore, since Ψ(

Results

An electron microscopy image

We apply our method to an electron microscopy image showing a vesicle with some internal structure (see figure

Edge extraction on a microscopy image of a vesicle

**Edge extraction on a microscopy image of a vesicle**. An electron microscopy image of a vesicle (a), to which the edge extraction scheme is applied. The magnitude of the directional field extracted from the curvelet coefficients is shown in (b), with an extract showing the direction of the field in a small part of the image shown in (c). The last image (d) shows the final result of the edge extraction overlaid on the original image, after the edges have been extracted using the non-maximal suppression, and extended along the directional field to connect the different edge segments. Image courtesy of Prof. Urs Greber, University of Zürich.

Edge detection scheme comparison on the vesicle image

**Edge detection scheme comparison on the vesicle image**. Comparison of different edge detection schemes applied to the vesicle image. a) raw output from out curvelet-based scheme; b) Canny edge detector with thresholds 0.12 and 0.35; c) odd Gabor filter with

Curvelets, provide a multi-scale decomposition of the image that makes it possible to pick just a few scales and ignore for example the finest scale, where most of the noise is, and the coarsest scale, where the large differences in intensity are. Curvelets also provide us with information about directionality in the image, which enables us to search for structures with a strong direction, and trace along them.

For the image in figure

The final result, after applying the non-maximal suppression step on the field data, using weak threshold 0.25 and strong threshold 0.33, and after extending the edges along the directional field to connect the adjacent edge segments, is shown on the bottom right of figure

In order to better understand which structures are detected by our edge detection scheme, we extract the pixel values along a horizontal line of the image in figure

Cross section of an edge in the image

**Cross section of an edge in the image**. Investigation of the cross section of an edge from the image in figure 2. Top: image data from the horizontal line indicated by arrows in figure 2, in the region of the rightmost membrane (solid line), and the curvelet magnitudes on level 4 as extracted by the algorithm (dotted line, arbitrary scale). Bottom: cross section of a curvelet on level 4, approximately aligned with the edge.

We compare our edge detection scheme to the Canny edge detector

A tube formation assay image

As a second example, we apply our method to a light microscopy image from a tube formation assay (see figure

Edge detection on the tube formation assay

**Edge detection on the tube formation assay**. The edge detection scheme applied to the image on the top left (a) from a tube formation assay, where endothelial cells form vessels (or tubes). The results of the edge detection using levels 4 and 5 of the 6 curvelet levels are shown in (b), with a close-up of a region shown in image (c). For the last image (d), we have also indicated the sheet-like structures, detected as areas where the sum of the curvelet coefficients over all directions is larger than a threshold. Before thresholding, we computed the opening of the image generated from these sums of coefficients to make the detected areas more connected.

On the top right of figure

Depending on the desired results, it might be useful to exclude the sheet-like structures from the detection. Increasing the thresholds reduces the detected sheet area, but might also remove some of the tubes. Therefore we use a different method to exclude sheet-like structures. Since the intensity variations in the sheets contain essentially all directions, and not only a few as in the tubes, we use equation (3) to get the total curvelet magnitude at each position and then threshold this information to find the sheets. To further enhance the detection of the sheets, we compute the morphological opening of the total magnitude image using a disk of radius 5 as structural element, before applying the threshold 0.7 times the maximal magnitude of the opened image. The results are shown on the bottom right of figure

In figure

Edge detection scheme comparison on the tube formation assay

**Edge detection scheme comparison on the tube formation assay**. A comparison of different edge detection schemes for the tube formation image. a) raw output from the thresholding step of our curvelet-based scheme; b) Canny edge detector with thresholds 0.1 and 0.4; c) Gabor filter-based scheme with

In order to assess the sensitivity of our algorithm to noisy data, we investigate how much the extracted edges change when uniform noise is added to the image. To this end, we added uniform random noise in the interval [-

Sensitivity to noise

**Sensitivity to noise**. The fraction of falsely classified pixels for images with different levels of added uniform random noise, compared to the edge map computed for the original image in figure 5a. Results are shown for our curvelet-based scheme (solid line) and for the Canny edge detector (dashed line). The analysis parameters were kept the same for all images.

Discussion and conclusion

We have described a method for edge detection based on the discrete curvelet transform, and have seen that it can be useful for finding edges and elongated structures in images where the edges may not easily be detected using traditional methods. The main advantages of the curvelet transform are that it is a multi-scale transform, which enables us to extract edge information from detail levels of our choice and disregard the other levels, and that it gives us directional information at each point which can be used to improve the edge detection. Discarding the information on the coarser levels implicitly performs a background substraction and makes the method insensitive to background intensity variations in the image, while discarding the finest level efficiently excludes most of the noise. Excluding however the finest level also excludes information about edges with the width of a single or very few pixels. Our method should therefore primarily be applied to images where the edges to be detected show variations on a wider scale than only a few pixels.

This is also seen in the precision with which we detect the edges. Since the curvelets are defined on a coarser grid than the original image, the edges we detect will not have pixel precision, but rather the precision of the grid defined by finest selected curvelet level. This makes sense when the edges we search for are several pixels wide.

Since a single edge often leaves a trace on several curvelet levels, and since we may be interested in edges of varying width, it makes sense to include several levels of curvelets in the analysis, and indeed the results often improve when including several levels, as we did in the tube formation example.

When extracting the directional field, we pick the direction with the maximal curvelet coefficient as in (2), using its magnitude as the magnitude of the directional field, since this gives us a measure of the strength of the edge in that particular direction without influence from edges in other directions. It is often a good idea to perform a running average over three or more neighboring directions before picking the maximum, since an edge may have a direction which is not perfectly aligned with a single predefined curvelet direction, in which case it influences the curvelet coefficients for several directions and these should be added to include the entire effect of the edge. By computing the sum over all directions as in (3), we can also detect areas where there are many weaker edges meeting, and by distinguishing these areas from those found by picking the maximum, we can distinguish between isotropic and highly anisotropic areas of the image, as in figure

The plots in figure

The comparisons to the Canny edge detector and the Gabor filter-based detector in figures

Authors' contributions

TG developed and programmed the edge detection method and drafted the manuscript. PK conceived the study, contributed to the development of the edge detection method, edited and finalized the manuscript. All authors read and approved the final manuscript.