Center for Bioinformatics, Harvard Center for Neurodegeneration and Repair, Harvard Medical School, Boston, MA, USA

School of Automation, Northwestern Polytechnic University, Xi'an, China

Functional and Molecular Imaging Center, Department of Radiology, Brigham and Women's Hospital, Boston, MA, USA

Department of Molecular, Cellular and Developmental Biology, Yale University, New Haven, CT, USA

Abstract

Background

Reliable segmentation of cell nuclei from three dimensional (3D) microscopic images is an important task in many biological studies. We present a novel, fully automated method for the segmentation of cell nuclei from 3D microscopic images. It was designed specifically to segment nuclei in images where the nuclei are closely juxtaposed or touching each other. The segmentation approach has three stages: 1) a gradient diffusion procedure, 2) gradient flow tracking and grouping, and 3) local adaptive thresholding.

Results

Both qualitative and quantitative results on synthesized and original 3D images are provided to demonstrate the performance and generality of the proposed method. Both the over-segmentation and under-segmentation percentages of the proposed method are around 5%. The volume overlap, compared to expert manual segmentation, is consistently over 90%.

Conclusion

The proposed algorithm is able to segment closely juxtaposed or touching cell nuclei obtained from 3D microscopy imaging with reasonable accuracy.

Background

Reliable segmentation of cell nuclei from three dimensional (3D) microscopic images is an important task in many biological studies as it is required for any subsequent comparison or classification of the nuclei. For example, zebrafish somitogenesis is governed by a clock that generates oscillations in gene expression within the presomitic mesoderm

In recent years, there has been significant effort towards the development of automated methods for 3D cell or cell nuclei image segmentation

Despite active research and progress in the literature, development of a fully automated and robust computational algorithm for 3D cell nuclei segmentation still remains a challenge when dealing with significant inherent nuclei shape and size variations in image data. Examples include cases where the contrast between nuclei and background is low, where there are differences in shapes and sizes of nuclei, and where we are dealing with 3D images of low quality

In this paper, we present a novel automated method that aims to tackle the aforementioned challenges of segmentation of clustered or connected 3D cell nuclei. We approach the segmentation problem by first generating the gradient vector field corresponding to the 3D volume image, and then diffusing the gradient vector field with an elastic deformable transform. After the elastic deformable transform is completed, the noisy gradient vector field is smoothed and the gradient vectors with large magnitude are propagated to the areas with weak gradient vectors. This gradient diffusion procedure results in a gradient flow field, in which the gradient vectors are smoothly flowing towards or outwards from the centers of the nuclei. Subsequently, a gradient flow tracking procedure is performed from each vector point to find the corresponding center to which the points flow. We group all points that flow to the same center into a region, and refer to this region as the attraction basin of the center. Once we have completed the process of tracking the gradient flow, the boundaries of juxtaposed nuclei are formed naturally and hence these juxtaposed nuclei are divided. The final step includes performing local thresholding in each attraction basin in order to extract the nuclei from their corresponding background. We have evaluated and validated this algorithm and have presented results attesting its validity.

Results

In this section, a series of experiments are designed to evaluate and validate the gradient flow tracking method for segmentation of 3D images with juxtaposed nuclei. Both qualitative and quantitative results on synthesized and original 3D images are provided to demonstrate the performance and general applicability of the proposed method.

Validation using synthesized 3D image

We present an example of the results obtained from applying our proposed nuclei segmentation method on a synthesized image. Despite the fact that objects are touching each other and the presence of additive noise, the proposed segmentation method has segmented the touching objects perfectly, as shown in Figure

Result on synthesized 3D image segmentation

**Result on synthesized 3D image segmentation**. (a) The cross-sectional views of binary mask of the synthesized image. (b) The cross-sectional views of synthesized image with added noise. (c) The cross-sectional views of the overlaid boundaries of segmentation on synthesized noisy 3D image. (d) The cross-sectional views of randomly color-coded segmentation result.

We employ volume overlap measurement methods in order to quantitatively validate our segmentation results. Since there is no ground truth information available for real data and it is very time consuming to manually segment juxtaposed nuclei, instead of using real data, we take advantage of synthesized 3D images. The volume overlap between the automated results and the ground truth is defined as:

where _{a }is the automated extracted region and _{g }is the ground truth region. The ⋂ operator takes the intersection of two regions.

The synthesized 3D touching cell nuclei image is generated as follows: 1) Randomly select a voxel as seed point, and construct a mask of a sphere with a radius 10 mm centered at that point. 2) Generate 6 masks of spheres, which are tangent to the central sphere, with their corresponding radii ranging from 7 mm to 15 mm. This is done to simulate the variations between radii of real nuclei. 3) Blur the mask images by convolving with a 3D Gaussian kernel, and corrupting it with additive Gaussian noise. For the convenience of visual inspection of the segmentation results, we provide the volume rendering of the original 3D image and surface rendering of the segmentation results. Figure

Volume and surface rendering of synthesized 3D cell nuclei image and segmentation results

**Volume and surface rendering of synthesized 3D cell nuclei image and segmentation results**. (a) Volume rendering of the synthesized noisy image. (b) Surface rendering of the segmentation results with the proposed method, in which each color indicates a segmented object. (c) Surface rendering of the segmentation results with the global Otsu thresholding, in which the touching objects are not divided correctly.

We have synthesized seven 3D nuclei images. In order to present a quantitative measure for the validation of our results, the average value of volume overlap measurement for the seven cases is around 0.971, and the standard deviation is 0.014. As it is clear from these results, our proposed segmentation method achieves significant volume overlap with the ground truth, indicating the accurate performance of the gradient flow tracking method.

Validation using 3D

In order to further quantitatively evaluate the segmentation method, we applied the method to

The segmentation results of 3D C. elegans embryo images

**Image Index**

**True Number**

**Over-segmentation**

**Under-segmentation**

**Number**

**Percentage**

**Number**

**Percentage**

1

187

2

1.07%

0

0.00%

2

187

3

1.60%

1

0.53%

3

185

3

1.62%

0

0.00%

4

192

4

2.08%

2

1.04%

Average

187.75

3

1.59%

0.75

0.39%

An example of segmentation result using C

**An example of segmentation result using C. elegans embryo image**. (a) Volume rendering of original 3D image. (b) Surface rendering of segmentation results. For the convenience of inspection of touching cell nuclei, the results are randomly color-coded.

Validation using 3D zebrafish nuclei images

To further evaluate the proposed segmentation method, we have applied the method to ten 3D zebrafish images in which nuclei are labeled. The 3D zebrafish image datasets are from the Holley Lab at Yale University. These are fluorescent confocal images of fixed 12 somite stage zebrafish embryos stained with propidium iodide. All images were collected on a BioRad 1024 confocal microscope using a 25X Zeiss Neofluor objective. For the convenience of visual inspection of the segmentation results, we provide the volume rendering of the original 3D image and surface rendering of the segmentation results. An example of the original 3D image and nuclei segmentation results are shown in Figure

An example of segmentation results of 3D cell nuclei image of zebrafish

**An example of segmentation results of 3D cell nuclei image of zebrafish**. (a) Volume rendering of original 3D image. (b) Surface rendering of segmentation results. For the convenience of inspection of touching cell nuclei, the results are randomly color-coded.

2D slice view of the original image and segmentation result in Figure 8

**2D slice view of the original image and segmentation result in Figure 8**. (a). Original image. (b). The segmented image in which the green curves represent the boundaries of the cell nuclei.

The segmentation results of 3D cell nuclei image of zebrafish

**Image Index**

**True Number**

**Over-segmentation**

**Under-segmentation**

**Number**

**Percentage**

**Number**

**Percentage**

1

283

13

4.59%

15

5.30%

2

309

15

4.85%

16

5.48%

3

320

17

5.31%

15

4.39%

4

293

14

4.78%

13

4.44%

5

325

17

5.23%

19

5.85%

6

306

15

4.90%

17

5.56%

7

332

13

3.92%

19

5.72%

8

304

16

5.26%

15

4.93%

9

311

16

5.14%

13

4.18%

10

320

17

5.31%

17

5.31%

Average

310.3

15.3

4.93%

15.6

5.03%

Discussion

Our method has several advantages over previous approaches. The major advantage of the method is the ability to robustly segment densely packed, touching, or connected nuclei. Additionally, no sophisticated rules are used. The only assumption is that the centers of nuclei are brighter or darker than a nearby region. The fundamental difference between our method and existing methods lies in the diffused gradient vector information. In existing methods such as the threshold or watershed methods, intensity is the only adopted information, hence those methods are sensitive to the noise in the image, which usually results in over-segmentation. In contrast, in our method the gradient vector diffusion procedure propagates gradient vectors with large magnitudes to the areas with weak gradient vectors and smoothes the noisy gradient vector field. Meanwhile, it preserves the potential structural information of the gradient vector field. For example when two nuclei are touching each other, the diffused gradient vectors point toward the corresponding centers of the nuclei. This step greatly contributes to the success of touching nuclei segmentation. The disadvantage of this method is that it may have difficulty in processing the images of textured blob objects, since in that situation the gradient vector at the centers of nuclei are cluttered and the condition is violated. Currently the method is implemented using the C/C++ language, without using any other common library. Without any optimization, it takes less than 50 seconds on an Intel Pentium4 2.4 GHz machine with 1 GB memory to segment a volume image with a size of 230*162*80. The running time can be reduced further with multi-resolution implementation and code optimisation. After evaluating this method on larger and more diverse image datasets, we intend to release the algorithm to the cell biology community.

Conclusion

We presented a novel, automated algorithm for 3D cell nuclei segmentation based on gradient flow tracking. To validate the efficacy and performance of the proposed segmentation algorithm, we evaluated it by using synthesized and real biological images. The results show that the algorithm is able to segment juxtaposed nuclei correctly, a persistent problem in the field of cellular image analysis.

Methods

Gradient vector diffusion by elastic deformation transformation

Gradient information is an important factor in three-dimensional (3D) nuclei segmentation due to the fact that in any given nuclei image, the gradient vectors either point towards the central area of a bright nucleus, or outwards from the central area of a dark nucleus. However, in practice, the gradient magnitude is very small, and the direction of the gradient vector is usually not trustworthy due to the noise present in the image when approaching the central area of a nucleus. Additionally, when we are dealing with nuclei that are of irregular shapes, the gradient vectors tend to be cluttered. Motivated by these facts, here, we introduce a physical model that incorporates the diffused gradient vectors from the boundaries of the image to generate a smooth gradient field. Our gradient vector diffusion procedure propagates gradient vectors with large magnitudes to the areas with weak gradient vectors and smoothes the noisy gradient vector field

The diffused gradient vector field **v**(

^{2}**v **+ (**v**) + **v**) = 0,

where ∇^{2 }is the Laplacian operator,

_{σ }(

where _{σ}(

In our current implementation, the

where **v**_{t}(**v**(

_{t}(^{2}**v**(_{x }+ _{x }-

_{t }(^{2}**v**(_{y }+ _{y }-

_{t}(^{2}**v**(_{z }+ _{z }-

With the finite difference method, by setting the spacing interval Δ

The solution to Equation 2 defines the displacement of each position in a 3D elastic object, where displacements at some locations are pre-fixed. In Equation 2, variable **v **represents velocity and hence, considering hydromechanics rules, the second term in Equation 2 denotes the compression level of a compressible fluid. Given this description, setting **v**) = 0 represents an uncompressible fluid. The terms

The 3D view of gradient vector field and diffused gradient vector field with elastic deformation transformation of a slice cropped from a 3D cell nuclei image

**The 3D view of gradient vector field and diffused gradient vector field with elastic deformation transformation of a slice cropped from a 3D cell nuclei image**. (a). A slice of the 3D image. (b). The original gradient vector field. (c). The diffused gradient flow field with elastic deformation transformation. Obviously, the diffused vector field with elastic deformation transformation smoothly flows toward the central areas of cell nuclei. (d). Zoomed view of (b). (e). Zoomed view of (c).

Gradient flow tracking

In the diffused gradient vector field, the vectors flow toward the sinks, which correspond to the centers of nuclei. To follow the vectors until they stop at the sinks, the gradient flow tracking procedure is performed as follows. From any starting point **x **= (**x**' = (**x **flows through in the diffused gradient field is computed as:

Here, **v**(**x**) is the diffused gradient vector at point **x**, and "round" returns the nearest integer. The angle between the diffused gradient vectors of these two adjacent points is determined as:

When the angle between two consecutive diffused gradient vectors is less than 90 degrees, the gradient flow tracking procedure continues. Otherwise, the gradient flow tracking procedure is stopped, and a sink is reached. In this way, the vectors at each point along the tracking curve define a smooth path leading to a sink. In practice, segmentation of images into nuclei can be obtained by starting a gradient flow tracking procedure from every point in image. The set of pixels that flow to the same sink naturally produce the attraction basin of the sink. All points in the same attraction basin are segmented as an object (nucleus). All points are tracked independently, thus the attraction basin can be of arbitrary shape. After the gradient flow tracking step if the sinks are very close to each other, the attraction basins of the sink are combined together to obtain a larger attraction basin. In all our experiments, if the distance between two sinks is less than three pixels the attraction basins of the two sinks are combined together to obtain a larger attraction basin. Figure

Illustration of the gradient flow tracking based segmentation method

**Illustration of the gradient flow tracking based segmentation method**. (a). Clustered gradient vectors in 3D space. (b). Zoomed view of selected clusters of gradient vectors. Each color represents a separated cell.

The algorithm of the gradient flow tracking is summarized as follows.

1. Randomly select a point **x **as the initial point **x**^{0}.

2. Obtain **x**^{n + 1 }(**x**^{n}.

3. Compute the angle _{n }of diffused gradient vector between **x**^{n + 1 }and **x**^{n }with Equation 4. If _{n }is larger than

4. Replace **x**^{n }with **x**^{n + 1}. Return to step 2.

Gradient flow tracking is applied to each point in the image. All points in the same attraction basin are grouped into the same cluster. Since it is time consuming to run the tracking algorithm for every point, in order to improve the performance of our method, gradient flow tracking is not applied to the points that have already been on the gradient flow trajectory of a previously processed pixel. Instead, these visited points are directly associated with the sink to which the path flows. This improvement not only speeds up the segmentation, but also yields reproducible segmentation results.

Local adaptive thresholding

After the gradient flow tracking step, the image is segmented into smaller regions each of which is expected to contain only a single nucleus. From here the nuclei segmentation problem is turned into binary classification problem where we are interested in distinguishing the nuclei from their background in a small region. Therefore an intensity thresholding method is appropriate for extracting the nuclei from the background. In order to approach this problem, we can take advantage of the method employed by Otsu in

Summary of 3D cell nuclei segmentation method

The algorithm of the 3D cell nuclei segmentation method based on gradient flow tracking is summarized as follows.

1. Obtain the diffuse gradient vector field using the elastic deformation transformation

2. From each point, run the gradient flow tracking procedure, and label each passed pixel with a converged sink position.

3. Combine the attraction basins of the sinks whose distance is less than three pixels.

4. Assign the same label to the points in the same attraction basin.

5. Perform local adaptive thresholding in each attraction basin to extract the nucleus.

6. Optional procedure: eliminate regions with smaller number of pixels than a threshold T.

Running example

Figure

Illustration of 3D cell nuclei segmentation on a 2D slice

**Illustration of 3D cell nuclei segmentation on a 2D slice**. (a) A slice from 3D cell nuclei image. (b)Boundaries of small regions overlaid on the slice. The boundaries are obtained by the method in Section 2.2. (c) Edges of cell nuclei overlaid on the slice after the step of adaptive thresholding in Section 2.3. (d) Randomly color-coded extracted cells.

Figure

Illustration of cell nuclei segmentation in 3D space

**Illustration of cell nuclei segmentation in 3D space**. (a). Cross-sectional views of a 3D cell nuclei image. (b) Volume rendering of 3D cell nuclei image. (c) Surface rendering of extracted 3D cell nuclei. For the convenience of inspection of touching cell nuclei, the results are randomly color-coded. (d) Zooming view of the results.

Authors' contributions

GL and TL proposed the algorithm. GL implemented the algorithm. JN helped implement the algorithm. AT and STCW verified the algorithm and the result. AM and SH acquired the image data and provided biological input to this work.

The computer program, called CellSegmentation3D, loads the 3D Analyze format image (the suffix ".img" or ".hdr" is not needed in the input image name), and the segmentation result is also saved as the Analyze format. For each 3D input image, the program will output two results: 1) the segmentation result, in which all voxels belonging to the same cell are labelled with the same unique intensity; 2) the boundary map that separates segmented cells. Usage: CellSegmentation3D image_Input -f fusion_threshold -m min_Region -d diffusion_iteration -s sigma; Default parameters: fusion_threshold 3, min_Region 50, diffusion_iteration 15, sigma 1.0; For example: CellSegmentation3D elegans-01-01 -f 3 -m 35. This will produce the two result images:

Click here for file

The elegans-01-01.img is an example image file.

Click here for file

The elegans-01-01.hdr is the header file associated with the image.

Click here for file

Acknowledgements

This work is funded by a Bioinformatics Research Center Program Grant from Harvard Center for Neurodegeneration and Repair, Harvard Medical School (STCW). We would like to thank Dr. Sean Megason of California Institute of Technology for providing the