At present, the DNA microarray technology allows simultaneous monitoring of the expression levels of thousands of genes. The technique produces large and complex datasets that are relatively easy to generate but non-trivial to analyze and extract information from. So far, much of the data mining efforts have been focused on the statistical analysis (see, for instance 1234) and less on acquiring high quality data from the image analysis. Image analysis is the process of extracting information from the scanned microarray images, which is an important step due to the sequential nature of the microarray analysis 5. Consequently, problems in the initial steps have large impact on the interpretation of the final results from the experiment.

A number of technical issues during microarray preparation potentially affect the spot quality.

• Low signal intensity is perhaps the most generally acknowledged property that affects spot quality due to the subsequent problems in distinguishing signal from noise for spots with weak signals. Weak signal intensities should result from physiologically low expression levels but might also be related to surface properties of the slide, signal bleaching, scanner problems or incomplete or irregular hybridization.

• Intensity distribution issues appear as regions of pixels containing signals that clearly deviate from the average signal, typically as distinct sub-areas within the foreground region. As the signal of any given spot on a microarray slide is expected to be uniform over the entire spot foreground area, intensity distribution issues are usually a consequence of non-specific binding or irregular distribution of the printed DNA on the slide.

• Morphological issues refer to unexpected shape-related variations of the spot foreground region. This includes very small or very large spot sizes, low spot circularities or spot mixing. Size aberrations might be a consequence of precipitates or impurities in the printing solution or needle clogging during printing. Spots are expected to be roughly circular in shape, but manufacturing issues might result in deviation from the circularity norm. Furthermore, imperfections on the slide or washing problems might cause the dye from several spots to mix, referred to as bleeding, making the separation of these signals difficult or even impossible.

• Background issues appear as intensity fluctuations in the local background region immediately surrounding the foreground region. An increase in local background intensity or variance compared to the global slide background typically result from dye contaminants due to non-specific binding or incomplete washing.

Microarray spot quality control is essentially the identification and removal of spots with properties that cause the subsequent interpretation of the signal from these spots to be unreliable or misleading. Analogously, it is the recognition of characteristics that enable dependable interpretations and conclusions from the generated data. The field of microarray spot quality control has been largely neglected in the past but has recently become an area of interest. Existing documented semi-automatic methodologies include Bayesian networks 6 as well as linear combinations of quality parameters allegedly related to the quality of the spot 78910. Interestingly, manual evaluation is still a common quality assessment procedure, despite the labor-intensive nature of visually inspecting spots in the order of hundreds of thousands. Increasing availability and usage of microarrays in the field of transcriptomics has caused experiments to involve an escalating number of slides, which in turn has highlighted the primary bottleneck of manual quality assessment.

Partial least squares (PLS) is a generalized regression method which aims to maximize the covariance between the X (descriptor) and Y (response) matrices. PLS can handle large data sets of multi-collinear and noisy data with moderate amounts of missing data in both X and Y. PLS-DA can be seen as a special case of PLS where the response matrix Y is categorical (numerically represented as 0 or 1) and determines class belonging of observations. PLS-DA has been widely applied in microarray analysis (see, for instance 1112) as well as other areas of life science (see, for instance 1314). For a more detailed description of the properties of PLS, please consult 151617 and references therein.

Here, we propose the

m

a

s

q

o

t

Spot descriptors were subsequently subjected to multivariate discriminant analysis by means of PLS-DA with the aim to categorize spots of low quality and spots of high quality by treating these spots as separate classes. The utilized descriptors aimed to describe foreground and background irregularity measures, spot morphology and foreground density attributes that were potentially useful for discriminating between the reliable (not bad) and unreliable (bad) spots. For instance, the circularity measure is a descriptor ranging from 0% to 100%. If the circularity descriptor approaches its minimum value, the predicted class belonging should typically be higher (closer to 1) for the bad class compared to the not bad class. The coefficient of variation for the foreground and background regions is an example of an employed descriptor that illustrates reverse characteristics; higher values should typically provide greater class conformity with the bad class compared to the not bad class. However, all employed descriptors together contribute to the regression model and consequently to the final class determination at varying degrees depending on the properties of the spot.

The MASQOT approach aims to provide a consistent assessment of spot quality, applicable to various types of microarray data, thus avoiding the labor-intensive manual quality assessment process. The methodology generates continuous values of spot quality which can be utilized to discard spots of undesired quality or used as weights in subsequent analysis procedures.

Five cDNA microarray slides using the Populus second generation microarray slide layout (POP2) where the samples originate from a previous investigation of leaf development (Sjödin et al, in preparation) were used for classification training. Five additional POP2 slides, not included in the training set, were employed for external validation. Segmentation of raw images was performed using an implementation of the Seeded Region Growing (SRG) algorithm 18. The properties of each spot were subsequently characterized using a large set of descriptors allegedly linked to spot quality. These properties include foreground and background variability measures, spot morphology and foreground intensity distribution measures. Please consult Additional file 1 for a complete list of all the utilized descriptors.

Following segmentation, spots were inspected by three experienced microarray users and independently assigned to the two quality categories {bad, not bad}. Spots in the bad category consisted of all the spots that were classified as bad by at least one of the experienced users while the remaining spots were categorized as not bad. For classification and evaluation purposes, the spots in the bad category were subsequently partitioned into different sub-classes based on visual properties as described in table 1. This can be seen as characterizing each spot as exhibiting

1. No issues (not bad); or

2. Foreground issues (FI); or

3. Background issues (BI); or

4. Any combination of 2 and 3

To avoid confounding of properties, only spots displaying pure issues (entries 1–3) were used in the classification training, although all spots were used in the model evaluation. The three-class problem was subjected to multivariate analysis by means of PLS 16 regression coupled with discriminant analysis (PLS-DA) with the aim to discriminate not bad spots from FI spots and BI spots. The result from the PLS-DA regression model is a predicted class conformity (CC) value for each of the classes: not bad (CC_nb), foreground issues (CC_FI) and background issues (CC_BI) with the added restriction that CC_nb = 1 - (CC_FI + CC_BI). Due to this restriction, only the conformity value of the not bad spots (CC_nb) will be interpreted in the upcoming sections. A CC_nb value approaching 1 denotes high compliance with the not bad class, which can be interpreted as a quality measure of the spot. Spots visually categorized as bad should thus exhibit a value of CC_nb close to 0 whereas spots categorized as not bad should exhibit a value of CC_nb close to 1.

Receiver Operating Characteristics (ROC) plot of the classifications of the POP2 training set and POP2 test set, respectively, is available in figure 2. A density plot of the CC_nb value for the bad and not bad spots in the POP2 training set is shown in figure 3, showing a partial overlap between the discrimination of the two classes. Due to this overlap, discrimination accuracy was dependent on a threshold value t denoting the separation point between the two classes.

Spots with a predicted CC_nb value below t were classified as bad whereas the remaining spots were classified as not bad. The threshold value can be set more or less stringently depending on the quality filtering requirements. This is illustrated in figures 4a–b, depicting different views of interpreting the classification accuracy of the POP2 training set and the POP2 test set. The threshold values were set either to maximize the overall classification accuracy or to maximize the class-wise classification accuracy. The overall classification accuracies for the POP2 training set (38 627 spots) and POP2 test set (39 421 spots) were calculated using equation 1 for all t in the interval (0,1) using CC_nb values for the {bad, not bad} spots. The classification accuracy peak at a level of approximately 98% where t = 0.4 (see figure 4a). Predicted class-wise accuracies were calculated using equation 1 employing CC_nb values for the bad spots and CC_nb values for the not bad spots separately. The predicted class-wise accuracies intersect at a level of 95% (see figure 4b) for t = 0.5. Exact classification accuracies per sub-class, based on the intersection threshold value t = 0.5 as illustrated in figure 4b, are available in table 2.

The presented MASQOT approach was compared to three existing quality control methods: the composite quality score q_com proposed by Wang et al 8, the mean-median correlation factor mm_corr evaluated by Tran et al 9 and the coefficient of variation (CV) parameter CV_spot evaluated by Sauer et al 10. Threshold values for all quality control parameters were set to achieve maximum overall classification accuracy. The result, shown in table 3, demonstrates that the MASQOT approach provides a greater level of class discrimination for the POP2 test set compared to the remaining evaluated quality control methods.

cDNA microarray spot quality control is, in many aspects, a complex problem. Naturally, the automatic assessment of quality of each spot is highly reliant on the characterization of the spot. Incorrect approximations of the spatial location will affect the properties of the segmented foreground region, which in turn will influence the values of the quality control descriptors. In such a sequential process, where each step is dependent on the preceding steps, errors will propagate down-stream at a high rate. However, the most striking intricacy is perhaps the visual assessment, which is the foundation of this computer-based classification, where even experienced microarray users tend to disagree. As shown from the results presented here, it is the spots with unanimous visual quality assessment that are the most complicated to reproduce accurately. This disagreement stems from the more fundamental issue of defining 'quality' and in understanding the basal aspects that affect this quality.

The approach described here aims to assess the technical precision of each spot, which is believed to be linked to the biological accuracy (see 19 for a discussion regarding precision and accuracy in the microarray field). It should thus be stated, in this context, that lack of precision in a microarray spot measurement does not necessarily infer lack of accuracy. However, it is arguably reasonable to handle spots of questionable precision with specific care during the analysis procedure to aid the concluding biological interpretations.

The spot quality control assessment is commonly treated as a discrete problem (essentially, separating 'bad' spots from 'good' spots) but the spot quality varies on a continuous scale, ranging from very bad to very good. Instead of discarding spots, one might weight the spots according to the quality assessment. The concept of relative spot weights has previously been acknowledged in microarray normalization techniques (see, for instance 220) but might also prove to be valuable in additional analysis steps. However, evaluation of the usage and validity of spot weights based on the quality assessments provided here remains the scope of a future paper.

The rate of accuracy in prediction of the true positives (bad spots predicted as bad) and the true negatives (not bad spots predicted as not bad) are illustrated separately since these accuracies are not consistently of equal importance. For instance, depending on the user and the question at hand, it might be more important to avoid the risk of removing spots of decent quality than to eliminate all of the bad spots from the data set. Simply using the overall classification accuracy could be rather illusive, merely since the number of bad spots is much lower than the number of not bad spots in a typical data set.

The methodology presented here is based on the scaled sum of the intensities from both channels but can, with minor adjustments, also be based on single channel intensities. By using the scaled channel intensity levels, one avoids the risk of drowning information, in particular when there is a great difference in intensity level between the channels. In addition, the presented approach greatly resembles the visual illustrations of the spots which, by design, will provide an advantage in finding correlation between the visual quality assessment and the spot descriptors. Furthermore, it is more feasible for the average user to achieve a per-spot quality measure than a per-channel quality measure since this avoids raising questions with regard to what to do when only one channel is of low or moderate quality.

The recent advances in spot quality control have clearly shown that a good explanation of training data is possible using several different methodologies of varying complexity. However, very few efforts have been made to evaluate further aspects of the quality (for instance, more refined descriptors) and, most importantly, the reproducibility of the classification on external data. External reproducibility has been the major aim here, partly overshadowing the aim of internal reproducibility on the training data, which is shown by the clear agreement in accuracy between the independent POP2 training set and the POP2 test set.

The presented MASQOT technique provides a robust methodology for semi-automated cDNA microarray spot quality control with high accuracy of training data as well as external data compared to other evaluated methods. The MASQOT methodology generates non-discrete values of spot quality which can be utilized as weights in subsequent analysis procedures as well as to discard spots of undesired quality using the proposed threshold values.

PLS Partial Least Squares

PLS-DA Partial Least Squares Discriminant Analysis

SRG Seeded Region Growing

ROC Receiver Operating Characteristics

EST Expressed Sequence Tag

MB implemented the segmentation and gridding tools, performed visual classification, implemented the D-optimal design, conceived and generated the classification model and drafted the manuscript. DE generated the microarray slides, performed visual classification and helped to draft the manuscript. AS conceived the study, collected leaf samples, performed visual classification and helped to draft the manuscript. MS, SJ, HA and JT supervised the project. All authors read and approved the final manuscript.