Statistical and Mathematical Sciences, Pacific Northwest National Laboratory, PO Box 999, Richland, WA, USA

Biological Sciences, Pacific Northwest National Laboratory, PO Box 999, Richland, WA, USA

Abstract

Background

Enzyme-linked immunosorbent assay (ELISA) is a standard immunoassay to estimate a protein's concentration in a sample. Deploying ELISA in a microarray format permits simultaneous estimation of the concentrations of numerous proteins in a small sample. These estimates, however, are uncertain due to processing error and biological variability. Evaluating estimation error is critical to interpreting biological significance and improving the ELISA microarray process. Estimation error evaluation must be automated to realize a reliable high-throughput ELISA microarray system.

In this paper, we present a statistical method based on propagation of error to evaluate concentration estimation errors in the ELISA microarray process. Although propagation of error is central to this method and the focus of this paper, it is most effective only when comparable data are available. Therefore, we briefly discuss the roles of experimental design, data screening, normalization, and statistical diagnostics when evaluating ELISA microarray concentration estimation errors.

Results

We use an ELISA microarray investigation of breast cancer biomarkers to illustrate the evaluation of concentration estimation errors. The illustration begins with a description of the design and resulting data, followed by a brief discussion of data screening and normalization. In our illustration, we fit a standard curve to the screened and normalized data, review the modeling diagnostics, and apply propagation of error.

We summarize the results with a simple, three-panel diagnostic visualization featuring a scatterplot of the standard data with logistic standard curve and 95% confidence intervals, an annotated histogram of sample measurements, and a plot of the 95% concentration coefficient of variation, or relative error, as a function of concentration.

Conclusions

This statistical method should be of value in the rapid evaluation and quality control of high-throughput ELISA microarray analyses. Applying propagation of error to a variety of ELISA microarray concentration estimation models is straightforward. Displaying the results in the three-panel layout succinctly summarizes both the standard and sample data while providing an informative critique of applicability of the fitted model, the uncertainty in concentration estimates, and the quality of both the experiment and the ELISA microarray process.

Background

Proteomic approaches are resulting in the identification of large numbers of proteins that can potentially be used as disease markers or drug targets. Unfortunately, proteomic approaches currently lack the throughput or quality metrics necessary to evaluate hundreds or thousands of samples that may be required to determine clinical usefulness of a biomarker

Processing a ELISA microarray experiment produces large volumes of data of wide variety and high complexity. Similar to traditional 96-well ELISA data, ELISA microarray data often are perturbed by processing error

Statistically evaluating ELISA microarray concentration estimation errors depends upon both the availability of the appropriate set of comparable measurements and the choice of data analysis methods. Sufficient replication within and across arrays is key to making precise estimates of both concentrations and errors

In this paper, we describe and illustrate a methodology for calculating the concentration estimation error of each assay in an ELISA microarray experiment based on a statistical analysis of the most likely sources of error. We expect the resulting data analysis algorithms to be a key component in a bioinformatics package for evaluating ELISA microarray data.

Methods

Making concentration estimates and estimating their errors in our ELISA microarray studies involve a sequence of steps beginning with the layout of the ELISA microarray and design of the experiment. Following execution of the analytical components of the experiment, the statistical analysis proceeds with data screening, normalization, and model identification. Estimation and evaluation of the standard curves and error estimation functions come next. Finally, the standard curves and error estimation functions are applied and then evaluated using a modeling diagnostic.

Layout of the ELISA microarray and design of the experiment

To estimate errors in concentration estimates, it is necessary to carefully lay out the microarray and design the experiment. Our layout features several distally separate replicates of each assay spot on each microarray to evaluate local processing effects. Our design addresses selection and application of treatments – in particular, replicate treatments – to a collection of arrays. This replication facilitates adjustments for the sources of variability that lead to ambiguous concentration estimates

To illustrate our technique for evaluating estimation errors in an ELISA microarray experiment, we used a subset of data from an ELISA microarray investigation of breast cancer biomarkers. The ELISA microarray experiments were performed as previously described

Data screening, normalization and model identification

Data screening, an exploratory data analysis, serves several purposes – identifying outliers, anomalous values, and experimental design shortcomings; identifying data transforms to improve curve-fitting and application; identifying measurement trends and other processing effects; and suggesting an appropriate functional form for the standard curve

Because our protein arrays feature fewer spots per array than do typical gene expression microarrays, a different approach to normalization, suitable for low spot-frequency arrays, is required. This normalization is critical, given that array-to-array processing error is common and that standard curves are estimated from reference spot intensities calculated from one set of arrays and then applied to sample spot intensities estimated from a separate set of arrays.

A scatterplot of intensity estimates of standard spots versus concentration is particularly useful. First, outliers and anomalies may be readily apparent. Second, the spacing between concentration values may be assessed. If standard concentrations follow from a dilution series, then the separation between concentrations decreases significantly with the decrease in concentration. This results in spot intensities measured at higher concentrations having much more leverage on the fit of the model than may be desirable. It should also be apparent whether the variability in spot intensity is increasing with mean spot intensity. Both increasing spacing in the concentrations and heteroskedasticity in the measured intensities affect the model fit and follow-on statistical inferences _{e }transformations of both concentrations and spot intensities.

A scatterplot of raw or transformed standard spot intensity versus concentration also provides an indication of the appropriate model for the data. In particular, data following a sigmoid curve favor the logistic curves while data apparently lacking the horizontal asymptotes of a sigmoid curve favor a linear or power law model. Although several models may be fit and one selected based on a goodness-of-fit statistic (see next section), the scatterplot is a useful visual check on this selection.

Several plots provide useful information about the quality of the fitted model. Of special importance are the scatterplot of residuals versus concentration and the scatterplot of residuals versus estimated intensity. In both cases, the variability of the residuals should be centered about zero and constant across concentration or intensity. Model bias is indicated by a systematic drift of residuals to one side of the zero line. Heteroskedasticity is indicated by a systematic change in the variation of the residuals. Both may indicate that a better model is necessary before proceeding to estimation of sample concentrations and estimation of concentration errors.

Standard curves and estimation errors

An ELISA standard curve expresses protein concentration as a function of spot intensity. One standard curve is required for each assay. In an ELISA microarray experiment, the standard data are collected by fixing a set of concentrations and measuring spot intensities via imagery of the treated arrays. A standard curve is estimated by fitting an appropriate function to the set of (concentration, intensity) measurement pairs

Common parametric choices for standard curve models are multiparameter logistic functions and power law functions. For an ELISA microarray, a strictly monotone model is consistent with the belief that a monotone change in concentration should result in a monotone change in spot intensity.

We estimate standard curves with both logistic and power law parametric models. The four-parameter logistic model _{1}, _{2}, _{3 }and _{4}, is defined as

The two-parameter power law model _{1 }and _{2}, in log_{e }terms, is

log_{e }(_{1 }+ _{2 }log_{e }(

We assume the errors, denoted by the term ^{2}. With either of these parametric models, concentration estimation errors may be estimated using propagation of error, also known as the delta method.

To choose between competing candidate models, a number of measures exist for evaluating model fit when replicate observations of each assay are available. These include partitioning the mean squared error, or MSE, into components representing pure error and lack of fit

To calculate the _{j}, _{j}) in turn and fit the model to the remaining points. We predict the value _{j }and calculate the _{j, - j }= _{j }-

The candidate model with the lowest

The basic approach to estimating concentration errors with the propagation of error method has three steps

Let **P**). Now, let _{s }be the estimated concentration from the sample intensity estimate _{S}, say _{S }= _{S}/**P**) and _{S}. Then, the propagation of error estimate for the concentration estimate _{S }is the square root of the product of _{S }and the parameter estimates **P**) with respect to the intensity **P**. Hence, the concentration estimation error of **P**) is the square root of the concentration estimation variance **P**))

**P**)) = **P**))^{T }Σ**P**))

where the Jacobian is

**P**))^{T }= [∂_{1},..., ∂_{N}]

and the augmented covariance matrix is

Hence, the formula for estimated standard error of _{S }is

For a given intensity estimate _{S }and standard error _{95%L}, _{95%U}) are

_{S }= _{S})

_{95%L }= _{S }- 2_{S}] (2)

and

_{95%U }= _{S }+ 2_{S}] (3)

For example, consider the four parameter logistic model, Eqn. 1. The concentration estimation equation is obtained by solving this equation for

The Jacobian matrix is obtained by taking the partial derivatives of the inverted four-parameter logistic function of _{1}, _{2}, _{3 }and _{4}

Diagnostic visualizations

A three-panel display combining a histogram of normalized sample spot intensities for a given antigen, its corresponding standard curve, and the graph of the concentration coefficient of variation, or relative error, versus concentration provides pertinent information about the conduct of the current experiment as well as information to improve future experiments. The standard curve panel presents a scatterplot of normalized standard spot intensities versus standard concentrations. The scatterplot is overlain with the estimated standard curve expressing concentration as a function of spot intensity. This panel also includes approximate 95% confidence intervals. These intervals summarize the uncertainty in concentration estimates due to both the uncertainty in estimating the standard curve and the uncertainty in the sample spot intensity estimate. Finally, a highlighted region helps distinguish concentration estimates s with acceptable errors from concentration estimates with possibly less than acceptable errors.

The segment of the standard curve corresponding to acceptable concentration errors may be determined using the 95% confidence intervals. The lower and upper endpoints of this segment, (_{L}, _{L}) and (_{U}, _{U}), are the two points such that the confidence intervals begin to increase significantly in length. This segment generally corresponds to the linear segment of a standard curve. We identify the intensity _{L }of the lower pair as the smallest intensity such that 95% UB(_{L}) is less than 95% UB(_{L}. Similarly, we identify _{U }as the largest intensity such that 95% LB(_{U}) is greater than 95% LB(_{U}. We define _{L }and _{U }to be _{L }= _{L}) and _{U }= _{U}), respectively. We believe that this is a conservative approach to identifying intensities that generate concentration estimates with acceptable errors.

An informative visualization of acceptable concentration estimates may be generated using the points (_{L}, _{L}) and (_{U}, _{U}). Consider the union of the two rectangular regions defined by the two sets of vertices [(_{L}, 0), (_{L}, _{L}), (_{U}, _{U}), (_{U}, 0)], and [(0, _{L}), (0, _{U}), (_{U}, _{U}), (_{U}, _{LU})]. This union defines an L-shaped region covering the standard curve segment and bound at its extremes by the intensity and concentration segments. From this visualization, one can quickly grasp the dynamic range of acceptable intensities and the potential range of acceptable concentration estimates.

In regard to this first panel, two notable aspects of this propagation of error methodology are noteworthy. First, the error bands are computed pointwise and provide reasonable error estimates for a small number of concentrations. As the number of concentration estimates grows, the impact of the multiple testing problem grows

The second panel in this display shows the concentration coefficient of variation – that is, %

The third panel in this display features an annotated histogram of sample spot intensity estimates on the intensity axis opposite the scatterplot. In this representation, it is easy to see the extent of overlap between the distribution of sample intensity estimates and the range of intensities that result in concentration s estimates with acceptable errors.

Results and discussion

To evaluate concentration estimation errors in the example analysis, we attempted to quantify or understand those errors that we can and minimize those errors that we cannot. We began with data screening. The most significant anomaly uncovered during this exploratory analysis of the cancer biomarker data was a decreasing trend in control spot intensity as a function of array print order (Figure _{e}(fluorescent intensity) the difference between the mean of its slide's control spot log_{e }(intensities) and the corresponding loess estimate.

Control spot fluorescence and slide print order

**Control spot fluorescence and slide print order **We expect control spot intensity to be constant across slide print order. The trend, estimated with loess regression, indicates a processing effect due to printing order. In this case, the decrease in control spot intensity, coupled with knowledge of the printing process, suggests that the quantity of printed material is decreasing over a printing run.

Our evaluation addressed the selection and fitting of an acceptable concentration estimation model. To that end, we examined two plots. The first displays the fluorescent intensities of the standards as a function of concentration (Figure

Spot fluorescence and standard concentration

**Spot fluorescence and standard concentration **A graph of spot fluorescence as a function of standard concentration (A) shows that both the difference in concentration and the variability in spot fluorescence increase with concentration. Both negatively affect model fitting and subsequent statistical inferences due to the excessive leverage of the larger concentrations and the heteroskedasticity of the spot intensities. In this case, log_{e }transforms of both concentration and spot fluorescence reduce the excessive leverage and improve the homogeneity (B). The log transformation is consistent with the featured concentration dilution series and is a common transformation of chemical measurements that often are assumed log normally distributed.

The first is heteroskedasticity, or the increasing variation in fluorescent intensity with increasing concentration. Meaningful statistical inferences about concentration estimation errors depend upon correct modeling assumptions. To apply propagation of error when estimating and then interpreting the approximate 95% confidence intervals, we rely on normal distribution theory and require that the random variability in spot intensities be homogeneous across concentrations _{e }transformation of the intensity estimates stabilizes the variability across concentrations (Figure

The second characteristic is the undue leverage of data at high concentrations due to the increasing separation between standard concentrations with increasing concentration. Although both are expected (the first due to the randomness generally observed when counting photons, and the second due to the use of a concentration dilution series in the design), each must be addressed to achieve the best fit of the standard curve and resulting concentration estimation inferences. A log_{e }transformation of the concentrations standardizes the separation in concentrations (Figure

With the heteroskedasticity and undue leverage addressed, we estimated a standard curve by selecting one of two models: a four-parameter logistic model (Eqn. 1) and a power curve model (Eqn. ??). We chose the logistic model as the model that fits the data best visually and in terms of the PRESS statistic (Figure

Candidate standard curves

**Candidate standard curves **A review of the graphs of a four-parameter logistic curve (A) and a power law curve (B) suggests that the former shows higher fidelity to the data and is a better choice for the standard curve in this case.

Modeling diagnostics

**Modeling diagnostics **The points in graphs of the standardized residuals versus the log_{e }standard concentrations (A) and the standardized residuals versus the estimated log_{e }spot intensities (B) are reasonably well behaved, showing no strong systematic trends or deviations from the zero line. These indicate that the four-parameter logistic model is acceptable, that subsequent statistical inferences are reasonable, but that a better model may exist in another model family.

Figure

Three-panel diagnostic summary

**Three-panel diagnostic summary **The standard curve panel (A) of the three-panel diagnostic summary features a scatterplot of the data, the estimated standard curve (black line), the 95% confidence intervals (blue lines), and the region of acceptable errors (grey). For this example, the acceptable segment of the standard curve (i.e., the segment with concentration estimation errors acceptable to us) covers spot intensities from about 1000 to 7500 intensity units and concentrations from about 25 to 500 pg/ml. The histogram of sample intensities (B) suggests that about one-fifth of the sample spot intensities is below the detection limit of about 1000 units, about three-fifths will produce estimated concentrations in the acceptable range, and one-fifth exceeds the acceptable range. With regard to remeasuring the standards in this experiment or imaging in future experiments, the three-panel graph suggests that it may be worthwhile to attempt to extend the acceptable range. It also suggests that the normalization between the standards measurements and sample measurements should be revisited. The graph of the concentration coefficient of variation as function of concentration (C) offers an alternative summary of the estimation errors over the range of concentrations.

In the standard curve panel (Figure

The histogram (Figure

The concentration coefficient of variation, or the ratio of the concentration estimation error to the corresponding estimated concentration, offers an alternative expression to a confidence interval as a means to evaluate concentration estimation error. A graph of this estimation error as a function of concentration offers a comprehensive summary of the variation in the concentration coefficient of variation over the concentration range (Figure

Presenting the results in this type of plot allows us to immediately look for several potential problems. First, does the fitted curve seem reasonable, given the data points to which we are fitting? We also can determine whether most of the unknown sample data fall within the acceptable range of the curve. The usable concentration range is made clear and, if it is too limited in range, it is immediately apparent. If problems are identified, several fixes are available, including changing the settings on the imager or using a different concentration range to create the standard curves.

Conclusions

Evaluation of errors in estimating concentrations is important to establishing confidence in protein concentration estimates. Propagation of error provides a straightforward approach to estimating concentration estimation errors in ELISA microarray experiments. When presented in a simple multi-panel visualization, the propagated errors provide valuable information about individual concentration estimates, the applicability of the estimated standard curve, quality of the experiment, and the conduct of the ELISA microarray processing. The visualization provides a rapid assessment of the quality of the data, particularly in regard to the goodness of fit of the estimated standard curve and its capability to estimate concentrations over the observed range of intensities of biological samples.

Authors' contributions

DSD framed evaluation of ELISA microarray concentration estimation errors as a statistical problem. KKA suggested the use of propagation of error and outlined the initial derivation. DSD and AMW derived the appropriate propagation of error equations, developed the algorithms, and designed the diagnostic visualizations. AMW encoded the algorithms, analyzed the microarray imagery, and produced the statistical results. SMV designed and printed the arrays and then carried out the ELISA microarray experiments. RCZ conceived the study, designed it with SMV, and then coordinated all our efforts. All authors submitted comments on drafts, then read and approved the final manuscript.

Acknowledgements

This research was supported by the Biomolecular Systems Initiative at the Pacific Northwest National Laboratory, a multiprogram national laboratory operated by Battelle for the U.S. Department of Energy under Contract DE-AC05-76RL01830.