Email updates

Keep up to date with the latest news and content from BMC Medical Informatics and Decision Making and BioMed Central.

Open Access Technical advance

Statistical process control for data without inherent order

Alan J Poots* and Thomas Woodcock

Author Affiliations

Imperial College, London and NIHR CLAHRC for NWL, Floor 4 Lift Bank D, Chelsea and Westminster Hospital, 369 Fulham Road, London, SW10 9NH, UK

For all author emails, please log on.

BMC Medical Informatics and Decision Making 2012, 12:86  doi:10.1186/1472-6947-12-86


The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1472-6947/12/86


Received:7 February 2012
Accepted:19 July 2012
Published:6 August 2012

© 2012 Poots and Woodcock; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

The XmR chart is a powerful analytical tool in statistical process control (SPC) for detecting special causes of variation in a measure of quality. In this analysis a statistic called the average moving range is used as a measure of dispersion of the data. This approach is correct for data with natural underlying order, such as time series data. There is however conflict in the literature over the appropriateness of the XmR chart to analyse data without an inherent ordering.

Methods

We derive the maxima and minima for the average moving range in data without inherent ordering, and show how to calculate this for any data set. We permute a real world data set and calculate control limits based on these extrema.

Results

In the real world data set, permuting the order of the data affected an absolute difference of 109 percent in the width of the control limits.

Discussion

We prove quantitatively that XmR chart analysis is problematic for data without an inherent ordering, and using real-world data, demonstrate the problem this causes for calculating control limits. The resulting ambiguity in the analysis renders it unacceptable as an approach to making decisions based on data without inherent order.

Conclusion

The XmR chart should only be used for data endowed with an inherent ordering, such as a time series. To detect special causes of variation in data without an inherent ordering we suggest that one of the many well-established approaches to outlier analysis should be adopted. Furthermore we recommend that in all SPC analyses authors should consistently report the type of control chart used, including the measure of variation used in calculating control limits.

Keywords:
Statistical process control (SPC); Individual and moving range (XmR); Ordering of data

Background

Statistical process control (SPC) is an approach to quality improvement that has seen increasing use in healthcare since the early 1990s [1]. Originated by Shewhart [2], SPC provides analytical tools to understand the variation displayed by measures of quality, and an approach to taking action on the resulting information with a view to making improvements. The control charts that form the mainstay of SPC analysis provide a simple graphical approach to understanding variation. Following Shewhart’s initial work, SPC was subsequently developed by Shewhart and Deming [3], and substantial literature now exists, including on the application of SPC in healthcare [4-10].

The most natural application of SPC in healthcare is to time series data - the natural ordering of the data in time is central to the correct application of the analysis. However, in the SPC literature there are conflicting opinions on the usage of control charts for data that does not come endowed with a natural ordering. Some authors recommend the use of control charts for such data [4], and some have used this analysis, for example in comparing hazard ratios for specific mortality rates [8]. Other authors argue against the use of control charts in such situations [9]. In this article we resolve objectively and quantitatively the question of whether it is acceptable to use control charts to analyse variation in data that does not have a natural ordering.

As all measured data exhibit variation, the idea behind a control chart is to provide concrete rules to assess the likely nature of the observed variation. Broadly speaking, the observed variation is classified as either “common cause” variation or “special cause” variation. Common cause variation is the variation exhibited by a process in its usual state, whereas special cause variation is caused by an exceptional or external event. The rules are couched in terms of a number of horizontal lines - the process (or control) limits - marked on a line graph of the data. The calculations of these features depend on which type of chart one is using in the SPC setting, and there are several available. The most commonly used are the individual values and moving range (XmR) charts; p-charts (used to monitor the proportion of faults in a sample); np-charts (an adaptation of the p-chart used to interpret performance in numbers of units rather than proportion); c-charts (to monitor count data - number of faults per unit - or to monitor the total number of events occurring over certain unit of time); and u-charts (for monitoring count data with the sample size greater than one, i.e. the average number of faults per unit). The XmR chart is one of the simplest of the charts to construct, and yet also one of the more robust in general practice, as the other charts rely on the data conforming to an assumed distribution. P- and np-charts rely on the binomial distribution; whilst c- and u-charts rely on the Poisson distribution. The XmR chart makes no such assumption and instead uses the data themselves to provide empirical limits through calculation of an average moving range; whilst, for example, the p- and np-charts assume the variation to be a function of the location and plot theoretical limits that will not hold if the binomial assumption is violated. Technical details of the different types of control chart and the relevant assumptions can be widely found, e.g.[3,4].

In healthcare, more so than in the manufacturing birthplace of SPC, we will seldom be in a position to justify stringent assumptions, such as those of the binomial model, satisfactorily. The simplest control chart, the XmR chart also has the distinct advantage of having the least stringent assumptions attached to it. In fact the only assumption required is that a rational sampling and sub-grouping regime is used [11]. In this sense rational means taking into account the context for the data, sources of variation, and the questions to be addressed by the charts. Thus in the complex real world of healthcare, the robustness of the XmR chart to distribution of the data is invaluable. Furthermore, even if the assumptions of a specific model do hold, in most cases the XmR chart will yield identical results to the more restrictive chart [4]. With this in mind, for the rest of this article we will focus on the XmR chart.

For the XmR analysis of data with a natural ordering, it is important that global measures of dispersion, such as the overall standard deviation, are not used to in the calculation of control limits [12]. This is because such a global measure only makes sense in the context of an assumption that the data is homogeneous; whilst the primary question that the control chart is designed to answer is precisely this: is the data homogeneous, or are there signals within of heterogeneity – “special causes”. Instead the correct method for calculating the control limits for an XmR chart is via the average moving range [2,7,11]. This subtle distinction is of fundamental importance in the correct application of the methodology of SPC.

Whilst the XmR chart was originally formulated with time-series data in mind, its use has been advocated for data in which there is logical comparability but no inherent ordering of the data, provided the order in which the data is placed is not determined by the data themselves [4]. In this article we will explore quantitatively the consequences of the lack of natural ordering for the average moving range, both theoretically and via an example using real world data. We then discuss alternative approaches to the detection of special causes for data without a natural order.

Methods

There are SPC charts for which permuting the order of the data does not affect the calculation of the control limits. These include the p-charts, np-charts, c-charts and u-charts, but as mentioned above, these charts rely on stringent assumptions that are unlikely to be met by real healthcare data. However, the control limits of the XmR chart are affected by permutations of the data, as is shown below.

Suppose that the data we are interested in, <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M1','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M1">View MathML</a> , do not possess a natural order. We may assume that the data are labelled in a non-descending order, i.e. that <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M2','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M2">View MathML</a>. By choosing an ordering of the data, say y1,y2,…,yn; and then applying the method outlined in [4], we obtain a value for the average moving range <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M3','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M3">View MathML</a>:

<a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M4','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M4">View MathML</a>

(1)

There are at most n! distinct possible orderings of the data, giving at most <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M5','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M5">View MathML</a> possible values for <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M6','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M6">View MathML</a> (the average moving range is invariant under a reversal of the order). There is, therefore, an order that results in the largest possible value of <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M7','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M7">View MathML</a>, and an order that results in the smallest. It is of interest to calculate these extrema in order to understand the appropriateness of the average moving range as a measure of variation in the data.

The minimum <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M8','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M8">View MathML</a>, <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M9','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M9">View MathML</a>, is easily seen to be <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M10','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M10">View MathML</a> - simply the difference between the smallest datum and the largest, averaged over the n-1 moving ranges. The maximum, <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M11','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M11">View MathML</a> , is somewhat more difficult to establish. To describe the maximum we must introduce some notation. Let <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M12','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M12">View MathML</a> for j = 1,2,…,n-1. Also, let <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M13','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M13">View MathML</a> and <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M14','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M14">View MathML</a>. The cases of n odd and even have to be considered separately.

Case I:n is odd

It can be shown that, if <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M15','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M15">View MathML</a> then

<a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M16','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M16">View MathML</a>

(2)

and if <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M17','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M17">View MathML</a>, then

<a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M18','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M18">View MathML</a>

(3)

Note that if the two central δ s are equal, these two expressions are the same.

Case II:n is even

It can be shown that

<a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M19','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M19">View MathML</a>

(4)

Since <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M20','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M20">View MathML</a> it is clear that the difference <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M21','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M21">View MathML</a> can be obtained by reducing all coefficients of the δj in the above expressions by one.

The distribution of the average moving ranges between the two extrema will necessarily depend on the underlying data, and the closed form distribution is not clear.

Real world example

Taking a real world research example from a quality improvement initiative, running as part of the National Institute of Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care for Northwest London (CLAHRC NWL) we investigated the consequences of using an XmR chart analysis on data that possess no underlying order, but that are logically comparable. In an improvement initiative aiming to improve ward compliance with hospital trust policy, 23 wards over 4 hospital sites spread across northwest London entered compliance data on a weekly basis to a centralised, multi-user web platform tailored to meet the project requirements [13]. Note that for these data, there is no justification for the assumption that the probability of each patient’s care being compliant within a given ward is the same, and therefore we cannot use a p-chart in this case.

We calculate average overall compliance in each ward for a year period (01-04-2010 to 31-03-2011) to allow comparison of these multiple sites, and then investigate the effect of different orderings of these data items on the average moving range through a resampling without replacement (shuffling) algorithm.

Results and discussion

The summary statistics for the set of ward percentage compliance figures over a year period are displayed in Table 1.

Table 1. Annual percentage compliance with hospital trust policy in 23 wards, rounded to 3 sf

In order to apply an XmR chart analysis to this data set, the data must be placed in a specific order. The 23 wards can be ordered in <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M22','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M22">View MathML</a> (23!) possible ways. The order in which the wards were added into the web platform as they joined the initiative was used to provide an “original” ordering, with <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M23','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M23">View MathML</a> 21.7%. By resampling with replacement (shuffling) from the population of possible orders (in MS Excel using formulae) 16,383 sample orders were generated - this large data set representing a tiny fraction <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M24','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M24">View MathML</a>of all the possible configurations. It is important to note that within each ordering, all 23 data points are present precisely once each. The average moving range for each of these resamples and summary statistics of those average moving ranges are displayed as Table 2. In addition, a histogram of the shuffled <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M25','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M25">View MathML</a> values from these was plotted as Figure 1.

Table 2. Summary statistics for the average moving ranges from the resampling exercise, to 3sf except count

thumbnailFigure 1. A histogram of the distribution of<a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M27','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M27">View MathML</a>from a resampling without replacement exercise on a real world data set of the ward compliance with hospital trust policy. This exemplifies the possible variation in the <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M28','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M28">View MathML</a> statistic that can arise when data are re-ordered, highlighting the extent of the problem when the data have no underlying order, such as time.

The formulae from the previous section show that the set of possible values of <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M29','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M29">View MathML</a> for this data has minimum 2.6% and maximum 23.1%, a range of 20.5%. This represents the degree of ambiguity in the calculation of the average moving range for this data set. The exact distribution of the <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M30','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M30">View MathML</a> values is not clear, and will depend on the underlying data. The lowest and highest values obtained in the distribution of the resamples were 6.7% and 22.8%, with the central 95% of sampled <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M31','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M31">View MathML</a> values falling between 12.8% and 21.2%. Therefore it is clear that the <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M32','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M32">View MathML</a> statistic is not robust to permuting the order of the data. It is now possible to examine the consequences of this fact for the control limits.

Calculating the control limits using <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M33','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M33">View MathML</a> yields 26.8% and 40.6% for the upper and lower limits respectively (width 13.8%). Similarly, calculating using <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M34','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M34">View MathML</a> yields −27.7% and 95.2% (width 123%). These control limits can be seen in relation to the data in Figure 2 – note that the negative lower control limit is replaced by 0% in interpreting the chart in practice. As a consequence of simply permuting the order of the data, an absolute difference of 109 percent in the width of the control limits can be effected – 1.87 times the actual range of the data. Since the control limits are intended to convey information about the process that generated the data, it is clear that this level of ambiguity in the analysis renders it unacceptable as an approach to making decisions based on the data.

thumbnailFigure 2. An Average and SD chart of the dataset, with Tukey’s Fences superimposed; as well as control limits based on<a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M37','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M37">View MathML</a>and<a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M38','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M38">View MathML</a>. This highlights the problem of using <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M39','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M39">View MathML</a> for datasets with no inherent order. Here the data are placed in “original” order. Limits falling outside of the range 0%-100% are not plotted. This shows the consequences of the issue highlighted in Figure 1, and suggests that one should instead fall back on classical outlier analyses in these instances.

Since the XmR chart is not an appropriate way to analyse this data, how should one attempt to distinguish special causes from common causes in this case? Without a natural ordering, the problem becomes one of outlier detection, and an appropriate technique may be selected from the well developed literature on this issue [14]. Examples of the simplest outlier detection methods include: the 3-sigma rule derived from the properties of the normal distribution (or more generally using the Vysochanskii-Petunin inequality [15] for unimodally distributed data) – equivalently an average and standard deviation chart [11]; and Tukey’s method of “fences” [16]. Neither of these depend upon the ordering of the data. In this case, as the data is skewed, it is appropriate to apply Tukey’s method, in other words to define lower and upper “fences” at Q1-k(Q3-Q1) and Q3+ k(Q3-Q1) respectively. Here Q1 and Q3 are the lower and upper quartiles of the data. Values of k = 1.5 and k = 3 are often taken to define “inner fences” and “outer fences”, these are plotted in Figure 2. Examining Figure 2 it is apparent that the number of points that should be investigated as potentially due to special causes using <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M40','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M40">View MathML</a> XmR control limits, <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M41','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M41">View MathML</a> XmR control limits, inner fences, outer fences are 14, 0, 4 and 1 respectively, out of 23 data points. This provides another means of understanding the lack of robustness of the XmR chart analysis to permutation of this data.

Conclusions

In conclusion, usage of p-, np-, c- and u-charts for data without natural ordering proceeds precisely as for data endowed with a natural ordering such as time. This is not the case for the simplest but more distribution-robust control chart, the XmR chart. The control limits on an XmR chart are dependent on the ordering of the data, and this dependency is such that the ambiguity in “expected variation” (as quantified by the range of possible widths of the control limits) is large when working with data that have no inherent natural order. We have given a real data set for which this range is almost double the range of the actual data – clearly an unacceptable degree of ambiguity.

Thus when one is faced with a problem of distinguishing special from routine variation in a univariate data set with no time order, the individuals and moving range (XmR) chart is not appropriate, and simply using a random order that is not based on the magnitude of the values, as advocated in primer texts [4,11,12] is not sufficient to address this issue. In this case one should fall back on the usual outlier analyses available to the statistician, e.g.[14]. In the example data above we have applied the well known and simple method of “fences”, due to Tukey [16]. In practice, the choice of outlier detection methodology will depend on the particular application.

As such, for identification of potential special causes in a dataset we recommend that:

1) In time series data when there is limited or no knowledge of the distribution of the data, the XmR chart is the appropriate method of analysis, using the <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M42','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M42">View MathML</a> for construction of limits, thereby accounting for the underlying data order.

2) In data without a natural order, an appropriate outlier detection method should be selected instead of using XmR - some simple examples being a) Tukey’s method of “fences” b) the 3 sigma rule (note this corresponds to using an average and standard deviation chart). See [14] for further methods of outlier detection.

3) Authors should explicitly state the method used, including how control limits were calculated.

Abbreviations

CLAHRC NWL, Collaboration for Leadership in Applied Health Research and Care for Northwest London; NIHR, National Institute for Health Research; <a onClick="popup('http://www.biomedcentral.com/1472-6947/12/86/mathml/M43','MathML',630,470);return false;" target="_blank" href="http://www.biomedcentral.com/1472-6947/12/86/mathml/M43">View MathML</a>, Average moving range; SPC, Statistical process control; XmR, Individuals chart also known as X and moving range chart.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

All authors contributed to the design of the project. AP set up and implemented the resampling algorithms and wrote the first draft of the manuscript. TW determined the mathematical proofs in determination of the maxima for average moving range. All authors contributed to later drafts and gave final approval to the manuscript.

Acknowledgements

Dr Vasa Curcin for co-design and code-writing of the Web Reporting Tool. TW and AP are employed by imperial College, London and work at NIHR CLAHRC for NWL, which funded the implementation project that provided the example data set.

Disclaimer

This article presents independent research commissioned by the National Institute for Health Research (NIHR) under the Collaborations for Leadership in Applied Health Research and Care (CLAHRC) programme for North West London. The views expressed in this publication are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

References

  1. Thor J, Lundberg J, Ask J, Olsson J, Carli C, Pukk Härenstam C, Brommels M: Application of statistical process control in healthcare improvement: systematic review.

    Qual Saf Health Care 2007, 16:387-399. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  2. Shewhart W: Economic Control of Quality of Manufactured Product. D. Van Nostrand Company, New York; 1931. OpenURL

  3. Shewhart WA, Edwards-Deming W: Statistical Method from the Viewpoint of Quality Control. Dover Publications, Mineola, New York; 1986. OpenURL

  4. Wheeler DJ: Making Sense of Data: SPC for the Service Sector. SPC Press, Knoxville, Tennessee; 2003. OpenURL

  5. Carey RG, Lloyd RC: Measuring quality improvement in healthcare: A guide to statistical process control application. Quality Press, Milwaukee, Wisconsin; 1995. OpenURL

  6. Hart MK, Hart R: Statistical process control for healthcare. Duxbury Thomson Learning, Pacific Grove, California; 2002. OpenURL

  7. Wheeler DJ: Understanding Variation: The Key to Managing Chaos. 2nd Edition. 2nd edition. SPC Press Inc., Knoxville, Tennessee; 1995. OpenURL

  8. Mohammed MA: Using statistical process control to improve the quality of health care.

    Qual Saf Health Care 2004, 13:243-245. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  9. Woodall WH: Use of control charts in health-care and public-health surveillance.

    Journal of Quality Technology 2006, 38:89-104. OpenURL

  10. Tsui KL, Chiu W, Gierlich P, Goldsman D, Liu X, Maschek T: A review of healthcare, public health, and syndromic surveillance.

    Quality Engineering 2008, 20:435-450. Publisher Full Text OpenURL

  11. Wheeler DJ: Understanding Statistical Process Control. SPC Press, Knoxville, Tennessee; 1992. OpenURL

  12. Wheeler DJ: Advanced Topics in Statistical Process Control: The Power of Shewhart's Charts. 2nd edition. SPC Press, Knoxville, Tennessee; 2004. OpenURL

  13. Collaboration for Leadership in Applied Health Research and Care for Northwest London (CLAHRC NWL): Web Reporting Tool. 2012.

    http://www.clahrc-northwestlondon.nihr.ac.uk/research-projects/web-reporting-tool webcite

    OpenURL

  14. Hawkins DM: Identification of outliers. Chapman and Hall, London, UK; 1980. OpenURL

  15. Vysochanskij DF, Petunin YI: Justification of the 3σ rule for unimodal distributions.

    Theory of Probability and Mathematical Statistics 1980, 21:25-36. OpenURL

  16. Tukey JW: Exploratory Data Analysis. Addison-Wesley Publishing Company Inc., Reading, Massachussetts; 1977. OpenURL

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1472-6947/12/86/prepub