Centre for Bayesian Statistics in Health Economics, University of Sheffield, Sheffield, UK

Clinical Sciences, AstraZeneca R&D Charnwood, Loughborough, UK

Abstract

Background

The study of cost-effectiveness comparisons between competing medical interventions has led to a variety of proposals for quantifying cost-effectiveness. The differences between the various approaches can be subtle, and one purpose of this article is to clarify some important distinctions.

Discussion

We discuss alternative measures in the framework of individual, patient-level, incremental net benefits. In particular we examine the probability of cost-effectiveness for an individual, proposed by Willan.

Summary

We argue that this is a useful addition to the range of cost-effectiveness measures, but will be of secondary interest to most decision makers. We also demonstrate that Willan's proposed estimate of this probability is logically flawed.

Background

The study of cost-effectiveness comparisons between competing medical interventions has led to a variety of proposals for quantifying cost-effectiveness. Although the most widely used measure is still the incremental cost-effectiveness ratio (ICER), there is increasing preference for the cost-effectiveness acceptability curve (CEAC). Willan

The differences between these various approaches can be subtle, and further complexity is introduced by some authors preferring a Bayesian formulation over more traditional frequentist analysis. One purpose of this article is to clarify some important issues, concerning (a) the perspectives of different decision makers and (b) the distinction between the true value of an unknown parameter and a statistical inference about that parameter.

Discussion

We first review various approaches to measuring cost-effectiveness, including the ICER, the mean incremental net benefit, and the measure proposed by Willan

Measures of cost-effectiveness

We consider two competing treatments, drugs, or other health technologies, which we refer to as Treatment 1 and Treatment 2. Conventionally, Treatment 1 is often the standard treatment whereas Treatment 2 is a new or comparator treatment. In reality there will usually be far more than two competing treatments for any condition, but for the purpose of this article it is enough to consider, like Willan

A little notation is necessary. Let _{i} be the cost associated with an individual patient when given Treatment _{i} be the value of an appropriate effectiveness measure associated with that patient when given Treatment _{i} and _{i} are random quantities, which we interpret as the cost and effectiveness under Treatment

In order to compare cost-effectiveness between the two treatments, we require a way to link costs to effectiveness, and this is done through a decision-maker's

_{i}(_{i} - _{i}.

This expresses net benefit on the monetary scale by converting the _{i} units of effectiveness into _{i} units of money before subtracting the cost _{i}. (We could equally express net benefit on the effectiveness scale as _{i} - _{i}/

Treatment 2 would be clearly more cost-effective than Treatment 1 for an individual (random) patient if _{2}(_{1}(

_{B}(_{2}(_{1}(_{E} - _{C},

where _{E} = _{2} - _{1} and _{C} = _{2} - _{1} are the increments in effectiveness and cost, respectively.

If all patients were the same, and experienced the same costs and effectiveness, then the individual INB would be the same for all patients, and could then be called

However, patients will vary, and the consequence of this is that individual INB will vary between patients, and there is no single value to represent the comparison between the two treatments. Across the population, there is a probability distribution of individual INB.

The measures of cost-effectiveness that are in widespread use in health economics are based on the _{B}(

Δ_{B}(_{B}(_{E} - Δ_{C},

where Δ_{E} = _{E}) and Δ_{C} = _{C}) are the population mean increments in effectiveness and cost. Then Treatment 2 is defined to be more cost-effective than Treatment 1, in terms of the population mean, if Δ_{B}(

The

ρ = Δ_{C}/Δ_{E},

and we can see that Δ_{B}(_{E} > 0, or if ρ >_{E} < 0.

The probability of cost-effectiveness as proposed by Willan _{B}(

Δ_{B}(_{B}(

Two symmetric distributions of net benefit.

Two symmetric distributions of net benefit.

Thus, the distribution represented by the solid curve in Figure _{B}(_{B}(

If, however, the distribution is not symmetric, then it is quite possible for the two measures to give apparently contradictory indications of relative cost-effectiveness. Figure _{B}(_{B}(

Two skewed distributions of net benefit.

Two skewed distributions of net benefit.

Which measure is best?

It is well-known in health economics that, from the perspective of a health care provider needing to decide which treatment to apply to the population of patients in their care, it is the mean cost and effectiveness over the whole population that matters _{B}(_{B}(_{B}(

As discussed in the previous section, this can be expressed in terms of comparing the ICER ρ with _{E}.

From the perspective of a health care provider, then, needing to make a decision between two treatments, the decision rests on mean INB, and in fact only on its

Willan

The perspective of a health care provider is not necessarily the only one of interest. An individual clinician wishing to decide how to treat an individual patient may be willing to regard that patient as randomly drawn from a large population, and might be interested in θ(

Inference about cost-effectiveness

The measures of cost-effectiveness described in the preceding section are all unknown in practice because they depend on the unknown distribution of individual INBs for patients in the population. From the statistical point of view they are unknown parameters. In order to learn about them, we will need to obtain some relevant evidence. This might, for instance, as supposed in Willan

We then need to construct appropriate methods of statistical inference for parameters of interest, based on the data. There is a substantial literature on this topic. Based on data from a clinical trial, various authors have presented estimators and confidence intervals for the ICER _{E} changes, mean that inference about the mean INB is generally much more straightforward

Inference about the mean INB is generally presented by means of a _{B}(_{B}(

In its more natural Bayesian form, the CEAC states, for given _{B}(

"The interpretation that the acceptability curve is the probability that the intervention is cost-effective is not entirely accurate and could easily be misunderstood by policy makers. Consider the situation in which the observed INB for treatment is very small, but due to a very large sample size the acceptability curve at the value of λ [our

We agree that to refer to the CEAC as simply 'the probability of cost-effectiveness', or 'the probability that Treatment 2 is more cost-effective than Treatment 1', is potentially misleading if its dependence on the available evidence and on the decision context is not clear. We advocate that the phrase 'based on available evidence' should be used to emphasise the first point, or for a technical audience the Bayesian formulation of 'the posterior probability of cost-effectiveness' would be appropriate. It might be helpful also to emphasise that we are judging cost-effectiveness from the perspective of a health care provider needing to decide between two treatments, although this context has been so pervasively adopted in health economics that we believe it can be taken as understood.

Willan proposes that θ(

Willan _{B}(_{B}(

Willan's θ(

Willan's estimator

Willan proposes an estimator of θ(_{S} patients given the standard, treatment 1, and another sample of _{T} patients given the intervention, treatment 2. Now since these data do not include any observations in which the same patient is given both treatments, it is completely impossible to learn the true value of θ(_{S} and _{T} might be.

It is easy to demonstrate this impossibility with a simple example. Suppose that we have enormous samples such that we learn the true distribution in the population of costs and effects for treatment 1 and the true distribution of costs and effects for treatment 2. In particular, we will also learn the true distribution of net benefits _{i}(_{1}(_{2}(_{B}(

Even with all this information we do not know θ(_{1}(_{2}(_{2}(_{1}(_{2}(_{1}(_{1}(

Willan's estimator effectively assumes that _{1}(_{2}(_{B}(

What Willan

Summary

1. From the perspective of a health care provider needing to decide which of two treatments to fund, it is the mean cost and mean effectiveness, over the whole population of patients within the provider's remit, that are of primary concern. This leads to the mean INB Δ_{B}(_{B}(

2. Any measure of cost-effectiveness is a property of the population of patients under consideration, and is an unknown parameter. We make statistical inferences about parameters, based on available evidence. The true value of the parameter is fixed, independent of the available evidence, but unknown. Any statistical inference statement about the parameter is liable to change as the evidence changes. The CEAC plots the probability, based on available evidence, that Δ_{B}(

3. When reporting the CEAC in practice, its dependence on the data should be made clear by referring to it in such phrases as 'the probability of cost-effectiveness based on available evidence' or 'the posterior probability of cost-effectiveness'. It may also be useful to emphasise that cost-effectiveness is being judged from the perspective of a health care provider needing to decide which of two treatments to fund.

4. Willan's probability of cost-effectiveness θ(_{B}(

5. θ(

6. The proposed estimator of θ(

Conclusion

In conclusion, therefore, we reiterate the appropriateness of the CEAC as the primary comparator of relative cost-effectiveness between two treatments from the perspective of a health care provider. Willan's 'probability of cost-effectiveness' would be of only secondary value in evidence presented to policy makers, and his proposed estimator of that probability is fatally flawed. However, we agree with Willan that assessments of cost-effectiveness should be more clearly stated, avoiding the unqualified phrase 'the probability of cost-effectiveness'.

Competing interests

None declared.

Pre-publication history

The pre-publication history for this paper can be accessed here: