Yitzhak Moda’i Chair in Technology and Economics, Technion—Israel Institute of Technology, Haifa 32000, Israel

Molecular & Cell Biology and Medicine, Baylor College of Medicine, Houston, Texas

Division of Infectious Diseases, Department of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania

Abstract

Background

Formulation and evaluation of public health policy commonly employs science-based mathematical models. For instance, epidemiological dynamics of TB is dominated, in general, by flow between actively and latently infected populations. Thus modelling is central in planning public health intervention. However, models are highly uncertain because they are based on observations that are geographically and temporally distinct from the population to which they are applied.

Aims

We aim to demonstrate the advantages of info-gap theory, a non-probabilistic approach to severe uncertainty when worst cases cannot be reliably identified and probability distributions are unreliable or unavailable. Info-gap is applied here to mathematical modelling of epidemics and analysis of public health decision-making.

Methods

Applying info-gap robustness analysis to tuberculosis/HIV (TB/HIV) epidemics, we illustrate the critical role of incorporating uncertainty in formulating recommendations for interventions. Robustness is assessed as the magnitude of uncertainty that can be tolerated by a given intervention. We illustrate the methodology by exploring interventions that alter the rates of diagnosis, cure, relapse and HIV infection.

Results

We demonstrate several policy implications. Equivalence among alternative rates of diagnosis and relapse are identified. The impact of initial TB and HIV prevalence on the robustness to uncertainty is quantified. In some configurations, increased aggressiveness of intervention improves the predicted outcome but also reduces the robustness to uncertainty. Similarly, predicted outcomes may be better at larger target times, but may also be more vulnerable to model error.

Conclusions

The info-gap framework is useful for managing model uncertainty and is attractive when uncertainties on model parameters are extreme. When a public health model underlies guidelines, info-gap decision theory provides valuable insight into the confidence of achieving agreed-upon goals.

Background

Public health policies affect millions of people and determine the allocation of health care funds. However, selecting an intervention for a given population at a given time is highly uncertain. Data supporting public health decisions are scarce, of poor quality, not fully generalizable and lack appropriate controls

Science-based mathematical models commonly support public health decisions

Despite severe uncertainty in public health decision-making, actions must be timely and cost-effective. Analysis of uncertainty is central in responsible decision making using uncertain data and models.

Information-gap (info-gap) theory

We develop a framework for the practical use of info-gap theory in public health for controlling infectious diseases. We focus on tuberculosis (TB) in the context of pandemic HIV as an example.

Methods

Epidemiological background

The World Health Organization reported 9.4 million incident TB cases and 1.7 million TB deaths in 2009 and estimated that only 63% of annual incident TB cases were detected and reported; of these, 86% were successfully treated

The HIV-AIDS pandemic is the major worldwide challenge to TB control

Many different epidemiological models have been used to evaluate treatment strategies. Deterministic compartment models are the most common, and we use a slightly modified version of the widely used Murray-Salomon model

Info-gap theory

The robustness function is the basic decision-support tool in an info-gap analysis. If our dynamic model were accurate we could evaluate any proposed intervention in terms of the outcome of that intervention that is predicted by the model. An intervention with low predicted TB prevalence is preferred over an intervention with higher predicted prevalence.

The problem is great model uncertainty. This means that predicted outcomes are unreliable and it is unrealistic to prioritize interventions in terms of their predicted outcomes. Using the model to find the intervention whose predicted outcome is best, is not suited to planning with highly uncertain models.

Model-based predictions are useful, but when deciding which public health intervention to implement, we should also ask: how wrong could the model be, and an acceptable outcome is still guaranteed? For any specified intervention we ask: what is the largest error in the model, up to which all realizations of the model would yield acceptable outcomes? Equivalently, what outcomes can reliably be anticipated from this intervention, given the unknown disparity between the model and reality? Answers to these questions lie in the robustness function, specified in Appendix “Definition of robustness” section. The robustness is dimensionless, and equals the greatest fractional error in the model parameters that is consistent with a specified outcome requirement. We use the robustness function to prioritize the interventions in terms of their robustness against uncertainty for achieving the required outcome.

Knight

We summarize here the main attributes of the info-gap robustness function: a plot of robustness-to-uncertainty versus required performance. This is the basic info-gap tool for prioritizing available options.

Robustness trades off against performance

More demanding performance requirements are less robust against uncertainty than less demanding requirements. This trade off is quantified and expressed graphically by the monotonic robustness curve.

Model predictions have zero robustness against uncertainty

When models are highly uncertain, it is unrealistic to prioritize one’s options based on predicted outcomes of those options, because those predictions have no robustness to errors in the underlying models. Options must be evaluated in terms of the level of performance that can be reliably achieved; this is expressed by robustness.

Combining the trade off and zeroing properties yields realistic prioritization of options.

Prioritization of options depends on performance requirements

Prioritization of options may change as requirements change. This is called “preference reversal” and is expressed by the intersection of the robustness curves of different options. Preference reversal provides insight to anomalous behavior such as the Ellsberg and Allais paradoxes in human decision making

Info-gap models of uncertainty are non-probabilistic

Info-gap robustness analysis is implementable even when probability distributions are unknown, and thus is suited to severe uncertainty. In contrast, Monte Carlo simulation, Bayesian analysis, or probabilistic risk assessment require knowledge of probabilities. Other non-probabilistic tools include interval analysis, fuzzy set theory

Info-gap is operationally distinct from the min-max or worst-case decision strategy

Info-gap robustness does not require knowledge of a worst case. When even typical scenarios are poorly characterized, it is usually impractical to characterize worst cases, which is required by the min-max strategy. Info-gap theory does require specifying acceptable outcomes. Thus it is well suited to policy making, because preferences on outcomes are the driving force.

Info-gap robustness may proxy for the probability of satisfying the performance requirement

A more robust option is often more likely to achieve the required outcome. By prioritizing the options using info-gap robustness, one maximizes the probability of satisfying the requirement, without knowing probability distributions. The proxy property is central to understanding survival in economic

Info-gap implementation

Info-gap methodology requires three main elements: a

The system model in our example is summarized in two functions.

The public health practitioner wishes to control the total number of TB cases: the fewer the better. However, trying to minimize this prevalence depends on model predictions that are highly uncertain. The performance requirement is to keep the total fraction of TB cases at a specified time, _{m}, below a critical value, _{m}, eq.(25) in Appendix “Performance requirements” section.

Grassly

A dominant uncertainty in TB dynamics with HIV prevalence is in model parameter values, though HIV causes significant uncertainties in model structure. Structural uncertainty refers to missing terms in the equations, missing equations, or unknown nonlinearities. Structural uncertainty is dealt with much less frequently than parameter uncertainty because of technical challenges. We focus on parameter uncertainty in this paper because of its importance and to facilitate the presentation of this first application of info-gap theory to public health.

We use info-gap theory

We aim to achieve the performance requirement by judicious choice of control variables, defined in Appendix “Control variables” section. Eligible control variables are any coefficients of the dynamic model that can be influenced by public health or related medical intervention. We use the diagnosis rate, cure rate, relapse rate, and HIV infection rate. We define an intervention in terms of the values of these variables

Results: robustness and policy evaluation

We use the info-gap robustness function to evaluate alternative interventions aimed at controlling the relative TB prevalence, _{m}, in the future. An intervention is specified by the values of the control variables. The evaluation leads to realistic assessment of outcomes and preferences among the interventions.

Interpreting robustness curves: trade off and zeroing

All info-gap robustness curves have two properties, mentioned earlier: trade off between performance and robustness, and zeroing of the robustness curve. These properties are central in using robustness curves to evaluate public health policy.

The coefficients of the epidemiological models are specified in Tables _{m}=10 years after initiation unless indicated otherwise.

Robustness of relative TB prevalence

**Robustness of relative TB prevalence.** Run 8.

**Symbol**

**Definition**

**HIV neg**

**HIV pos**

**Model parameters** in the Murray-Salomon basic model, Table Two, p.42, in ref.

^{a}Footnote 8, p.24,

^{b}Footnote e, Table A5, p.63,

^{c}Table A5, p.63,

^{d}Footnote c, Table A5, p.63,

^{e}Footnote

^{f}Footnote b, Table A5, p.63,

^{g}Rate: per person per year. In Botswana the average is 477 cases per year per 100,000 people. 62% of them are HIV infected.

^{h}Depends on HIV prevalence. In areas with HIV and without preventive treatment, 25% of babies born from HIV mothers are infected.

^{i}In Botswana.

^{j}[Dye C, Scheele S, Dolin P, Pathania V, Raviglione MC. Consensus statement. Global burden of tuberculosis: estimated incidence, prevalence, and mortality by country. WHO Global Surveillance and Monitoring Project.

^{k}Not needed since

^{ℓ}Decision variable.

^{m}Can also be treated as a decision variable.

Birth rate ^{g}

births/year/person

0.03^{
c
}

0^{
c,h
}

population size

0.821^{
i
}

0.179^{
i
}

births per year = birth rate×^{
c
}

^{g}

infection rate

1.81×10^{−3}
^{
m
}

2.96×10^{−3}
^{
m
}

^{
k
}

# respiratory contacts with infected/person/year

^{
k
}

probability that respir. contact with infectious source leads to infection

5–15^{
a
}

^{
k
}

# infectious cases in population

^{g}

non-TB death rate

0.009^{
c
}

0.05^{
c
}

proportion of new infections entering slow breakdown

0.9 (0.85–0.95)^{
a
}

0.4 (0.3–0.5^{
a
})

_{
F
}
^{
g
}

fast breakdown rate

2^{
c
}

3

_{
S
}
^{g}

slow breakdown rate

0.001 (5–15×10^{−4}
^{
a
})

0.075 (0.05–0.10^{
a
})

^{g}

rate of application of INH to infected individuals

0.75 (^{
ℓ
})

protection from superinfection conferred by primary infection

0.75 (0.5–1^{
a
})

short-term INH effectiveness

0.7

long-term INH effectiveness

0.7

^{
i,j
}

proportion of pre-diagnosed cases in clinical category

^{1,1}

0.45 (0.4–0.5)^{
j
}

0.35 (0.3–0.4)^{
j
}, ^{
e
}

^{2,1}

0.55 (0.5–0.6)

0.65 (0.6–0.7) ^{
e
}

^{3,1}

^{
e
}

^{
i,2}

^{
i,2}=1−^{
i,1}

^{
f
}

^{
i
}

proportion of new cases in clinical category

^{1}, ^{2}

proportion of new cases in clinical category

0.45 (0.4–0.5^{
a
})

^{3}

proportion of new cases in clinical category

^{3}=1−^{1}−^{2}
^{
d
}

^{
j
}
^{g}

diagnosis rate for category

0.6 ^{
ℓ
}

0.6 ^{
ℓ
}

^{g}

smear conversion rate

0.03 ^{c}

**Symbol**

**Definition**

**HIV neg**

**HIV pos**

**Model parameters** in the Murray-Salomon basic model, Table Two, p.42, in ref.

^{a}Footnote 8, p.24,

^{b}Footnote e, Table A5, p.63,

^{c}Table A5, p.63,

^{d}Footnote

^{e}Footnote d, Table A5, p.63,

^{f}Rate: per person per year.

^{g}
^{
a
}.

^{h}

_{
U
}
^{
f
}

spontaneous cure rate for untreated cases

0.2 (0.14–0.25)^{
g
}

^{
f
}

TB death rate for untreated cases in clinical category

0.12 (0.075–0.20^{
a
})

0.45 (0.3–0.6^{
a
})

^{
b
}

^{
b
}

^{
i,k
}

proportion of treated cases in clinical category

^{1,1}

0.5 ^{
d
}

^{2,1}

0.28 ^{
d
}

^{3,1}

^{
d
}

^{
i,2}

^{
i,2}=1−^{
i,1}
^{
e
}

^{
f
}

cure rate for treated case in treatment category

0.8

0.5^{
c
}

^{
f
}

TB death rate for treated cases in clinical category

0.075^{
c
}

0.16^{
c
}

0.12^{
c
}

0.24^{
c
}

^{
b
}

^{
b
}

_{
U
}

proportion of spontaneously recovered cases entering the slow relapse category

0.009^{
c
}

proportion of recovered cases from treatment category

0.0096^{
c
}

0.0094^{
c
}

_{
F
}
^{
f
}

fast relapse rate

2^{
c
}

3

_{
S
}
^{
f
}

slow relapse rate

0.001 (5–15×10^{−4}
^{
a
})

^{
f
}

rate of HIV infection

0.075 (0.011–0.95)^{
h
}

**Symbol**

**Low TB Prevalence**

**Medium TB Prev.**

**High TB Prev.**

**Low**

**Med**

**High**

**Low**

**Med**

**High**

**Low**

**Med**

**High**

**HIV**

**HIV**

**HIV**

**HIV**

**HIV**

**HIV**

**HIV**

**HIV**

**HIV**

1

0.9

0.8

0.6

0.8

0.650

0.500

0.6

0.5

0.4

2

_{
F
}

0.0075

0.015

0.03

0.011

0.018

0.028

0.03

0.0375

0.045

3

_{
S
}

0.03

0.06

0.12

0.05

0.088

0.125

0.12

0.15

0.18

4

_{
F
}

0.003

0.006

0.012

0.01

0.018

0.025

0.012

0.015

0.018

5

_{
S
}

0.009

0.018

0.036

0.02

0.035

0.050

0.036

0.045

0.054

6

(

0.005

0.01

0.02

0.008

0.014

0.020

0.02

0.025

0.03

7

(

0.002

0.004

0.008

0.003

0.005

0.008

0.008

0.01

0.012

8

(

0.002

0.004

0.008

0.003

0.005

0.008

0.008

0.01

0.012

9

(

0.002

0.004

0.008

0.003

0.005

0.008

0.008

0.01

0.012

10

(

0.001

0.002

0.004

0.002

0.004

0.005

0.004

0.005

0.006

11

(

0.001

0.002

0.004

0.002

0.004

0.005

0.004

0.005

0.006

12

(

0.01

0.02

0.04

0.02

0.035

0.050

0.04

0.05

0.06

13

(

0.005

0.01

0.02

0.01

0.018

0.025

0.02

0.025

0.03

14

(

0.005

0.01

0.02

0.01

0.018

0.025

0.02

0.025

0.03

15

(

0.005

0.01

0.02

0.02

0.035

0.050

0.02

0.025

0.03

16

(

0.002

0.004

0.008

0.005

0.009

0.013

0.008

0.01

0.012

17

(

0.0025

0.005

0.01

0.003

0.005

0.008

0.01

0.0125

0.015

18

_{
F
}

0.002

0.004

0.008

0.005

0.009

0.013

0.008

0.01

0.012

19

_{
S
}

0.006

0.012

0.024

0.015

0.025

0.038

0.024

0.03

0.036

1.000

1.000

1.000

1.000

1.000

1.000

1.000

1.000

1.000

**Symbol**

**Nominal value,**

**Error weight,****
s
**

1.81×10^{−3}

0.0009

2.96×10^{−3}

0.0018

0.075

0.2

_{
S
}

0.001

0.0005

0.001

0.001

_{
F
}

2

1

3

1.5

Figures

Relative TB prevalence vs. time

**Relative TB prevalence vs. time.** Run 8.

Relative relapses vs. time

**Relative relapses vs. time.** Run 8.

Trade off

Key to understanding the trade off expressed by the robustness curve is the concept of satisficing. In contrast to optimizing, satisficing asks for an outcome that meets minimal needs but may not be the best imaginable. The satisficing strategy is not merely “accepting second best.” Satisficing is aspirational, setting a goal just like optimization, but also requiring robustness to uncertainty. The satisficing strategy induces a trade off between the aspiration for good outcome and the robustness against uncertainty in attaining that outcome.

The robustness curve in Figure _{m}. Figure

We can interpret the numerical values along the robustness curve as follows. The prevalence, _{m}, are normalized to the initial population size. For instance, _{m}=0.025 means that the prevalence at time _{m} must not exceed 2.5% of the initial population size. The robustness corresponding to this value of _{m}, is 0.1 as seen in Figure

The public health practitioner may feel that robustness to 10% uncertainty in the model parameters is rather small, given the substantial uncertainty in the epidemiological dynamics of TB with HIV prevalence. If we want robustness to, say, 25% uncertainty in the model parameters we must accept a larger final case load, namely, _{m}=0.033 as seen in Figure

Zeroing

We note that the robustness curve in Figure _{m}=0.021. This means that requiring the prevalence not to exceed 2.1% of the initial population has no robustness against model uncertainty. The value of _{m} at which the robustness becomes zero is precisely the nominal prediction of the prevalence at time _{m} as seen by the right end-point in Figure _{m}), evaluated with the best estimates of the model parameters, equals 0.021. The horizontal intercept in Figure

It is not surprising that the predicted outcome is extremely vulnerable to error in the model upon which the prediction is based. However, the zero-robustness of predicted outcomes has an important implication for policy selection.

The robustness curve in Figure _{m}=10 years does not reliably reflect the performance of these control variables. Due to the trade off property, only larger prevalence can reliably be expected to result from this choice of the control variables. Predicted outcomes are not reliable for prioritizing the interventions.

Equivalent interventions

Different combinations of interventions can yield essentially equivalent results, as in Figure

Equivalent robustness for two interventions

**Equivalent robustness for two interventions.** Run 8: —, run 15: – –.

**Run**

**Init**

**
t
**

**
ρ
**

**
λ
**

**
γ
**

**
β
**

**Prev**
^{
a
}

**(yr)**

^{a}Data-column in Table

8

1

10

(0.6, 0.6)

(0.8, 0.5)

2

3

0.00181

0.00296

0.075

2

3

9

1

10

(0.65, 0.65)

(0.8, 0.5)

2

3

0.00181

0.00296

0.075

2

3

10

1

10

(0.65, 0.65)

(0.88, 0.55)

2

3

0.00181

0.00296

0.075

2

3

11

1

10

(0.65, 0.65)

(0.88, 0.55)

1.5

2.25

0.00181

0.00296

0.075

2

3

12

1

10

(0.65, 0.65)

(0.88, 0.55)

1

1.5

0.00181

0.00296

0.075

2

3

15

1

10

(0.85, 0.85)

(0.8, 0.5)

1.2

2

0.00181

0.00296

0.075

2

3

19

5

10

(0.6, 0.6)

(0.8, 0.5)

2

3

0.00181

0.00296

0.075

2

3

21

5

10

(0.65, 0.65)

(0.8, 0.5)

2

3

0.00181

0.00296

0.075

2

3

22

5

10

(0.65, 0.65)

(0.88, 0.55)

2

3

0.00181

0.00296

0.075

2

3

23

5

10

(0.65, 0.65)

(0.88, 0.55)

1

1.5

0.00181

0.00296

0.075

2

3

20

9

10

(0.6, 0.6)

(0.8, 0.5)

2

3

0.00181

0.00296

0.075

2

3

24

9

10

(0.65, 0.65)

(0.8, 0.5)

2

3

0.00181

0.00296

0.075

2

3

25

9

10

(0.65, 0.65)

(0.88, 0.55)

2

3

0.00181

0.00296

0.075

2

3

26

9

10

(0.65, 0.65)

(0.88, 0.55)

1

1.5

0.00181

0.00296

0.075

2

3

27

1

20

(0.6, 0.6)

(0.8, 0.5)

2

3

0.00181

0.00296

0.075

2

3

28

1

30

(0.6, 0.6)

(0.8, 0.5)

2

3

0.00181

0.00296

0.075

2

3

29

1

10

(0.6, 0.6)

(0.8, 0.5)

2

3

0.00181

0.00296

0.0375

2

3

30

1

10

(0.6, 0.6)

(0.8, 0.5)

2

3

0.00181

0.00296

0.05

2

3

31

1

10

(0.6, 0.6)

(0.8, 0.5)

2

3

0.00181

0.00296

0.06

2

3

32

1

10

(0.6, 0.6)

(0.8, 0.5)

2

3

0.0009

0.00148

0.075

2

3

33

1

10

(0.6, 0.6)

(0.8, 0.5)

2

3

0.0003

0.0005

0.075

2

3

38

1

30

(0.85, 0.85)

(0.8, 0.5)

2

3

0.00181

0.00296

0.075

2

3

39

1

10

(0.6, 0.6)

(0.8, 0.5)

2

3

0.00181

0.00296

0.075

1

1.5

41

1

10

(0.65, 0.65)

(0.8, 0.5)

2

3

0.00181

0.00296

0.075

1

1.5

Robustness curves at 10, 20 and 30 years

**Robustness curves at 10, 20 and 30 years.** Run 8: —, run 27: – –, run 28: ·–.

Figure _{m}=0.018. These predictions result from estimated model parameters, so one might be inclined to conclude that TB prevalence of 0.018 can be achieved at either 10 or 30 years by using the corresponding intervention.

Nominal equivalence of two interventions

**Nominal equivalence of two interventions.** Run12: —, run 38: – –.

However, the epidemiological model is highly uncertain, and the robustness curves in Figure

Impact of initial TB and HIV prevalence

We now consider higher initial prevalences. The overall shape of the dynamic response is very similar in each case, except that the prevalence increases significantly as the initial prevalence increases. As in Figures

Figure

Robustness curves for low, medium and high initial TB and HIV prevalence

**Robustness curves for low, medium and high initial TB and HIV prevalence.** Run 8: —, run 19: – – run 20: ·–.

Intervention aggressiveness

Figure

Robustness with varying aggressiveness

**Robustness with varying aggressiveness.** Run 8: —, run 9: – –, run 10:·–, run 12: ⋯.

The progression from solid to dash to dot-dash in Figure

The top curve in Figure

Different target times

Most of the results discussed so far evaluated the robustness for a target time 10 years after initiation. We now consider the implications of different target times.

Figure _{m}, of 10, 20 and 30 years (solid, dash, dot-dash respectively). The initial prevalences of TB and HIV are low. The interventions are all at the baseline.

The predicted prevalence decreases as the target time increases, as shown by the horizontal intercepts in Figure

From Figure _{m} less than 3%, the 30-year TB prevalence is more robust than the 20-year prevalence which is more robust than the 10 year prevalence. For instance, at critical TB prevalence of _{m}=0.02, the robustnesses for 10, 20 and 30 year horizons are 0, 0.08 and 0.12, respectively. This intervention has no robustness to uncertainty when requiring a 2% prevalence after 10 years; in fact, the estimated prevalence at 10 years is greater than 2%. The prevalence at 20 years will be no worse than 2% provided that the model coefficients err by no more than 8%, and at 30 years the robustness to error is 12%.

The practitioner may feel that even 12% robustness against model-coefficient error is rather small, given the severe uncertainty of TB epidemiology in the context of epidemic HIV. This means that, even at a 30-year horizon, this intervention cannot reliably achieve a relative prevalence as low as 2%.

Suppose we are willing to aim at a final TB prevalence of 3.7%. We see from Figure

Results like Figure

Impact of HIV mortality

Figure

Robustness for various HIV infection rates

**Robustness for various HIV infection rates.** Run 8: —, run 31: – –, run 30: ·–, run 29: ⋯.

Conclusion

We demonstrated a generic info-gap framework for managing model uncertainty in public health decision making. By applying it to a mathematical model of TB/HIV epidemics, we illustrated specific recommendations for interventions in the control of TB with HIV in various settings.

The complicated multi-dimensional epidemiological dynamics are dominated by the flow back and forth between the actively and latently infected TB populations and the different rates of progression of different subpopulations between these compartments. Counter-intuitively, the total TB case load even decades after initiation can increase as a result of increased diagnosis and cure rates, and it can increase as the control of HIV becomes more aggressive. These findings highlight the critical importance of modeling in the assessment and planning of public health intervention. Model predictions are often used to choose interventions. However, model predictions must be interpreted in light of model uncertainties. Predicted outcomes have zero robustness to model error. Only worse-than-predicted outcomes (higher relative prevalence) have positive robustness against model error. This means that predicted outcomes are not reliable for prioritizing the interventions. The trade off between robustness and outcome is quantified by the info-gap model analysis and is a critical component of the decision-making process.

We explore the performance of interventions that alter the rate constants of diagnosis, cure, relapse and HIV infection. Some interventions have quite similar predicted outcomes and robustness curves. This enables the policy maker to choose between these interventions based on additional criteria, such as ease or cost of implementation. It is not true, however, that interventions with the same estimated outcomes necessarily have the same robustness against model error.

We demonstrate the policy implications of initial TB and HIV prevalence, of HIV mortality, of degree of treatment aggressiveness, and of the target time at which outcomes are evaluated. Public health policies are evaluated in terms of confidence—expressed as robustness to modeling error—in achieving specified TB prevalence at the target time. Predicted outcomes have zero robustness and thus are not reliable for evaluating and comparing interventions. Instead, interventions must be prioritized in terms of their capacity for achieving specified outcomes, with robustness to uncertainty. Failure to quantify the uncertainty inherent in public health interventions leads to disappointment from unrealized expectations, and failed policy. Where a public health model underlies guidelines, info-gap decision theory provides valuable insight into the confidence of achieving agreed-upon goals.

Appendices

The Murray-Salomon model

The Murray-Salomon (M-S) model

**Index**

**Symbol**

**Definition**

**Initial value**
^{
a
}

The definition of superscript ^{⋆}is in

^{a}As fraction of total population at start of simulation.

^{b}Low prevalence.

^{c}High prevalence.

**HIV Neg**

**HIV pos**

**Ref.**

1

Uninfected

0.9^{b}

0.2^{c}

2

_{
F
}

Infected subject to fast breakdown

0.05–0.1

0.1

3

_{
S
}

Infected subject to slow breakdown

0.1

0.1

4

_{
F
}

Superinfected subject to fast breakdown

0

5

_{
S
}

INH recipient subject to slow breakdown

0.01

0.01–0.03

6–11

Untreated cases, of 6 types:

0

^{⋆}=2 if

^{⋆}=1 if

^{⋆}=0 if

12–17

Treated cases, of 6 types:

0

18

_{
F
}

Recovered cases subject to fast relapse

0.4 (0.28–0.52)

19

_{
S
}

Recovered cases subject to slow relapse

0.050 (0.035–0.065)

The basic Murray-Salomon model: No HIV

The basic M-S model is the following 19 differential equations (eqs.(6) and (7) occur in 6 different forms each) appearing on pp.19–20 of Murray and Salomon

The term ‘±^{a}

It should be noted in the equations for ^{⋆}, where ^{⋆}=2 (smear-negative) for ^{⋆}=

However, the ‘vice versa’ is a mistake. The correct equations for

Eq.(10) states that smear-negative individuals join the smear-positive population at rate

The instantaneous rate of infection,

The HIV-Extended model

Individuals move from each category in the HIV-negative sub-model to the corresponding category in the HIV-positive sub-model at the HIV infection rate, which varies over time. Because the effects of HIV on immune function are not marked with respect to tuberculosis until the CD4 count has dropped below 500, we actually move individuals from the HIV-negative to the HIV-positive sub-model after they have been infected with HIV for 3 years. The two sub-models are also linked through the annual risk of infection, as HIV-negative tuberculosis cases can infect HIV-positive individuals, and vice versa

Our model does not delay transfer from the HIV-negative sub-model.

Let us denote the HIV-negative state variables as before, and the HIV-positive state variables with the same letters but with an over-bar. For compactness we represent these two sets of variables with two vectors:

The model parameters listed in Tables

Eqs.(1)–(9) are 1st-order linear inhomogeneous differential equations. Only eq.(1) has an inhomogeneous term: _{1}denote the 19-vector with a 1 in the first element and zeros elsewhere. We can now compactly denote eqs.(1) as:

Let

The term ‘−

M-S introduce further highly structured coupling between eqs.(17) and (18) through the TB infection rate,

Uncertainty

Many uncertainties accompany the dynamic model. We concentrate on uncertainty in the values of some of the model parameters, as this is the dominant impact of HIV prevalence. We use info-gap theory to model and manage these uncertainties

The dominant uncertain parameters are:

Let us denote uncertain variables generically as _{
i
}, compiled in a vector

For each uncertain parameter, _{
i
}, we have an estimated value, denoted _{
i
}typically chosen as half of an interval estimate of the parameter. The error estimate may be derived from a statistical confidence interval, or from a plausible extension of a confidence interval as discussed by Grassly

More precisely, the fractional error of the estimate, _{
i
}, is unknown. That is, this fractional error is bounded by a number,

But this must be further refined to reflect the fact that the uncertain parameters are 1st-order removal-rate constants^{b}, which means that they cannot be negative. Thus we adjoin these constraints to the inequality as:

Finally, we write our info-gap model of uncertainty as a family of nested sets of uncertain vectors:

In some situations one may not be able to estimate error weights, _{
i
}. In such situations the fractional error in eq.(20) can be replaced by a fractional error relative to the estimate,

Robustness: formulation

Performance requirements

We will consider an aggregated variable for monitoring the TB status of the population. Our goal is to keep the value of this variable acceptably small. The variable we consider is the total number of cases, untreated and treated, HIV-positive and HIV-negative, as a fraction of the initial population:

There are other variables that one could consider. For instance, one could consider the total number of relapses, fast and slow, HIV-positive and HIV-negative, as a fraction of the initial population:

One could also consider the instantaneous or the average rates of change of

Returning to the aggregate prevalence, _{m}. Thus the performance requirement is:

A relation such as eq.(25) is called a “satisficing” requirement, as opposed to an optimization requirement. We do not aim to minimize the aggregate prevalence, _{m}). Our goal is to make the TB prevalence adequately small: no greater than the critical value _{m}, as stated in eq.(25). Note that the satisficing requirement includes optimization as a special case. Satisficing and optimizing are the same when _{m} is chosen as the predicted minimal value.

Control variables

We aim to achieve this goal by judicious choice of control variables that we denote generically as _{
i
}, combined in a vector

We define an intervention in terms of the values of these variables. None of these control variables corresponds directly to any of the standard performance measures such as the incidence, prevalence, and death rates associated with TB. For instance, the coefficients ^{
j
}and _{
U
}(_{
T
}(

Definition of robustness

An intervention is specified by specifying the values of the control variables,

The problem is that the dynamic model is highly uncertain. This means that it is unrealistic to prioritize interventions in terms of their predicted outcomes. Since those predictions are highly uncertain, it is unwise to evaluate interventions only in terms of their model-based predictions.

The model-based predictions are useful, but we also ask: how wrong could the model be, and the predicted outcome is still acceptable? That is, for any specified intervention,

The robustness function for the performance requirement in eq.(25) is:

We can “read” this relation from left to right as follows. The robustness, _{m}, is the maximum horizon of uncertainty, _{m}. We are not ameliorating a worst case; the worst case is unknown because the horizon of uncertainty,

Endnotes

^{a}Footnote 1 in the full online version, pp. 20–21.^{b}This means that these parameters are the coefficients in equations such as ^{−ut
}. In order for this to be a removal process, the coefficient

Abbreviations

AIDS: Acquired immunodeficiency syndrome; HIV: Human immunodeficiency virus; RDM: Robust Decision Making.

Competing interests

The authors have no competing interests.

Authors’ contributions

YB-H formulated the decision analysis and implemented the calculations. CD and NZ formulated the medical model. All authors had access to all data, participated in interpreting the results of the analysis, contributed to writing the manuscript and approved the last version of the manuscript.

Acknowledgements

Financial Support: This work was supported in part by NIH grant R01AI097045. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

One author (NMZ) is in debt to the University of Pennsylvania CFAR Developmental and International Cores (NIH grant P30AI45008, Penn Center for AIDS Research) for their continuous support in this and other related studies.

Pre-publication history

The pre-publication history for this paper can be accessed here: