New page

Reference	Yang Liu. Cost-effective management of invasive species: An application of info-gap decision theory.
Publication type	PhD Thesis (157 pages)
Year of publication	2022
Downloads	Whole Thesis.
Subsections of special interest	(show/hide) 1.2.3.2 Wald’s Maximin Model & IGDT Wald’s Maximin Model considers a family of possible events, but without probability to occurrence assigned, among which the one with the worst outcome would be identified. Policy would be designed to minimize such worst case scenarios. IGDT requires initial judgement that identifies the worst acceptable outcome, and policy whose outcome is no worse than the worst acceptable outcome over the greatest range of possible contingencies would be chosen (Ben-Haim, 2019). Both of these theories are developed to support decisions against Knightian uncertainty. Wald’s Maximin Model (robust optimization) is extremely pessimistic and conservative. It may over protect us against severe uncertainty at a significant effort (Sniedovich, 2007). Rather, IGDT offers an alternative to quantify the confidence in realising specified aspirations and enables a balance between them. As a robust-satisficing decision-making method, IGDT maximizes the robustness to uncertainties while satisfying the performance requirement. The satisfactory policy results are believed to be good enough based on explicitly described sets of criteria (Yemshanov et al., 2010b). In comparison, the final decision of Wald’s Maximin Model and IGDT may not agree with each other with different initial judgements (Ben-Haim and Demertzis, 2016). Liu (2020, p. 22)   (show/hide) 1.2.3.3 Radius of stability model & IGDT Radius of stability model is used to quantify the sensitivity of system stability to perturbations in model parameters or modelling uncertainties, and further assists in identifying error bounds and accuracy of simulation (Bingham and Ting, 2013). It is the radius of the largest ball centred at the best estimate, with all elements within satisfying system stability. The larger the radius of stability of the system, the more stable it is (Sniedovich, 2010). Radius of stability model as a local robustness model was developed for modelling robustness against small deviations around the estimated parameter value (Sniedovich, 2012a). Under certainty or risk it could be informative, while under severe uncertainty it is incomprehensible (Sniedovich, 2007). This is because sensitivity around the immediate neighbourhood of a poor estimate shows no indication of sensitivities somewhere else in the uncertainty region (Hayes et al., 2013). Similarly, info-gap theory offers a method to examine the sensitivity of a model’s outcome to a hypothesis, with the horizon of uncertainty represented by unbounded and unknown α (Regan et al., 2005). Sniedovich (2012a) states that radius of stability model is actually the same as info-gap robustness model, both of which are instances of Wald’s Maximin. Burgman (2014) claims that such criticism strengthens the consistency of info-gap theory. How the rank of decisions varies is examined along the continuous and expanding horizon of uncertainty in info-gap theory. This is different from Wald’s rule which maximises horizon of uncertainty. Liu (2020, p. 22-23)
Key words	Invasive species, biosecurity, quarantine, surveillance, eradication, portfolio allocation, decision making, info-gap decision theory, robustness, opportuneness, spatial spread model, Asian house gecko, Barrow Island.
Reviewer	Moshe Sniedovich
IF-IG perspective	This thesis obscures, rather than clarify, what IGDT actually is, its capabilities and limitations, its relations to other methods, and its role and place in the state of the art in decision-making under severe uncertainty.

 

 

---

 

Information-gap decision theory creates a gap in ecological applications and then fills it

David Fox (2014)
Environmetrics Australia
Melbourne, Australia

---

 

Introduction

The title of this review is borrowed from a short piece that Professor David Fox posted on his website in 2014. You can read the full post (strongly recommended!) in the Appendix A. Here is copy of its first paragraph:

 

In their recent letter to Ecological Applications, Burgman and Regan (2014) provide counter arguments to some of Sniedovich’s (2012) severe, and mostly harsh criticisms of Ben-Haim’s info-gap decision theory (IGDT) (Ben-Haim, 2006). While I have a deep respect for Professor Burgman and Dr. Regan, I believe their unwavering faith in info-gap theory is misplaced. As the title of this note suggests, I agree with Sniedovich (2014) that ‘the gap’ referred to by Burgman and Regan (2014) is illusionary.

 

To understand the story behind the illusionary ‘gap’ that Burgman and Regan (2014) created, consider this quote from Sniedovich (2014, p. 229)

In their Letter to the Editor, Information-gap decision theory fills a gap in ecological applications, henceforth Letter, Burgman and Regan (2014) create a spurious, indeed a nonexistent gap in the state of the art, arguing that information-gap decision theory (IGDT) fills this gap. This approach to dealing with the difficulties bedeviling IGDT that are identified and discussed in the article Fooled by local robustness: an applied ecology perspective (Sniedovich 2012a), is yet another vivid illustration of the effect that the huge elephant in the IGDT room has on the manner in which proponents of IGDT discourse on this theory and related topics.

The origins of the `illusionary gap’ dates back to the early publications on IGDT, in particular to Ben-Haim (2001, 2006), where claims were made that IGDT is not only new, but also radically different from all existing methods and theories, at that time, for decision-making under uncertainty. Here is the famous statement (Ben-Haim 2001, 2006, p. xii):

Info-gap decision theory is radically different from all current theories of decision under uncertainty. The difference originates in the modeling of uncertainty as an information gap rather than as a probability. The need for info-gap modeling and management of uncertainty arises in dealing with severe lack of information and highly unstructured uncertainty.

One of the illusionary powers attributed to IGDT in applied ecology and elsewhere, is its ability to deal with unbounded uncertainty. Images like this, adapted from Halpern (2006, p. 3), distinguish IGDT from other decision theories.

Figure 1Another illusionary “axiom” in IGDT has todo with its relationship to its sole arch-rival: Wald’s maximin paradigm. The IGDT literature goes out of its way to convince you that although there are some similarities between IGDT and Wald’s maximin paradigm, the two frameworks for decision-making under severe uncertainty are different.

For example in Liu (2022, p. 11, Figure 1.7) there is a comparison between two decision-making frameworks, indicating the following:

 

e.g. Wald’s Maximin		e.g. Info-gap decision theory
1. Ameliorate the worst case/ provide insurance against the worst anticipated outcome. 2. Principle: robust optimize. 3. Confidence: not measured explicitly. 4. To choose the policy for which the contingency with the worst possible outcome is as benign as possible. 5. Prior judgment: identify a worst contingency. 6. More aggressive radical.		1. Allow decision makers to rank different portfolio in a way that it can pick those that provide satisfactory outcomes for the greatest range of adverse outcomes. 2. Principle: robust-saisficing. 3. Confidence: measured using robustness. 4. To rank: between policies of similar ambitions, those that provide the greater robustness (greater confidence) are preferred. 5. Prior judgement: specify worst tolerable outcome. 6. More or less aggressive depends very much on the decision makers’ preferences.
Figure 2

 

The Elephant in the IGDT that Sniedovich (2014a) refers to is the following result, dating back to 2006, that has been proven formally and rigorously in many publications, including peer-reviewed articles. For the purposes of this review, I call it:

The Fundamental Theorem IGDT

IGDT’s two core models, namely its robustness model (IGRM) and its robust-satisficing decision model (IRSDM), are simple maximin models.

 

Reminder

The three protagonists in the drama summarized by the Fundamental Theorem are as follows:

$$
\begin{align}
v:&= \max_{d\in D} \min_{d\in S(d)}\ \big\{ f(d,s): g(d,s) \in G, \forall s\in S(d)\big\}\tag{MaxiMin}\\
\alpha(x):&= \max_{\alpha\ge 0} \ \big\{\alpha: r(x,u) \le c, \forall u\in U(\alpha,\tilde{u})\big\} \ , \ \ x\in X\tag{IGRM}\\
\alpha^{*}: & = \max_{x\in X} \ \alpha(x)\\
& = \max_{\substack{x\in X\\ \alpha\ge 0}} \ \big\{\alpha: r(x,u) \le c, \forall u\in U(\alpha,\tilde{u})\big\}\tag{IRSDM}
\end{align}
$$

In this review we examine these models carefully and provide a formal, rigorous, step-by-step proof of The Fundamental Theorem.

In plain language, the Fundamental Theorem says that the two core models of IGDT are simple instances, that is special cases, of (MaxiMin). The implication of this theorem is far reaching in that, assuming that the Theorem is indeed valid, the claims in the IGDT literature to the contrary are false, and comparisons between them, such as the one conducted in Liu (2023), are ill-conceived in the first place, factually misleading, and technically (mathematically) wrong, as they imply that these IGDT models are not maximin models.

In my humble opinion, the most important aspect of IGDT’s illusionary gap are the persisting claims in the IGDT literature that the theory provides a reliable and tested tool for handling decision-making situations involving severe uncertainty. It is therefore important to clarify at the outset what is meant by “severe uncertainty”, or what is become more in vogue these days, “deep uncertainty”.

Severe Uncertainty a la IGDT

According to the main texts on IGDT, that is Ben-Haim (2001, 2006, 2010), the severe uncertainty that IGDT was designed to handle is characterized by the following properties:

• The uncertainty space is vast and diverse. In practice it is often unbounded.
• The point estimate of the true value of the uncertainty parameter is poor, can be substantially wrong, it is often an “educated” guess, or even just a “wild” guess.
• The uncertainty is probability-free, likelihood-free, chance-free, plausibility-free, belief-free, etc.
• There is no reason to believe that the true value of the uncertainty parameter is more/less likely to be in the immediate neighborhood of the point estimate, or in any other neighborhood in the uncertainty space.

 

In view of this, and the emphasis in the IGDT literature that the theory has no competitor on this front, it is not surprising that some IGDT scholars even claimed that the theory can handle … Unknown Unknown.

I should therefore make it crystal clear that IGDT conducts its robustness analysis, in the first instance, in the immediate neighborhood of the point estimate, and therefore it is a theory of local robustness. That is,

Fact of life No. 1 about IGDT

IGDT’s robustness analysis is inherently local in nature, that is it is conducted in the first place in the immediate neighborhood of the point estimate. Larger neighborhoods around the estimate are examined only if all smaller neighborhoods pass the worst-case test, requiring all the points in the neighborhood to satisfy the performance constraint imposed on the decision being evaluated.

 

Fact of life No. 2 about IGDT

The info-gap robustness of decision $x$, denoted above as $\alpha(x)$, is the size of the largest neighborhood around the estimate all whose points satisfy the performance constraint imposed on $x$.

 

Fact of life No. 3 about IGDT

Methodologically speaking, the strong local orientation of IGDT’s robustness analysis makes it an unsuitable, unreliable tool for the treatment of severe uncertainties of the type it was designed to handle. The more severe the uncertainty, the more unsuitable IGDT is for this purpose.

 

Fact of life No. 4 about IGDT

What emerges therefore, is that because IGDT does not have the methodological foundation to provide a reliable treatment of severe uncertainties of the type that it was designed to handle, its applicability must be assessed on a case-by-case basis.

One does not have to be an expert on decision-making under severe uncertainty to realize that the following question is inevitable:

An inevitable question about IGDT

Since IGDT is supposed to deal with uncertainties of the type discussed above, that are indeed severe, how can one justify the strong local orientation of IGDT’s robustness analysis? What is the rationale for focusing the robustness analysis on the immediate neighborhood of the estimate, knowing full well that the estimate is poor, and can be just a “wild guess”, and furthermore, knowing that there is no reason to believe that the true value of the uncertainty parameter is more likely to be in the neighborhood of the estimate than in any other neighborhood in the uncertainty space?

The issue here is clear: the local robustness conducted by IGDT is a suitable tool for determining the robustness of decisions against (small) perturbations in the value of the estimate. But this tool is definitely unsuitable for determining the robustness of decisions against the severe uncertainty under consideration. How can one justify the fact that IGDT’s robustness analysis may completely ignore large sections of the uncertainty space?

The bottom line is then this:

Bottom line rhetorical question

On what basis can one claim that IGDT is a reliable tool for measure or assess the robustness of decisions against severe uncertainties of the type that it was designed to handle, and that it is able to identify the most robust decision?

This issue leads us to a very popular model of local robustness that is discussed briefly in Liu (2023), namely the intuitive concept Radius of Stability (circa 1960). In this review I draw attention to the following implication of the Fundamental Theorem:

 

The Radius of Stability Corollary

IGDT’s robustness model, namely (IGRM) is a simple Radius of Stability model, namely a generic model of this form
$$
\rho(y):= \max_{\alpha\ge 0} \ \big\{\alpha: h(y,w) \in H, \forall w\in \mathcal{N}(\alpha, \tilde{w}) \big\}\ \ , \ \ y\in Y \tag{RoS}
$$

Note that (RoS) itself is a simple maximin model.

In summary, this review focuses on the following issues:

• The relationship between IGDT and Wald’s maximin paradigm.
• The strong local orientation of IGDT’s robustness analysis.
• The mismatch between IGDT’s local robustness analysis and the assumed severity of the uncertainty under consideration.
• The relationship between IGDT’s robustness model and the concept Radius of Stability.
• The role and place of IGDT in the state of the art in decision-making under severe uncertainty.
• The IGDT literature’s treatment of these issues.

Math Inside! The decision-making models under discussion in this review are mathematical in nature and their analysis and assessment requires the use of mathematical tools. However, I deliberately make the discussion as non-technical as possible. I should stress, though, that understanding why IGDT is not suitable for the treatment of severe uncertainty of the type it postulates does not require mathematical knowledge: common sense would suffice. Readers who are not familiar with IGDT may wish to do something supplementary reading before they read the review. A good way to start is to read Review 2.

 

The Elephant in the IGDT room

Suppose that the Fundamental Theorem of IGDT is indeed valid. Then, as in the case of the comparison shown in Figure 2, what are we to make of the following discussion in Liu (2022, p. 22) that is dedicated to a comparison between these two rivals.

 

1.2.3.2 Wald’s Maximin Model & IGDT

Wald’s Maximin Model considers a family of possible events, but without probability to occurrence assigned, among which the one with the worst outcome would be identified. Policy would be designed to minimize such worst case scenarios. IGDT requires initial judgement that identifies the worst acceptable outcome, and policy whose outcome is no worse than the worst acceptable outcome over the greatest range of possible contingencies would be chosen (Ben- Haim, 2019). Both of these theories are developed to support decisions against Knightian uncertainty.

Wald’s Maximin Model (robust optimization) is extremely pessimistic and conservative. It may over protect us against severe uncertainty at a significant effort (Sniedovich, 2007). Rather, IGDT offers an alternative to quantify the confidence in realising specified aspirations and enables a balance between them. As a robust-satisficing decision-making method, IGDT maximizes the robustness to uncertainties while satisfying the performance requirement. The satisfactory policy results are believed to be good enough based on explicitly described sets of criteria (Yemshanov et al., 2010b). In comparison, the final decision of Wald’s Maximin Model and IGDT may not agree with each other with different initial judgements (Ben-Haim and Demertzis, 2016).

 

Since Liu (2022) does not specify what Maximin Model is used for this comparison, it is impossible to comment meaningfully on the claims made in this quote. What I can say with confident is that comparisons like this betray a severe lack of appreciation of the extremely wide scope and enormous modeling flexibility of Wald’s maximin paradigm. This paradigm offers a wide range of models some of which are much more general and powerful than IGDT’s two core models.

From a methodological point of view, the flaw in the above quote, and many similar analyses in the IGDT literature, stems from an appeal to the following false conjecture:

False Conjecture

If Model A belongs to a given class of models, call it $\mathscr{C}$, and Model B is quite different from Model A, then Model B does not belong to class $\mathscr{C}$.

In particular, if Model A is a maximin model and Model B is quite different than Model A, then Model B is not a maximin model.

Had this conjecture been true, the basic concepts “class”, “prototype” and “instance” would have lost their meaning, as a class would have consisted of only one object! For example, since these two functions are so different, and may yield different results, we could not have classified them, both, as “quadratic functions”:

$$
\begin{align}
f(x) :&= Bx + Cx^{2} \\
g(x) : &= 36.7 + \frac{\,c\,}{d} x + \frac{\,a\,}{b}x^{2}
\end{align}
$$

Note that $f(0)=0$ and $g(0)=36.7$, irrespective of the the values assigned to the parameters $B$, $C$, $a$, $b$, $c$, and $d$.

The cancellation process would go like this: Suppose that $f$ is indeed a quadratic function. Since function $g$ is so different from $f$ and can yield results that are different from the results yielded by $f$, the Conjecture implies that $g$ is not a quadratic function. On the other hand, if $g$ is a quadratic function, the same argument will force us to conclude that $f$ cannot be a quadratic function.

Go figure!

In particular, we could not possibly have argued that (M-1), (M-2) and (M-3) above are all maximin models: any one of them would have “cancelled” the other two (for being different).

Now consider this:

Modeling Task

Decide whether (Mystery Model 1) that was analyzed in Davidovitch and Ben-Haim (2008), is a maximin model:
$$
\begin{align}
\hat{\alpha}(L_{c}) & = \max_{q\in \mathcal{Q}} \hat{\alpha}(q,L_{c})\tag{Mystery Model 1} \\
\text{where} &\\
\hat{\alpha}(q, L_{c}) &= \max\left\{\alpha: \left( \max_{u\in \mathscr{U}(\alpha,\tilde{u})} L(q,u) \right) \le L_{c} \right\}\tag{Mystery Model 2}
\end{align}
$$

How would you go about accomplishing this task in a formal, rigorous manner?

To help you complete this task, I display below in more detail how the model specified above as (Mystery Model 1) looks like after incorporating in it (Mystery Model 2) and simplifying a bit the expression involving the inner $\max$ operation in (Mystery 2). Here is the new look:

$$
\begin{align}
\hat{\alpha}(L_{c}) = \max_{\substack{q\in \mathcal{Q}\\ \alpha\ge 0}} \big\{\alpha: L(q,u) \le L_{c}, \forall u\in \mathscr{U}(\alpha,\tilde{u})\big\}\tag{New Mystery Model 1}
\end{align}
$$

And so

New Modeling Task

Decide whether (New Mystery Model 1) is a maximin model.

Before I suggest some modeling tips for this task, I take the liberty of recommending something that you should definitely not do, especially if you are very eager to show that (New Mystery Model 1) is not a maximin model.

Modeling Tip 1

Try very hard, in fact resist all temptations, to use the False Conjecture.

For instance, try very hard, to accomplish Modeling Task 1 by comparing (New Mystery Model 1) to this certified maximin minimax model.

$$
L^{*} = \min_{q\in \mathcal{Q}} \max_{u\in \mathscr{U}(\alpha_{m}, \tilde{u})} L(q,u)\tag{Certified MiniMax Model}\ \ \ \ \ \
$$

where $\alpha_{m}$ is a given positive number, specified by the analyst.

It will be easier to compare the two models when they are placed next to each other, like this

$$
\begin{array}{c|c}
\text{A certified Minimax model} &\text{New Mystery Model 1} \\
\hline
\displaystyle L^{*} = \min_{q\in \mathcal{Q}} \max_{u\in \mathscr{U}(\alpha_{m}, \tilde{u})} L(q,u) &\displaystyle \hat{\alpha}(L_{c}) = \max_{\substack{q\in \mathcal{Q}\\ \alpha\ge 0}} \big\{\alpha: L(q,u) \le L_{c}, \forall u\in \mathscr{U}(\alpha,\tilde{u})\big\} \tag{The Odd Couple}
\end{array}
$$

In Appendix B I explain why this comparison does not make sense if it is conducted for accomplishing Model Task 1, namely if the goal of the comparison is to determine whether (New Mystery Model 1) is, or isn’t, a maximin model. Here it is sufficient to say that for this comparison to make sense, we have to choose a maximin model that is similar to the New Mystery Model 1.

So, for reasons explained in Appendix B, I suggest the following comparison:

Note that in the context of this maximin model the control (decision) variable is $x=(q,\alpha)$, as it should be.

It should be stressed that this maximin model was not invented specifically for this occasion. Models having such a structure are present all around us, and are available, free of charge, to the general public. They may appear a bit different in terms of the notation, letters, and symbols used, but essentially they are the same. For instance, in Wikipedia we find this maximin model in the section called Constrained maximin models, where it belong.

$$
v^{*}:= \max_{d\in D}\min_{s\in S(d)} \ \big\{f(d,s): g(d,s)\le 0, \forall s\in S(d) \big\} \ \ \tag{Wikipedia Model}
$$

With the aid of the above certified maximin model can shed light on (New Mystery Model 1).

Theorem about Mystery Model 1

New Mystery Model 1, hence Mystery Model 1, is a simple maximin model.

Proof (By inspection) Consider the instance (special case} of the Certified Maximin model where $$h(q,\alpha,u) = \alpha.$$

Substituting this specification of $h$ in the Certified Maximin Model yields this maximin model:

$$
\alpha^{\circ} = \max_{\substack{q\in \mathcal{Q}\\ \alpha\ge 0}}\min_{u\in \mathscr{U}(\alpha,\tilde{u})} \big\{\alpha: L(q,u) \le L_{c}, \forall u\in \mathscr{U}(\alpha,\tilde{u})\big\}
$$

Observe that the $\displaystyle \min_{u\in \mathscr{U}(\alpha,\tilde{u})} $ operation is now superfluous, hence the model can be rewritten as follows:

$$
\alpha^{\circ} = \max_{\substack{q\in \mathcal{Q}\\ \alpha\ge 0}} \big\{\alpha: L(q,u) \le L_{c}, \forall u\in \mathscr{U}(\alpha,\tilde{u})\big\}
$$

Clearly, this is no other than the New Mystery Model 1. We therefore conclude that being an instance of a certified maximin model, the New Mystery Model 1 is also a maximin model. Specifically, it is a very simple instance of the Certified Maximin Model. This imply that Mystery Model 1 is a simple maximin model.

 

Theorem about Mystery Model 2

Mystery Model 2 is a simple maximin model.

Proof (By inspection) Consider the instance (special case} of New Mystery Model 1 where

$$\mathcal{Q}=\mathcal{Q}’=\{q’\}$$

namely the case where $\mathcal{Q}$ is a singleton. In this case the value of $q$ in the $\max_{\substack{q\in \mathcal{Q}\\ \alpha\\ge 0}}$ operation is fixed, that is, $q=q’$, and therefore its appearance under the $\max$ operation is superfluous and serves no purpose. Hence,

$$
\alpha^{\circ} = \max_{\substack{q\in \mathcal{Q}\\ \alpha\ge 0}} \big\{\alpha: L(q,u) \le L_{c}, \forall u\in \mathscr{U}(\alpha,\tilde{u})\big\} = \max_{\alpha\ge 0} \big\{\alpha: L(q,u) \le L_{c}, \forall u\in \mathscr{U}(\alpha,\tilde{u})\big\}\ \ , \ \ q=q’.
$$

This is no other than Mystery Model 2. This imply that Mystery Model 2 is a simple instance of Mystery Model 1, and since by the previous theorem Mystery Model 1 is a maximin model, it follows that so is Mystery Model 2.

We reached the climax of the story:

Proof of the Fundamental Theorem of IGDT

It is not a secrete at all that Mystery Model 1 is a clone of IGDT’s robust-satisficing decision model (IRSDM) and Mystery Model 2 is a clone of IGDT’s robustness model (IGRM). The above two theorems therefore imply that IGDT’s two core models are simple maximin models.

To prove that a given model is not a maximin model it is necessary to show that it is different from ALL maximin models, and yields different results from all of them.

On the other hand, there are two ways to prove that a given model IS a kosher maximin model:

• Showing that the given model satisfies the sufficient conditions required by the definition of what constitute a maximin model.
• Showing that the given model is an admissible instance (special case) of a “certified” maximin model.

In practice, it is not necessary to go through a formal proofing process in order to decide whether a given model is or isn’t a maximin model. This is usually done, if at all, by inspection. Regarding the first method, one way to define what constitute a maximin model is as follows:

What is a maximin model?

A maximin model is a valid mathematical transliteration of the famous Wald’s Maximin Rule (circa 1939).

There are many versions to the Maximin Rule. One of the most famous version was formulated by the American philosopher John Rawls. It reads as follows (Rawls 1971, p. 152-3):

Maximin Rule Version 1

The maximin rule tells us to rank alternatives by their worst possible outcome: we are to adopt the alternative the worst outcome of which is superior to the worst outcomes of the others.
In the context of decision-making under severe uncertainty, I prefer this slightly different phrasing.

Maximin Rule Version 2

Consider a situation where the outcome generated by an alternative depends not only on the alternative itself, but also on the realized value of the uncertain state of the process. To determine the best alternative, rank the alternatives by their security level, namely by the alternatives’ worst-case outcome. The best alternative is one whose worst-case-outcome is at least as good as the worst-case-outcome of any other alternative.

The three most popular mathematical transliterations of the Maximin Rule are as follows:

$$
\begin{align}
v:&= \max_{a\in A} \min_{s \in S} \ f(a,s) \tag{M-1}\\
v&=\max_{a\in A} \min_{s\in S} \ \big\{ f(a,s): g(a,s)\in G, \forall s\in S\big\}\tag{M-2}\\
v&=\max_{a\in A} \ \big\{ f(a): g(a,s)\in G, \forall s\in S\big\}\tag{M-3}\\
\text{where}&\\
a & = \text{alternative.}\\
A & = \text{Set of alternatives available to the decision maker.}\\
s & = \text{state variable (uncertainty parameter).} \\
S & = \text{State space (uncertainty set).}\\
f(a,s) & = \text{payoff generated by alternative $a$ when state $s$ is realized.}\\
g(a,s)\in G & = \text{constraint(s) imposed on $(a,s)$ pairs.}\\
f(a) & = \text{payoff generated by alternative, irrespective of what state is realized.}
\end{align}
$$

In the last two models, the outcomes are pairs: one element is the payoff determined by the objective function $f$, the other is an indication of whether the constraint under consideration is satisfied or violated. The $\forall s\in S$ clause in the constraints is a consequence of the fact that in optimization theory, constraint satisfaction trumps improved (increased) payoffs. Hence constraint violations are immediately viewed as worst outcomes, irrespective of the payoff, and therefore they are to be avoided at all cost (payoff), if possible. Thus according to the Maximin rule, preference in terms of the payoffs enters the picture only with respect to alternatives that satisfy the $\forall s\in S$ clause. In short, this clause eliminates at the outset alternatives whose worst outcome involves violation of the constraint. (M-3) is very popular in the field of robust optimization.

Despite the visual differences, these three models are all equivalent to each other in the sense that any one of them can be re-written in the formats of the other two. In particular, note that (M-3) is the instance of (M-2) characterized by the property does not depend on the uncertainty (state) variable.

Sometime, as in this review, it is convenient to generalize a bit these models and allow the state space $S$ to depend on the alternatives $a\in A$, so that each alternative would have its very own state space. This yields the following slightly modified generic maximin models:

$$
\begin{align}
v:&= \max_{a\in A} \min_{s \in S(a)} \ f(a,s) \tag{MM-1}\\
v&=\max_{a\in A} \min_{s\in S(a)} \ \big\{ f(a,s): g(a,s)\in G, \forall s\in S(a)\big\}\tag{MM-2}\\
v&=\max_{a\in A} \ \big\{ f(a): g(a,s)\in G, \forall s\in S(a)\big\}\tag{MM-3}
\end{align}
$$

In this review it is convenient to use (MM-3) as the prototype maximin model. Note that this maximin model lost the $\min$ part because the objective function $f$ does not depend on the uncertainty parameter (state variable) $s$. As indicated above, models of this type are very popular in Robust Optimization.

The following is a step-by-step proof showing that IGDT’s robustness model is an instance of (MM-3), hence that it is a maximin model.

A Proof that IGDT’s robustness model is a maximin model

We show that a sequence of instantiations transform (MM-3) into IGDT’s robustness model.

Step 1: Consider the case where $A = [0,\infty)$and $S(a) = U(a,\tilde{u})$. For this case we obtain the following instance of (MM-3).

$$
\begin{align}
v:= \max_{a\ge 0} \big\{ f(a): g(a,s) \in G, \forall s\in U(a, \tilde{u})\big\} \tag{MM-3-1}
\end{align}
$$

Step 2: Consider the case where $f(a) =a$, $g(a,s) = r(x,s)$ and $G = (-\infty, r_{c}]$. This yields the following instance of (MM-3).

$$
\begin{align}
v(x):&= \max_{a\ge 0} \big\{ a: r(x,s) \in (-\infty, r_{c}], \forall s\in U(a, \tilde{u})\big\}\ \ \text{($\tilde{u}$ and $x$ are given)} \\
& = \max_{a\ge 0} \big\{ a: r(x,s)\le r_{c}, \forall s\in U(a, \tilde{u})\big\}\tag{MM-3-2}
\end{align}
$$

Step 4. Rename $a$ and $s$ and call them $\alpha$ and $u$. This yields the following instance of (MM-3).

$$
\begin{align}
v(x) = \max_{\alpha\ge 0} \big\{ \alpha: r(x,u)\le r_{c}, \forall u\in U(\alpha, \tilde{u})\big\}\tag{MM-3-3}
\end{align}
$$

This is no other than (IGRM).

Note that in the framework, $\alpha$ is treated as a decision variable, as it should.

One of the arguments used by IGDT scholars (e.g. Ben-Haim 2001, 2005, 2006) to explain why IGDT is not a maximin theory, is that the application of maximin models requires the existence of worst outcomes, and therefore such models cannot deal with unbounded uncertainty spaces. This technical argument is obviously very wrong, as there are many cases where maximin models have no problem handling unbounded uncertainty spaces. This is not surprising at all, as it is well known that global maximum and global minimum may exist even in cases where the domain of the objective function is unbounded. The following two maximin models have no problem handling their unbounded uncertainty space $S=(-\infty,\infty)$, as well as their unbounded decision space $X=(-\infty, \infty)$.

f(x,y) = sin(x)cos(y)There are infinitely many global minimum and global maximum but the domain is unbounded

f(x,y) = y^{2} + yx- x^{2} (unbounded domain, unbounded objective function (below and above), yet has a saddle point (maximin point)Give links to informarion on the saddle points with a video

https://www.statisticshowto.com/saddle-point/
and the youtube video

Good video od video of the Khan Academy

The Radius of Stability Connection

The Severe Uncertainty illusion

According to the main texts on IGDT, that is Ben-Haim (2001, 2006, 2010) the severe uncertainty that IGDT was designed to handle is characterized by the following properties:

The end result is that, unfortunately, IGDT’s illusionary gap is still generating peer reviewed publications that attribute to the theory illusionary powers, illusionary relations to other methods and theories, and an illusionary position in the state of the art in decision-making under severe uncertainty. I began discussing this aspect of IGDT with colleagues at the end of 2003, and then writing about it at the end of 2006 when I launch a campaign to contain the spread of IGDT in Australia.

The main objective of this review is to re-examine this illusionary gap, yet again, in the context of Liu (2022). For the benefit of readers who are not familiar with IGDT, here is a short list of facts about IGDT. Those readers may wish to read Review 2 to confirm that the facts below are indeed … facts.

Some basic facts about IGDT

http://info-gap.moshe-online.com/myths_facts.html

Appendix A

The text below was copied on January 13, 2021: 16:31 from:
URL: http://www.environmetrics.net.au/index.php?news&nid=73
The Link is now (May 11, 2023: 13:26) broken.
The layout was slightly edited.
Information-gap decision theory creates a gap in ecological applications and then fills it
May 14, 2014

 

You may not of heard of Info Gap Decision Theory (IGDT) but don’t worry, not many people have.

While the theoretical foundations of IGDT have been well developed and articulated by its architect Yakov Ben-Haim at the Israel Institute of Technology, controversy continues to surround its legitimacy as a credible alternative to existing methodologies.

The issue has again resurfaced with the publication of a letter to the Editor of Ecological Applications by Professor Mark Burgman and Dr. Helen Regan arguing that IGDT is both useful and credible.

Professor David Fox, a one-time IGDT follower, weighs into the the debate. His views are expressed below.

---

In their recent letter to Ecological Applications, Burgman and Regan (2014) provide counter arguments to some of Sniedovich’s (2012) severe, and mostly harsh criticisms of Ben-Haim’s info-gap decision theory (IGDT) (Ben-Haim, 2006). While I have a deep respect for Professor Burgman and Dr. Regan, I believe their unwavering faith in info-gap theory is misplaced. As the title of this note suggests, I agree with Sniedovich (2014) that ‘the gap’ referred to by Burgman and Regan (2014) is illusionary.

For the record, I have worked alongside Ben-Haim, Burgman, Regan, and many others who (myself included) got caught up in a rather unscientific infatuation with a ‘new’ paradigm some ten years ago. I also plead mea culpa to having co-authored a paper on the application of IGDT to the problem of statistical power analysis (Fox et al., 2007). It was, in essence a case of a solution in search of a problem. On reflection, the problem we tackled was eminently solvable within existing frameworks – and possibly better handled by those frameworks (see for example Reyes and Ghosh, 2013).

Sniedovich has waged a vigorous campaign against IGDT which, as noted by Burgman and Regan (2014), has at times been “disingenuous” – a case perhaps of what we football-loving Australians refer to as playing the man and not the ball. Nevertheless, Sniedovich has played a pivotal role in stress-testing the theory as well as urging IGDT practitioners to think more carefully about their models and analysis.

Not long after the publication of our own IGDT application paper (Fox et al., 2007), I began to have reservations about the utility or, more correctly, the necessity of the whole approach. To be clear, I don’t think there is anything fundamentally wrong with IGDT, but when you strip it of its rather obtuse mathematics, it is essentially little more than the formalisation of a deterministic sensitivity analysis (a fact readily acknowledged by Ben-Haim himself). While I expressed concerns about the use and interpretation of the robustness metric and questioned the ability of IGDT to handle simultaneous (and correlated) uncertainty in more complex multi-parameter models (Fox, 2008), a more fundamental question is “do we need IGDT at all”? I believe not. As noted by Burgman and Regan (2014) there already exists a plethora of ‘conventional’ tools to deal with uncertainty and, unlike IGDT these come ‘certified’ by virtue of their long history of use and acceptance by the broad scientific community. Outside the isolated pockets of support for IGDT, the theory remains largely unknown. Certainly within statistical circles, no one I’ve spoken to has heard of Ben-Haim or IGDT. In 2009 I sent a post titled “What is Info-Gap Theory” to Andrew Gelman’s blog (http://goo.gl/EPKp3h). Gelman, a highly-credentialed Bayesian statistician at Columbia University and co-author of the popular text “Bayesian Data Analysis” (Gelman et al., 2013) frankly admitted he had never heard of IGDT and after having looked at some of the material concluded that the complicated mathematics “appeared to be a distraction from the more important goals of modelling the decision problems directly”. Another contributor to the blog noted that “there seems to be interesting sociological questions about how such theories come to be dominant in certain narrow fields” to which Gelman offered the following insight:

 

Regarding the sociological question, I have a theory, which I believe I mentioned in the rejoinder to my recent Bayesian Analysis article. The theory is that (a) there are a lot of ways to get a good solution to any particular statistical problem, and (b) people will often attribute the success to the method rather than to the analyst. The result is that, first, people in applied fields can become easily convinced of the efficacy of any particular method, if applied by a charismatic practitioner; and, conversely, said practitioner will become even more confident of the virtues of his or her method, once it is endorsed by practical researchers in applied fields.

 

Ben-Haim is charismatic, articulate, and intelligent. These qualities resonated within the newly conceived Australian Centre of Excellence for Risk Analysis (ACERA) at the University of Melbourne whose mandate was broadly to provide knowledge, tools, and advice to better manage and understand biosecurity risk. And so it was that IGDT rapidly embedded itself within ACERA as a tool of choice for assessing ‘risk’ although to be fair, ACERA project 0705 was commissioned to review the role and treatment of uncertainty in risk assessments (Hayes, 2011). Section 4.4.3 of this comprehensive review examined the role of IGDT in a biosecurity context. In his introduction, Hayes (2011) notes that “IGT is different because it offers a non-probabilistic approach to decision-making under uncertainty” although later acknowledges that “deterministic models … have limited utility in a risk assessment context”.

For me, the ‘IGDT debate’ has largely been a technical one that has been dominated by one protagonist and one defendant and a small, but loyal bunch of supporters. What appears to be lacking is evidence in the form of case studies where the superiority of actual decisions made on the basis of an IG analysis can be demonstrated when compared to decisions that would have be made had more traditional methods been employed. If this evidence exists and stands the scrutiny of normal scientific review, then I believe IGDT has a rightful role in the risk analysts’ tool box – even if it shares features with or can be subsumed within other, more established paradigms. Mathematicians and Statisticians are used to the rebadging of their techniques. Genichi Taguchi cleverly repackaged ANOVA for engineers by using familiar terms such as signal-to-noise ratio in place of Mean Square Error and orthogonal arrays instead of fractional factorial designs while Multi Criteria Analysis (MCA) is a favoured tool of environmental scientists – otherwise known by its original name of Goal Programming (which interestingly utilises the concept of satisficing as does IGDT!).

In the end, it doesn’t matter what you call it and how it’s packaged if it leads to more informed decision-making. If practitioners find Ben-Haim’s IGDT and concepts like robustness easier to use and interpret than Wald’s maximin criterion – so be it. But I doubt it!

 

Prof. David Fox
May 14, 2014

 

Literature Cited

• Ben-Haim Y. 2006 Info-Gap Decision Theory: Decisions Under Severe Uncertainty. 2nd ed, Academic Press, Oxford, UK.
• Burgman, M.A. and Regan, H.M. 2014 Information-gap decision theory fills a gap in ecological applications.
  Ecological Applications 24:227-228.
• Fox, D.R. 2008 To IG or not to IG? – that is the question. Decision Point, 24:10-11.
• Fox D.R., Ben-Haim Y., Hayes K.R., McCarthy M., Wintle B. and Dunstan P. 2007. An Info-Gap Approach to Power and Sample-size calculations. Environmetrics, 18:189-203.
• Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B., Vehtari, A., and Rubin, D.B. 2013. Bayesian Data Analysis, Third edition, Chapman and Hall/ CRC.
• Hayes, K.R. 2011 Issues in quantitative and qualitative risk modelling with application to import risk assessment ACERA project (0705). Australian Centre of Excellence for Risk Analysis, University of Melbourne.
• Reyes, E.M. and Ghosh, S.K. 2013. Bayesian average error-based approach to sample size calculations for hypothesis testing. J. Biopharmaceutical Statistics, 23:569:588.
• Sniedovich, M. 2012. Fooled by local robustness: an applied ecology perspective. Ecological Applications 22:1421-1427.
• Sniedovich, M. 2014. Response to Burgman and Regan: The elephant in the rhetoric on info-gap decision theory. Ecological Applications 24(1):229-233.

Appendix B

Since the publication under review here share multiple authors of the paper on a similar topic that I reviewed in 2022, and both have their origin in a 2022 PhD thesis that I plan to review soon, I tend to conclude that, against my advice, some Australian scholars still have unwavering faith in IGDT.

Having clarified that, I wish to stress that this review is not an attempt to reason with followers of IGDT. I discovered a long time ago that his is a futile mission-impossible. What I do plan to do in this review is remind new users of IGDT of the fact that in the during the period 2004-2014 Australia used to be an IGDT strong hold. See my 2011 report to ACERA. And that a lot of resources were allocated to the study and promotion of this theory in academic, research, and government organizations in Australia.

It is a pity that some current research activities in Australia are unaware of, or deliberately ignoring, important lessons learned from the “Australian IGDT experience”.

I plan to discuss this matter in more detail in my review of the PhD Thesis entitled Cost-effective management of invasive species: An application of info-gap decision theory (Liu 2022).

So this review is (relatively) short.

Preface

Introduction

The Fundamental Theorem of IGDT IGDT’s two core models, namely its robustness model (IGRM) and its robust-satisficing decision model (IRSDM) are definitely and definitively simple maximin models. Recall that, $$ \begin{align} \widetilde{\alpha}(x):&= \max_{\alpha\ge 0} \big\{ \alpha: r(x,u) \le r_{c}, \forall u\in U(\alpha,\tilde{u})\big\}\tag{IGRM}\\ \widehat{\alpha}:&= \max_{\substack{x\in X\\ \alpha\ge 0}} \big\{ \alpha: r(x,u) \le r_{c}, \forall u\in U(\alpha,\tilde{u})\big\}\tag{IRSDM} \end{align} $$ Both are simple instances of this generic maximin model: $$ v:= \max_{d\in D} \min_{s\in S(d)} \big\{f(d, s): g(d,s) \le c, \forall s\in S(d)\big\}\tag{MM-1} $$