International Journal of Forecasting xxx (xxxx) xxx

Contents lists available at ScienceDirect

International Journal of Forecasting

journal homepage: www.elsevier.com/locate/ijforecast

On single point forecasts for fat-tailed variables

∗
Nassim Nicholas Taleb a,b , , Yaneer Bar-Yam c , Pasquale Cirillo d,e
a
Universa Investments, Miami, United States of America
b
Tandon School of Engineering, New York University, New York City, NY, United States of America
c
New England Complex Systems Institute, Cambridge, United States of America
d
M Open Forecasting Center and Institute For the Future, University of Nicosia, Nicosia, Cyprus
e
S-T-A-T-S GmbH, Bern, Switzerland

article info a b s t r a c t

Keywords: We discuss common errors and fallacies when using naive ‘‘evidence based’’ empiricism
COVID-19 and point forecasts for fat-tailed variables, as well as the insufficiency of using naive
Forecasting first-order scientific methods for tail risk management.
Debate
We use the COVID-19 pandemic as the background for the discussion and as an
Evidence-based science
example of a phenomenon characterized by a multiplicative nature, and what mitigating
Tail risk
Risk fallacies policies must result from the statistical properties and associated risks. In doing so, we
also respond to the points raised by Ioannidis et al. (2020).
© 2020 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved.

1. Main statements (vi) There are feedback mechanisms between forecast

and reaction that affect the validity of some pre-
(i) Forecasting single variables in fat-tailed domains is dictions.
in violation of both common sense and probability (vii) Individual risks fail to translate into systemic risks
theory. under multiplicative processes.
(ii) Pandemics are extremely fat-tailed events, with po- (viii) One should never treat the ‘‘costs’’ of mitigation
tentially destructive tail risk. Any model ignoring without taking into account the costs of the disease,
this is necessarily flawed. and in some cases naive cost–benefit analyses fail
(iii) Science is not about making single point predic- (for sure when statistical averages are nonconver-
tions but about understanding properties (which gent or invalid for tail risk purposes).
can sometimes be tested by single point estimates (ix) Historically, in the aftermath of the Great Plague,
and predictions). economies were less fragile to pandemics, equipped
(iv) Sound risk management is concerned with to factor in effective mechanisms of containment
(quarantines) in their operating costs. It is more
extremes, tails and their full properties, and not
cogent to blame overoptimization than reaction to
with averages, the bulk of a distribution or naive
disease.
estimates.
(v) Naive fortune-cookie evidentiary methods fail to The article is organized at three levels. First, we make
work under both risk management and fat tails, general comments around the nine points in the Main
because the absence of raw evidence can play a Statements, explaining how single point forecasts are an
large role in the properties. unscientific simplification incompatible with processes
with richer properties. Next we go deeper into the tech-
∗ Corresponding author at: Tandon School of Engineering, New York nical arguments. Finally we address specific points in
University, New York City, NY, United States of America. Ioannidis, Cripps, and Tanner (2020) and answer their
E-mail address: [email protected] (N.N. Taleb). arguments concerning our piece.

https://doi.org/10.1016/j.ijforecast.2020.08.008
0169-2070/© 2020 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved.

Please cite this article as: N.N. Taleb, Y. Bar-Yam and P. Cirillo, On single point forecasts for fat-tailed variables. International Journal of Forecasting (2020),
https://doi.org/10.1016/j.ijforecast.2020.08.008.
N.N. Taleb, Y. Bar-Yam and P. Cirillo International Journal of Forecasting xxx (xxxx) xxx

2. Commentary Remark 1 (Observed events vs Observed Properties). Ran-

dom variables with unstable (and uninformative) sample
2.1. Both forecasters and their critics are wrong moments may still have extremely stable and informative
tail properties, centrally useful for robust inference and
At the onset of the COVID-19 pandemic, many re- risk taking. Furthermore, these reveal evidence.
search groups and agencies produced single point ‘‘fore- This is the central problem with the misunderstanding
casts’’ for the pandemic – most relied upon trivial of The Black Swan (Taleb, 2001-2018): some events may
logistic regressions, or upon the compartmental SIR (Sus-
have stable and well-known properties, yet they do not
ceptible, Infectious/Infected, or Recovered) model, some-
lend themselves to prediction.
times supplemented with cellular automata, or with agent-
based models assuming various social rules and behav-
iors. Apparently, the prevailing idea is that producing a 2.3. Fortune-cookie evidentiary methods
single numerical estimate is how science is done, and how
science-informed decision-making ought to be done: bean In the early stages of the COVID-19 pandemic, scholars
counters producing precise numbers. And always within like Ioannidis (Ioannidis, 2020) suggested that one should
a narrowly considered set of options identified by the wait for ‘‘more evidence’’ before acting with respect to
researchers. that pandemic, claiming that ‘‘we are making decisions
Well, no. That is not how ‘‘science is done’’, at least without reliable data’’.
in this domain, and that is not how informed decision- First, there seems to be some probabilistic confusion,
making should develop. leading towards the so-called delay fallacy (Hansson,
Furthermore, subsequently and ironically, many criti- 2004): ‘‘if we wait we will know more about X, hence no
cized the plethora of predictions produced, because these
decision about X should be made now’’.
did not play out (no surprise there). This is also wrong,
In front of potentially fat-tailed random variables, more
because both forecasters (who missed) and their crit-
evidence is not necessarily needed. Extra (usually impre-
ics (complaining) were wrong. Indeed, forecasters would
cise) observations, especially when coming from the bulk
have been wrong anyway, even if they had got their
predictions right. In fact, as we will clarify throughout of the distribution, will not guarantee extra knowledge.
this article, (1) in some domains (i.e. under fat tails) Extremes are rare by definition, and when they manifest
naive forecasts are poor descriptors of a system (hence themselves it is often too late to intervene. Sufficient
highly unscientific), even when they might appear reason- – and solid – evidence, in particular for risk manage-
able; (2) for some functions (risk-management related), or ment purposes, is already available in the tail properties
some classes of exposures (systemic ones), these forecasts themselves. An existential risk needs to be killed in the
are extremely misplaced. egg, when it is still cheap to do so. Events of the last
few months have shown that waiting for better data has
2.2. Statistical attributes of pandemics generated substantial delays, causing millions of deaths
and serious economic consequences.
Using tools from extreme value theory (EVT), Cirillo Second, unreliable data2 – or any source of serious
and Taleb (Cirillo & Taleb, 2020) have recently shown uncertainty – should, under some conditions, make us
that pandemic deaths are patently fat-tailed1 – a fact follow the ‘‘paranoid’’ route. More uncertainty in a system
some people like Benoit Mandelbrot (or one of the au-
makes precautionary decisions more obvious. If you are
thors, in The Black Swan (Taleb, 2001-2018)) had already
uncertain about the skills of the pilot, you get off the plane
guessed, but never formally investigated. Even more, the
when it is still possible to do so. If there is an asteroid
estimated tail parameter α is smaller than 1, suggesting
an apparently infinite risk (Cirillo & Taleb, 2020), in line headed for earth, should we wait for it to arrive to see
with destructive events like wars (Cirillo & Taleb, 2016a, what the impact will be? We might counter that there
2016b; Taleb & Cirillo, 2019), and the so-called ‘‘dismal’’ were asteroids in the past that had devastating impacts,
theorem (Weitzman, 2009). Pandemics do therefore rep- and besides we can calculate the physics. The logical fal-
resent a source of existential risk. The implication is that lacy runs deeper: ‘‘We did not see this particular asteroid
much of what takes place in the bulk of the distribution is yet’’ misses the very nature of the power of science to
just noise, according to ‘‘the tail wags the dog’’ effect (Cir- generalize (and classify), and the power of actions to
illo & Taleb, 2020; Taleb, 2020a). And one should never possibly change the outcome of events. Similarly, if we
forecast, pontificate, or theorize from noise! Under fat had a hurricane headed for Florida, a statement like ‘‘We
tails, all relevant and vital information lies in fact in the have not seen this hurricane yet, perhaps it will not be
tails themselves (hence in the extremes), which can show like the other hurricanes!’’ misses the essential role of risk
remarkably stable properties. management: to take preventive actions, not to complain
ex post. And if people take action boarding up windows,
1 A non-negative continuous random variable X has a fat-tailed and evacuating, the claim ‘‘look it was not so devastating’’,
distribution, if its survival function S(x) = P(X ≥ x) is regularly varying,
formally S(x) = L(x)x−α , where L(x) is a slowly varying function,
for which limx→∞ L(x) = 1 for t > 0 (de Haan & Ferreira, 2006;
L(tx) 2 Many of those complaining about the quality of data and asking
Embrechts, Klüppelberg, & Mikosch, 2003; Falk, Hüsler, & Reiss, 2004). for more evidence before taking action, even in extremely risky
The parameter α is known as the tail parameter, and it governs the situations, rarely treat the inputs of their predictive models as im-
fatness of the tail (the smaller α the fatter the tail) and the existence precise (Cirillo & Taleb, 2016b; Viertl, 1995), stressing them, and
of moments (E [X p ] < ∞ if and only if α > p). performing serious robustness checks of their claims.

2
N.N. Taleb, Y. Bar-Yam and P. Cirillo International Journal of Forecasting xxx (xxxx) xxx

that someone might afterwards make, should be consid- Notice in fact that 1014 observations are needed for the
ered closer to a lunatic conspiracy fringe than scientific sample mean of a Pareto ‘‘80/20’’, with α ≈ 1.13, to
discourse. emulate the gains in reliability of the sample average of a
By definition, evidence follows and never precedes rare 30-data-points sample from a normal distribution (Taleb,
impactful events. Waiting for the accident before putting 2020a).
the seat belt on, or evidence of fire before buying in- Assuming significance and reliability with a low n is
surance would make the perpetrator exit the gene pool. an insult to everything we have learned since Bernoulli,
Ancestral wisdom has numerous versions such as Cineri or perhaps even Cardano.
nunc medicina datur (one does not give remedies to the Also notice that discussing the optimality of any alarm
dead), or the famous saying by Seneca Serum est cavendi system (Amaral-Turkman & Turkman, 1990; Lindgren,
tempus in mediis malis (you don’t wait for peril to run its 1975; Svensson, Lindquist, & Lindgren, 1996) trying to
course to start defending yourself). perform predictions on averages would prove meaning-
However, just as there are frivolous lawsuits there are less under extremely fat tails, i.e. when α ≤ 2, that
frivolous risk claims and, as we will see further down, is when the LLN works very slowly or does not work.
we limit these precautionary considerations to a precise In fact, even when the expected value is well defined
class of fat-tailed multiplicative processes – when there (i.e. 1 < α < 2), the non-existence of the variance would
is systemic risk. affect all the relevant quantities for the verification of
optimality (de Maré, 1980), from the size of the alarm
Remark 2 (Fundamental Risk Asymmetry). For matters of to the number of correct and false alarms, from the
survival, particularly when systemic, and in the presence probability of detection of catastrophes to the chance of
of multiplicative processes (like a pandemic), we require undetected events. For all these quantities, the naive sam-
‘‘evidence of no harm’’ rather than ‘‘evidence of harm’’. ple estimates commonly used would prove misleading.
A solution could be the implementation on EVT-based
3. Technical comments approaches, possibly with the additional tools of Cirillo
and Taleb (2016a) or Nes̆lehová, Embrechts, and Chavez-
3.1. The law of large numbers (LLN) and evidence Demoulin (2006), but at this stage nothing similar exists,
to the best of our knowledge.
In order to leave the domain of ancient divination (or For this and other reasons specified later, the appli-
modern anecdote) and enter proper empirical science, cation of a non-naive precautionary principle (Norman,
forecasting must abide by both evidentiary and proba- Bar-Yam, & Taleb, 2020) appears to be the viable solution
bilistic rigor. Any forecasting activity about the mean (or in front of potentially existential risks.
the parameter) of a phenomenon requires the working
of the law of large numbers (LLN), guaranteeing the con- 3.2. Science is about understanding properties, not forecast-
vergence of the sample mean at a known rate, when the ing single outcomes
number n of observations increases. This is surely well
known and established, except that some are not aware Figs. 1 and 2 show the extent of the problem of fore-
that, even if the theory remains the same, the actual story casting the average (and so other quantities) under fat
changes under fat tails. tails. Most of the information is away from the center of
Even in front of the most well-behaved and non-erratic the distribution. The most likely observations are far from
random phenomenon, if someone claimed fitness or non- the true mean of the phenomenon and very large samples
fitness of a forecasting ability on the basis of a single are needed for reliable estimation. In the lognormal case
observation (n = 1), they would be rightly accused of of Fig. 1, 85% of all observations fall below the mean; half
unscientific claim. Unfortunately, with fat-tailed variables the observations even fall below 13% of the mean. In the
that ‘‘n = 1’’ error can be made with n = 106 . In the case Paretian situation of Fig. 2, mimicking the distribution of
of events like pandemics, even larger n → ∞ can still be pandemic deaths, the situation gets even worse: the mean
anecdotal. is so far away that we will almost never observe it. It is
therefore preferable to look at other quantities, like for
Remark 3 (LLN and Speed of Convergence). Fat-tailed ran- example the tail exponent.
dom variables with tail exponent α ≤ 1 are simply not In some situations of fast-acting LLN, as (sometimes)
forecastable. They do not obey the LLN, as their theoretical in physics, properties can be revealed by single predictive
mean is not defined, so there is nothing the sample mean experiments. But it is a fallacy to assume that a single
can converge to. But we can still understand several useful predictive experiment can actually validate any theory; it
tail properties. And even for random variables with 1 < is rather a single tail event that can falsify a theory.
α ≤ 2, the LLN can be extremely slow, requiring an often Sometimes, as recently shown in the International Jour-
nal of Forecasting by one of the authors (Taleb, 2020b),
unavailable number of observations to produce somehow
a forecaster may find a single quantity that is actually
reliable forecasts.
forecastable, say the survival function. For n observations
As a matter of fact, owing to preasymptotic properties, a tail survival function has an error of o( 1n ), even when
a conservative heuristic is to consider variables with α ≤ tail moments are not tractable, which is why many pre-
2.5 as not forecastable in practice. Their sample mean will dict binary outcomes – as with the ‘‘superforecasting’’
be too unstable and will require way too much data for masquerade. In Taleb (2020a), it is shown how – para-
forecasts to be reliable in a reasonable amount of time. doxically – the more intractable the higher moments of
3
N.N. Taleb, Y. Bar-Yam and P. Cirillo International Journal of Forecasting xxx (xxxx) xxx

Fig. 1. A high variance lognormal distribution: 85% of observations fall below the mean; half the observations fall below 13% of the mean. The
lognormal has milder tails than the Pareto which has been shown to represent pandemics.

Fig. 2. A Pareto distribution with a tail similar to that of the pandemics. It makes no sense to forecast a single point. The ‘‘mean’’ is so far away
you almost never observe it. You need to forecast things other than the mean. And most of the density is where there is noise.

the variable, the more tractable the survival function be- The problem is actually worse. In fact, distributional
comes. Metrics such as the Brier score are well adapted to forecasts are more than hard – and often uninforma-
binary survival functions, though not to the correspond- tive. Building so-called empirical distributions by sur-
ing random variables. That is why survival functions are vival functions does not reveal tail properties since it
essentially useless for risk management purposes. In in- will necessarily be censored and miss tail observations –
surance, for instance, one never uses survival functions for those that under very fat tails (say α ≤ 2) harbor not
most, but literally all of the properties (Taleb, 2020a). In
hedging, but rather expected shortfalls – binary functions
other words, probabilities are thin tailed (since they are
are reserved for (illegal) gambling.3
bounded by 0 and 1) but the corresponding payoff is not,
so small errors in probability translate into large changes
3.3. We do not observe properties of empirical distributions in payoffs. However, as further discussed in Cirillo and
Taleb (2020), the tail parameters themselves have thin-
tailed distribution, hence reveal their properties rather
A commentator (Andrew Gelman) (Gelman, 2020) rapidly. Simply, tail parameters extrapolate – whereas
wrote ‘‘The sad truth, I’m afraid, is that Taleb is right: survival functions do not – and methods to measure the
point forecasts are close to useless, and distributional tail are quite potent.4
forecasts are really hard’’.
4 This also relates to the superforecasters masquerade mentioned
earlier: building survival functions for tail assessments via sports-like
3 The main problem is that the conditional expectation is not ‘‘tournaments’’ as in Tetlock and Gardner (2016), instead of using more
convergent: limK →∞ K1 E(X |X > K ) > 1, see Taleb (2020b) for a lengthy rigorous approaches like EVT, is simply wrong and violates elementary
discussion. probability theory.

4
N.N. Taleb, Y. Bar-Yam and P. Cirillo International Journal of Forecasting xxx (xxxx) xxx

Fig. 3. Above, a histogram of 106 realizations of r, from an exponential distribution with sole parameter λ = 21 . Below, that of X = er . We can see
the difference between the two distributions. The sample kurtoses are 9 and 106 , respectively (in fact it is infinite for the second) – all values for
the second one are dominated by a single large deviation.

3.4. Uncertainty goes one way; errors in growth rates induce Remark 4 (Errors in Exponential Growth). (1) Errors in
biases and massive fat tails for the quantity of interest growth rates of a disease increase the fatness of tails in
the distribution of fatalities.
Consider the simple model (2) Errors in growth rates translate, on balance, into
higher expected casualties.
Xt = X0 er(t −t0 ) ,
We note that in the context of dynamical systems
where Xt represents the quantity of interest (say the
an exponential dynamics is defined as chaotic (Lorenz,
number of fatalities in pandemics) between periods t0
1963). While the study of chaos often considers systems
and t,
with fixed parameters and variable initial conditions, the
∫ t
1 same sensitivities arise due to variations in parameters;
r = rs ds
(t − t0 ) t0
in this case, the value of contagion rate (R) and the so-
cial behaviors that affect it. Indeed this means that by
and rs is an instantaneous rate. changing human behavior, the dynamics can be strongly
Using the histograms of r and X , Fig. 3 shows some-
affected, thus allowing for the opening of opportunities
thing fundamental: a well-behaved distribution, that of
for extinction.5
r, may lead to an untractable one, that of X ; further-
more, the more volatile r, the more downward-biased
your observation of the mean of X . 5 One of the authors has shown (Rauch & Bar-Yam, 2006) that
Implication: one cannot naively translate between the with increasing global transportation there is a phase transition to
rate of growth r and XT , because errors in r could be small global extinction with probability 1. This indicates that historical
distributions do not account for the severity or frequency of current or
(but surely not zero), but their impact will be explosive on future extreme events because the fat-tailed distributions themselves
X , because of exponentiation. are coalescing to unit probability extreme events over shorter time
Simply, if r is exponentially distributed (or part of that intervals due to global changes in societal behaviors. During this
family), X will be power law. The tail α is a direct function process the probability distributions for events in any time interval
becomes progressively more weighted to large scale events. Thus his-
of the variance: the higher the variance of r, the thicker torical decadal or century intervals between pandemics are inadequate
the tail of X . descriptors of current risk.

5
N.N. Taleb, Y. Bar-Yam and P. Cirillo International Journal of Forecasting xxx (xxxx) xxx

3.5. Never cross a river that is 4 feet deep on average Ioannidis et al. (2000) write: ‘‘the average daily risk
of dying from coronavirus for a person <65 years old is
Risk management (or policy making) should focus on equivalent to the risk of dying driving a distance of 13 to
tail properties and not on the body of probability dis- 101 miles by car per day during that COVID-19 fatality
tributions. For instance, The Netherlands have a policy season in 17 of the 24 hotbeds (...) For many hotbeds, the
of building and calibrating their dams and dykes not risk of death is in the same level roughly as dying from a
on the average height of the sea level, but on the ex- car accident during the daily commute’’.
tremes, and not only on the historical ones, but also on Even if Ioannidis et al.’s computation were to hold
those one can expect by modeling the tail using EVT, via true for one individual (it does not), conditionally on an
semi-parametric approaches (de Haan & Ferreira, 2006; excess of 103 of such individuals dying, the probability
Embrechts et al., 2003). that the cause of death is COVID-19 and not a car accident
converges to 1. When you die of a contagious disease,
3.6. Science is not about safety people around you are at risk of contagion, and they
can then infect other people, in a cascading effect. It is
Science is a procedure to update knowledge; and it quite elementary: car accidents are not contagious, while
can be wrong provided it produces interesting discussions COVID-19 is. You cannot conflate the two objects: one
that lead to more discoveries. But real life is not an ex- is additive in the aggregate, the other is multiplicative.
periment. If we used a p-value of .01 or other methods of In Taleb (2020a), it has been shown that this is a severe
statistical comfort for airplane safety, few pilots and flight error, leading to macroscopic blunders.7
attendants would still be alive. For matters that have
systemic effects and/or entail survival, the asymmetry is Remark 5 (Additive vs. Multiplicative Risks (Scaling of
even more pronounced. Probabilities)). Under multiplicative effects the risks for a
collective do not scale up from the risks of an individ-
3.7. Forecasts can result in adjustments that make forecasts ual. Trivially, systemic risks can be extreme, where the
less accurate individual ones are low, or vice versa.

It is obvious that if forecasts lead to adjustments, and 4.2. Trade-offs and ergodicity
responses that affect the studied phenomenon, then one
can no longer judge these forecasts on their subsequent One could say: panic saves lives, but at what economic
accuracy. Yet the point does not seem to be part of the price? Let us put aside ethical arguments, and answer it,
standard discourse on COVID-19. ignoring for a moment the value of human life.
By various mechanisms, including what is known as The fact is that some classes of (systemic) risks require
Goodhart’s law (Strathern, 1997), a forecast can become a being killed in the egg, also from an economic point of
target that is gamed by participants – see also the Lucas view. The good news is that there are not so many – but
critique applying the point more generally to dynamical pandemics as we said fall squarely within the category.
systems Lucas (1976). In that sense a forecast can be a The ‘‘dismal’’ theorem (Weitzman, 2009) mentioned
warning of the style ‘‘if you do not act, these are the earlier tells us that it is an error to use trade-off analysis
costs’’.6 under existential risk. There have been many proofs of
More generally, any game theoretical framework has similar arguments on grounds of ergodicity, well-known
an interplay of information and expectation that causes by insurance companies since Cramèr: simply, you can-
forecasts to become self-canceling. The entire apparatus not use naive B-school cost–benefit analyses for Rus-
of efficient markets – and modern economics – is based sian roulette, because of the presence of an absorption
on such self-canceling aspect of prediction, under both barrier (Taleb, 2001-2018). But one should not blame
rational expectations and an arbitrage-free world. Ioannidis et al. (2000) for this error in reasoning: it has
been shown to be unfortunately prevalent in the decision-
4. Remarks specific to Ioannidis et al. science literature (Peters & Gell-Mann, 2016).

4.1. Systemic risks vs individual risks Remark 6 (Ruin Problems). Traditional cost–benefit anal-
ysis fails to apply to situations where statistical averages
A fundamental problem, in both Ioannidis, Axfors, and are unreliable, if not invalid.
Contopoulos-Ioannidis (2000) and Ioannidis et al. (2020),
lies in ignoring scaling: systemic risks do not resemble Moreover, it is not correct to assume, more or less
(even qualitatively) individual risks. The macro- and the implicitly, that a disease brings no or little costs, while
micro-properties of contagious events, given their infec- mitigation is burdensome. There are indeed severe non-
tive multiplicative nature, do not map directly onto one linearities at play.
another. First of all, risk is beyond the simple and direct disease-
specific mortality rate. In fact, letting the disease run
6 For instance Dr. Fauci’s warning that the number of (verified)
infections could reach 100 K per day (New York Times, June 30, 2020) 7 Note that this is also a typical example of ‘‘size fallacy’’, in which
should not be interpreted as a forecast to be judged according to its different risky events are compared just on the basis of their probabili-
accuracy; rather a signal about what could happen should one avoid ties of occurrence, without caring about their different nature (Hansson,
taking action. 2004).

6
N.N. Taleb, Y. Bar-Yam and P. Cirillo International Journal of Forecasting xxx (xxxx) xxx

above a certain threshold would compound its effect (in Likewise, it would be a good idea to question first the
an explosive manner), because of the saturation of ser- excessive burden on Western economies, particularly the
vices, causing for example the displacement of other pa- United States, of measures taken to ensure workplace and
tients, many in potentially critical conditions; something transportation safety which, we saw, are driven by the
that we have seen happening in the Region of Lombardy legal system and the tort mechanisms.
in Italy, in New York City, and elsewhere for several weeks
during the spring of 2020 (Sø reide, Hallet, Matthews, Remark 8 (Domain Dependence). It is not rational to worry
Schnitzbauer, Line, Lai, et al., 2020). Furthermore, for sur- about pandemic costs (extremely fat-tailed exposure),
vivors the illness itself represents a large economic drain, while not also questioning other sizable insurance-style
be it only from lost working hours, not counting the costs expenditures for transportation and workers’ safety.
of hospitalization. And for every severe infection, there It is therefore incorrect to claim that it is the author-
is an unspecified number of morbidities, with unknown ities’ response to the pandemic that caused unemploy-
(but definitely larger than zero) additional mortality and ment in the transportation or hospitality industries. As
long-term costs for the health system (Ackermann et al., a matter of fact, the arguments proposed by two of the
2020; Sø reide et al., 2020), as has been the case for other authors (Norman et al., 2020), last January 2020, were
diseases like SARS (Ngai, Ko, To, Tong, & D.S. Hui, 2010).8 aimed at lowering the economic effect of the pandemic:
prevention is orders of magnitude cheaper than the cure
Remark 7 (False Dichotomies). One should not treat the – recall that sed prior est sanitas quam sit curatio morbi.
economy and the disease as separate independent items, We note that many comments of the type ‘‘the pan-
particularly by viewing a naive trade-off between eco- demic has caused only 640 K fatalities’’ (as of July 25,
2020) simply ignore the fact that, in practically every
nomic costs and pandemic mitigation.
location subject to the pandemic, there has been local or
Moreover, never underestimate consumers’ (nonlin- governmental action to mitigate it – we do not consider
ear) behavior. When risks are visible (and a pandemic the counterfactual of ‘‘what if’’ because it is not visible.
definitely is), people tend to modify their behavior, ratio-
nally or not, also switching to alternatives, with nonlinear Remark 9 (Economic Fragility). The argument in Taleb
effects on the businesses concerned (Rose, 1992). This is (2001-2018) is that we live in an over-optimized envi-
the reason why the airline industry in the United States ronment, in which a slight drop in sales or a change in
manages to have fewer than 1 fatal crash in 25 × 106 consumer preferences may cause wild interlocking indus-
try collapses. This nonlinearity is similar to ‘‘a large movie
flights (and aims at an even more favorable ratio). One
theater with a very small door at the times of fire’’.
may claim that it is irrational to spend so much of our
It is more cogent to blame the over-optimized eco-
resources mitigating plane crashes, but airline companies
nomic structure than the general reaction to the disease.
know that, in case of fewer checks and efforts, consumers
would then probably switch to other companies, if not 4.3. Early mitigation and economic history
directly to other types of transportation.
Take the hospitality industry. Unless there is once We note here that although the Great Plague took
again comfort on the part of the public, restaurants and place in the fourteenth century, quarantines were en-
hotels will be unprofitable. The rule of thumb in New forced five centuries later as economies understood they
York City is that a drop of 15% in revenues is sufficient to could not afford recurrences. Between the Hapsburg and
make a restaurant shutter permanently; there has been a the Ottoman Empires, there were lazarettos along the bor-
large drop in restaurant attendance in Sweden where the der, and every active Mediterranean port enforced quar-
state did not enforce lockdowns, owing to a high rate of antines for travelers along the expanded silk road, while
voluntary self-isolation (Kamerlin & Kasson, 2020). pilgrim routes were subjected to similar measures. For
The United States (and many other countries world- instance, in the 1830s, in the Count of Monte Cristo, a
wide) have spent trillions of dollars on sophisticated traveler from Paris to Ioanina (where Prof. Ioannidis was
weaponry in the past decades, to counter uncertain previously located), had to spend 4 days in quarantine to
threats. It would be a good idea to question these ex- get there, while there was no particular threat of disease.
penditures first, before doubting the spending to stave off In fact, the novelist was underestimating, for historical
certain pandemics. records show mandatory 9 days for ordinary travelers
and 15 days for merchants according to Roberts (2017),
Sariyildiz and Daglar Macar (2017). Economies adapted to
8 Geronticide: This discussion does not even cover the ethical early mitigation throughout the centuries preceding our
discussion of trade-offs and their inapplicability in some domains, era. Furthermore, the Ottoman Empire has ready lazaret-
perhaps the most central discussion. At what price will you kill
tos for additional quarantining along specified stations at
your parents/grandparents? A million dollars? Ten million? A billion?
Furthermore, the fact that older people are more vulnerable to the
the first signs of a pandemic.
disease brings considerations of geronticide (senicide): one misses that Mitigation has another effect: to delay and tempo-
the silver rule (Taleb, 2001-2018) commands treating older generations rize, while we can understand the properties of the dis-
under a moral liability, as one wishes to be treated by the next ease. While initial treatments are under high opacity, later
generation. Letting the disease run through older generations violates treatments allow for gains of collective experience.9
the interdicts on geronticide and intergenerational obligations. The
fact that your parents did not sacrifice their own parents creates an
obligation to not sacrifice them; your children will spare you in turn, 9 Aside from considerations of geronticide, when the costs of the
under the same rule. Swedish experiment are finally told, one of the factors will be the early

7
N.N. Taleb, Y. Bar-Yam and P. Cirillo International Journal of Forecasting xxx (xxxx) xxx

Fig. 4. Zipf plots (log–log plots of the empirical survival function P> ) for nine random selections of 30 out of the 72 pandemics in Cirillo and
Taleb (2020). The number in the center represents the a naive (OLS) estimate of the tail parameter α , readable as the absolute slope of the red
negative line. The values of α appear to be stable notwithstanding the sampling, signaling the robustness of the approach and the inconsistency of
the ‘‘selection bias’’ critique. The values are also in line with the more rigorous EVT-based findings of Cirillo and Taleb (2020).

4.4. Selection bias and class of events overlook the fact that the analysis deals with pandemics
and not with a single sternutation. The class of events
In Ioannidis et al. (2020), the authors erroneously under consideration in Cirillo and Taleb (2020) is precisely
maintain that choosing tail events as done by Cirillo and defined as ‘‘pandemics with fatalities in excess of 1K ’’,
Taleb (2020) is ‘‘selection bias’’. Actually, the standard and their dataset likely contains most (if not all) of them.
technique there used is the exact opposite of selection Worrying about many missing observations in the left tail
bias: in EVT, one purposely focuses on extremes to derive of the distribution of pandemic deaths is thus misplaced.
properties that influence the outcomes, especially from
a risk management point of view. One could more rea-
4.5. Conditional information
sonably argue that the data in Cirillo and Taleb (2020)
do not contain all the extremes, but, by jackknifing and
bootstrapping the data, the authors actually show the One may be entitled to ask: as we get to know the
robustness of their results to variations and holes in disease, do the tails get thinner? Early in the game one
historical observations: the tail index α is consistently must rely on conditional information, but as our knowl-
lower than 1. In Fig. 4 a simple illustration is given, edge of the disease progresses, should we not be allowed
showing that one can be quite radical in dealing with the to ignore tails?
uncertainty in pandemic fatalities, and still find out that Alas, no. The scale of the pandemic might change,
the findings of Cirillo and Taleb (2020) hold true. but the tail properties will remain invariant. Further-
When the authors in Ioannidis et al. (2020) state that more, there is an additional paradox. If one does not
‘‘Tens of millions of outbreaks with a couple deaths must take the pandemic seriously, it will likely run wild (par-
have happened throughout time’’, to support their selec- ticularly under the connectivity of the modern world,
tion bias claim against (Cirillo & Taleb, 2020), they seem to several orders of magnitude higher than in the past (Al-
bert & Barabasi, 2002)). And diseases mutate, increasing
loss of life when later (current) medical practice would have saved or decreasing in both lethality and contagiousness. The
them even before a vaccine. argument would therefore resemble the following: ‘‘we
8
N.N. Taleb, Y. Bar-Yam and P. Cirillo International Journal of Forecasting xxx (xxxx) xxx

have not observed many plane crashes lately, let’s relax Declaration of competing interest
our safety measures’’.
Finally, we conclude this section with an encouraging The authors declare that they have no known com-
point: fat tails do not make the world more compli- peting financial interests or personal relationships that
cated and do not cause frivolous worries; on the contrary. could have appeared to influence the work reported in
Understanding them actually reduces costs of reaction this paper.
because they tell us what to target – and when to do so.
Because network models tend to follow certain patterns Acknowledgments
to generate large tail events (Albert & Barabasi, 2002;
Garibaldi & Scalas, 2010), in front of contagious diseases We thank Pierre Pinson and Spyros Makridakis for
wisdom in action is to kill the exponential growth in their help. And we also thank John Ioannidis, Sally Cripps
the egg via three central measures: (1) reducing super- and Martin Tanner for their gracious engagement in our
spreader events; (2) monitoring and reducing mobility debate.
for those coming from far-away places (via quarantines);
(3) looking for cheap measures with large payoffs in terms References
of the reduction of the multiplicative effects (e.g. face
masks10 ). Anything that ‘‘demultiplies the multiplicative’’ Ackermann, M., Verleden, S. E., Kuehnel, M., Haverich, A., Welte, T.,
Laenger, F., et al. (2020). Pulmonary vascular endothelialitis,
helps (Taleb, 2020c).
thrombosis, and angiogenesis in Covid-19. New England Journal of
Drastic shotgun measures such as lockdowns are the Medicine, 383, 120–128.
price of avoiding early traveler quarantines and border Albert, R., & Barabasi, A.-L. (2002). Statistical mechanics of complex
monitoring; they can be – temporarily and cum grano salis networks. Reviews of Modern Physics, 74, 47.
Amaral-Turkman, M. A., & Turkman, K. F. (1990). Optimal alarm
– of help, especially in the very early stages of the new
systems for autoregressive process; a Bayesian approach. Computa-
contagious disease, when uncertainty is maximal, to help tional Statistics & Data Analysis, 19, 307–314.
isolating and tracing the infections, and also buying some Cirillo, P., & Taleb, N. N. (2016a). Expected shortfall estimation for
time for understanding the disease and the way it spreads. apparently infinite-mean models of operational risk. Quantitative
Indeed such drastic and painful measures can carry long- Finance, 16, 1485–1494.
Cirillo, P., & Taleb, N. N. (2016b). On the statistical properties and
lasting damages to the system, not counting an excessive tail risk of violent conflicts. Physica A. Statistical Mechanics and its
price in terms of personal freedoms. Applications, 452, 29–45.
But they are the price of not having a good coordi- Cirillo, P., & Taleb, N. N. (2020). Tail risk of contagious diseases. Nature
nated tail risk management in place – to repeat, border Physics, 16, 606–613.
de Haan, L., & Ferreira, A. (2006). Extreme value theory: An introduction.
monitoring and control of superspreader events being the
Springer.
very first such measures. And lockdowns are the costs of de Maré, J. (1980). Optimal prediction of catastrophes with application
ignoring arguments such as increased connectivity in our to Gaussian process. The Annals of Probability, 8, 841–850.
environment and conflating additive and multiplicative Embrechts, P., Klüppelberg, C., & Mikosch, T. (2003). Modelling extremal
events. Springer.
risks.
Falk, M., Hüsler, J., & Reiss, R. D. (2004). Laws of small numbers: Extremes
To conclude, as the trader lore transmitted by genera- and rare events. Birkhäuser.
tions of operators goes, ‘‘if you must panic, it pays to panic Garibaldi, U., & Scalas, E. (2010). Finitary probabilistic methods in
early’’. econophysics. Cambridge: Cambridge University Press.
The Ottoman Empire integrated Byzantine knowledge Gelman, A. (2020). Some forecasting for COVID-19 has failed: a
discussion of Taleb and Ioannidis et al. Available online at https:
accumulated since at least the Plague of Justinian; it is sad //statmodeling.stat.columbia.edu/2020/06/17/some-forecasting-for-
to see ancient cultures more risk-conscious, better learn- covid-19-has-failed-a-discussion-of-taleb-and-ioannidis-et-al.
ers from history, and economically more effective than Hansson, S. O. (2004). Fallacies of risk. Journal of Risk Research, 7,
modern governments. They avoided modern ‘‘evidence- 353–360.
Ioannidis, J. P. A. (2020). A fiasco in the making? As the coronavirus
based’’ reductions that, as we saw, are insulting to both
pandemic takes hold, we are making decisions without reliable
science and wisdom. And, had it not been for such a data. Stat March 17, https://www.statnews.com/2020/03/17/a-
collective ancestral risk-awareness and understanding of fiasco-in-the-making-as-the-coronavirus-pandemic-takes-hold-
asymmetry, we doubt that many of us would be here we-are-making-decisions-without-reliable-data/.
Ioannidis, J. P. A., Axfors, C., & Contopoulos-Ioannidis, D. G. (2000).
today.
Population-level COVID-19 mortality risk for non-elderly individ-
Now, what did we learn from the pandemic? That uals overall and for non-elderly individuals without underlying
an intelligent application of the precautionary princi- diseases in pandemic epicenters. Environmental Research, 188,
ple (Norman et al., 2020) consists in formulating deci- Article 109890.
Ioannidis, J. P. A., Cripps, S., & Tanner, M. A. (2020). Forecasting for
sions that are wise in both foresight and hindsight. Here
COVID-19 has failed. Int. J. Forecast., http://dx.doi.org/10.1016/j.
again, this is ancient: it maps to Aristotle’s phronesis as ijforecast.2020.08.004, (in press).
presented in his Nichomachean Ethics. Kamerlin, S. C. L., & Kasson, Peter M. (2020). Managing COVID-19
spread with voluntary public-health measures: Sweden as a case
study for pandemic control. Clinical Infectious Diseases, http://dx.
10 Most of the trillions spent could have been saved if authorities un- doi.org/10.1093/cid/ciaa864.
derstood the double nonlinearities in face masks: (1) the compounding Lindgren, G. (1975). Prediction for a random time point. The Annals of
effect of both parties having protection, (2) the nonlinearity of the dose Probability, 3, 412–433.
response with disproportional drop in the probability of infection from Lorenz, E. N. (1963). Deterministic non-periodic flow. Journal of the
a reduction in viral load (Taleb, 2020c). Atmospheric Sciences, 20(2), 130–141.

9
N.N. Taleb, Y. Bar-Yam and P. Cirillo International Journal of Forecasting xxx (xxxx) xxx

Lucas, R. (1976). Econometric policy evaluation: A critique. In K. Brun- Sø reide, K., Hallet, J., Matthews, J. B., Schnitzbauer, A. A., Line, P. D.,
ner, & A. Meltzer (Eds.), The phillips curve and labor markets. Lai, P., et al. (2020). Immediate and long-term impact of the COVID-
Carnegie-rochester conference series on public policy 1 (pp. 19–46). 19 pandemic on delivery of surgical services. The British Journal of
American Elsevier. Surgery, http://dx.doi.org/10.1002/bjs.11670.
Nes̆lehová, J., Embrechts, P., & Chavez-Demoulin, V. (2006). Infinite- Strathern, M. (1997). Improving ratings: Audit in the British university
mean models and the LDA for operational risk. Journal of system. European Review, 5, 305–321.
Operational Risk, 1, 3–25. Svensson, A., Lindquist, R., & Lindgren, G. (1996). Optimal prediction of
Ngai, J. C., Ko, F. W., To, S. W., Tong, M., & D.S. Hui, D. S. (2010). catastrophes in autoregressive moving average processes. Journal of
The long-term impact of severe acute respiratory syndrome on pul- Time Series Analysis, 17, 511–531.
monary function, exercise capacity and health status. Respirology, Taleb, N. N. (2001-2018). Incerto: Fooled by randomness, the black swan,
15, 543–550. the bed of procrustes, antifragile, and skin in the game. Penguin.
Norman, J., Bar-Yam, Y., & Taleb, N. N. (2020). Systemic risk of pandemic Taleb, N. N. (2020a). Statistical consequences of fat tails. STEM Academic
via novel pathogens - Coronavirus: A note. New England Complex Press.
Systems Institute. Taleb, N. N. (2020b). On the statistical differences between binary
Peters, O., & Gell-Mann, M. (2016). Evaluating gambles using dynamics. forecasts and real-world payoffs. International Journal of Forecasting,
Chaos. An Interdisciplinary Journal of Nonlinear Science, 26, Article 36(4), 1228–1240, https://doi.org/10.1016/j.ijforecast.2019.12.004.
023103. Taleb, N. N. (2020c). The masks masquerade. https://medium.com/
Rauch, E. M., & Bar-Yam, Y. (2006). Long-range interaction and evolu- incerto/the-masks-masquerade-7de897b517b7.
tionary stability in a predator–prey system. Physical Review E, 73, Taleb, N. N., & Cirillo, P. (2019). The decline of violent conflict: What
Article 020903. do the data really say?. In A. Toje, & N. V. S. Bård (Eds.), The causes
Roberts, A. (2017). In Nukhet Varlik (Ed.), Plague and contagion in the of peace: What we know now. Nobel symposium proceedings (pp.
Islamic mediterranean. Arc Humanities Press. 57–85). Norwegian Nobel Institute.
Rose, N. L. (1992). Fear of flying? Economic analysis of airline safety. Tetlock, P. E., & Gardner, D. (2016). Superforecasting: The art and science
Journal of Economic Perspectives, 6(2), 75–94. of prediction. Random House.
Sariyildiz, G., & Daglar Macar, O. (2017). In Nukhet Varlik (Ed.), Viertl, R. (1995). Statistical methods for non-precise data. CRC Press.
Plague and contagion in the Islamic Mediterranean. Arc Humanities Weitzman, M. L. (2009). On modeling and intepreting the economics of
Press. catastrophic climate change. The Review of Economics and Statistics,
1, 1–19.

10