Professional Documents
Culture Documents
Instructors Materials For Cases in Intel Analysis
Instructors Materials For Cases in Intel Analysis
MATERIALS
Sarah Miller Beebe
Randolph H. Pherson
SECOND EDITION
Cases in
Intelligence
Analysis
STRUCTURED
ANALYTIC TECHNIQUES
IN ACTION
Foreword by Jack Davis
Cases in
Intelligence Analysis
Instructor Materials
Second Edition
Cases in
Intelligence Analysis
Structured Analytic Techniques in Action
Instructor Materials
Second Edition
FOR INFORMATION:
CQ Press
An Imprint of SAGE Publications, Inc.
2455 Teller Road
Thousand Oaks, California 91320
E-mail: [email protected]
SAGE Publications Ltd.
1 Olivers Yard
55 City Road
London EC1Y 1SP
United Kingdom
SAGE Publications India Pvt. Ltd.
B 1/I 1 Mohan Cooperative Industrial Area
Mathura Road, New Delhi 110 044
India
SAGE Publications Asia-Pacific Pte. Ltd.
3 Church Street
#10-04 Samsung Hub
Singapore 049483
Contents
Tables, Figures, and Boxes ix
Matrix of Techniques xiii
Foreword to the Second Edition xv
BY JACK DAVIS, CIA TRAILBLAZER
Prefacexvii
About the Authors xix
Introduction1
Cyber H 2O23
Technique 1: Getting Started Checklist 23
Technique 2: Key Assumptions Check 24
Technique 3: Devils Advocacy 26
Conclusion27
Key Takeaways28
vi Contents
10
Contents vii
Key Takeaways116
Instructors Reading List116
11
13
Understanding Revolutionary
Organization 17 November 147
Technique 1: Multiple Hypothesis Generation: Simple
Hypotheses147
Technique 2: What If? Analysis 149
Technique 3: Foresight Quadrant
CrunchingTM150
Conclusion154
Key Takeaways155
14
Defending Mumbai
from Terrorist Attack 157
Technique 1: Structured Brainstorming 157
Technique 2: Red Hat Analysis 160
Technique 3: Classic Quadrant CrunchingTM162
Technique 4: Indicators 165
Technique 5: Indicators ValidatorTM168
Conclusion174
Key Takeaways179
Instructors Reading List 179
15
viii Contents
16
17
Table 3.1
Table 3.2
Table 3.3
Table 4.1
Table 4.5
Table 4.6
Table 4.7
Table 4.8
Table 4.9
Table 4.10
Table 5.1
Table 5.5
Figure 5.3
Table 5.6
Table 5.7
Figure 5.4
Table 5.8
Figure 5.5
Table 5.9
Table 5.10
Figure 5.6
ix
Table 7.1
Figure 7.1
Map 7.2
Table 7.2
Table 7.3
Table 7.4
Table 7.5
Table 7.6
Table 7.7
Figure 7.2
Figure 7.3
Table 8.1
Figure 8.2
Figure 8.3
Table 8.2
Table 8.3
Figure 8.4
Table 9.2
Box 9.1
Figure 9.2
Table 9.3
Table 9.5
Table 9.6
Figure 9.3
Figure 9.4
Table 10.1
Table 10.5
Table 10.6
Table 9.7
Table 9.8
Table 9.9
Table 10.7
Table 10.8
Table 10.9
Table 10.10
Table 10.11
Table 11.1
Table 11.6
Table 11.7
Table 11.8
Table 11.9
Table 11.10
Table 11.11
Table 11.12
Table 12.2
Figure 12.3
Figure 12.4
Table 12.5
Table 12.6
Table 12.7
Table 12.8
Table 12.9
Table 12.10
Table 13.2
Table 13.4
Figure 13.1
Table 13.5
Table 13.6
Table 13.7
Table 13.8
Figure 13.2
Table 14.2
Box 14.2
Table 14.4
Table 14.5
Table 14.6
Box 14.3
Table 15.4
Figure 15.2
Table 15.6
Figure 15.3
Figure 15.4
Table 16.1
Box 16.4
Figure 16.3
Figure 16.4
Table 16.3
Figure 16.5
Table 17.1
Table 17.5
Table 17.7
Table 17.8
Table 14.11
Table 14.12
Table 14.13
Map 14.2
Figure 14.1
MIND MAP
TECHNIQUES
OF
MATRIX
STRUCTURED BRAINSTORMING
STARBURSTING
MORPHOLOGICAL ANALYSIS
SIMPLE SCENARIOS
INDICATORS
INDICATORS VALIDATOR
SIMPLE HYPOTHESES
DECEPTION DETECTION
OUTSIDE-IN THINKING
PREMORTEM ANALYSIS
STRUCTURED SELF-CRITIQUE
CHALLENGE
ANALYSIS
ASSESSMENT
OF CAUSE
AND EFFECT
DEVILS ADVOCACY
HYPOTHESIS
GENERATION AND
TESTING
DECISION SUPPORT
DECISION MATRIX
SCENARIOS
AND INDICATORS
IDEA
GENERATION
PROS-CONS-FAULTS-AND-FIXES
DECOMPOSITION
AND
VISUALIZATION
STRENGTHS, WEAKNESSES,
OPPORTUNITIES, THREATS
xv
I believe that combining the best of substantive expertise and critical thinking with the best of structured
analysis provides the best protection against avoidable
analytic shortfalls. Cases in Intelligence Analysis provides
the wherewithal for helping IC analysts move toward that
goal.
Preface
AUDIENCE
This book is for anyone who wants to explore new ways of
thinking more deeply and thoroughly. It is primarily
intended to help up-and-coming analysts in colleges and
universities, as well as intelligence professionals, learn
techniques that can make them better analysts throughout
their careers. But this book is just as salient for seasoned
intelligence veterans who are looking for ways to brush up
on skillsor even learn new ones. The cases also are
intended for teams of analysts who want to rehearse and
refine their collaboration skills so that when reallife situations arise, they are prepared to rise to the
challenge together.
CONTENT AND DESIGN
We chose the case study format because it provides an
opportunity to practice the techniques with real-life
contemporary issues. It is also a proven teaching method in
many disciplines. We chose subject matter that is relatively
recentusually from within the past decadeand that
comprises a mix of better- and lesser-known issues. In all
cases, we strove to produce compelling and historically
accurate portrayals of events; however, for learning purposes,
we have tailored the content of the cases to focus on key
learning objectives. For example, we end many of the cases
without revealing the full outcome. Several cases, such as
Who Murdered Jonathan Luna?, have no known outcome.
But whether or not the outcome is known, we urge students to
judge their performance on the merits of their analytic process.
Like mathematics, just arriving at a numerical value or
correct outcome is not enough; we need to show our work.
The value of the cases lies in the process itself and in learning
how to replicate it when real-life analytic challenges arise.
The seventeen cases and analytic exercises in this book
help prepare analysts to deal with the authentic problems and
real-life situations they encounter every day. Taken as a
whole, the seventeen cases walk through a broad array of
xvii
xviii Preface
issues such as how to identify mindsets, mitigate biases,
challenge assumptions, think expansively and creatively,
develop and test multiple hypotheses, create plausible
scenarios, identify indicators of change, validate those indica
tors, frame a decision-making process, and troubleshoot
analytic judgmentsall of which reinforce the main elements
of critical thinking that are so important for successful
analysis. Individually, each chapter employs a consistent
organization that models a robust analytic process by
presenting the key questions in the case, a compelling and
well-illustrated narrative, and carefully chosen recommended
readings. Each also includes question-based analytic
exercises that challenge students to employ structured
analytic techniques and to explicate the value added by
employing structured techniques.
INSTRUCTOR RESOURCES
As instructors ourselves, we understand how important it is
to provide truly turnkey instructor resources. The Instructor
Materials that accompany this book are free to all readers of
this book as a downloadable .pdf, and graphics from both
the case book and the Instructor Materials are available as
free, downloadable .jpeg and PowerPoint slides. We have
classroom-tested each case study and applied what we have
learned to enhance the Instructor Materials and better
anticipate the instructors needs. We believe they are just as
useful to working analysts and students seeking to learn
how best to apply the techn iques. Just like the cases
themselves, the Instructor Materials employ a consistent
organization across all cases that puts the case and the
analytic challenges in context, offers step-by-step solutions
for each exercise, and provides detailed conclusions and key
takeaways to enhance classroom discussion.
ACKNOWLEDGMENTS
Both authors thank Flannery Becker, Ray Converse, Claudia
Pea Crossland, Mary OSullivan, James Steiner, and Roy
Sullivan for their substantial contributions to the book. Both
authors are grateful to many other individuals who helped
review, test, and otherwise improve the cases, including
Nigah Ajaj, Todd Bacastow, Milton Bearden, George Beebe,
Mark T. Clark, Eric Dahl, Jack Davis, Matthew Degn, John
Evans, Roger George, Joseph Gordon, Thomas Graham,
Richards J. Heuer Jr., Georgia Holmer, Daryl Johnson, Laura
Lenz, Austin Long, Frank Marsh, Richard Miles, Gregory
Moore, Polly Nayak, Rudolph Perina, Marilyn Peterson,
Kathy Pherson, Richard Pherson, Mark Polyak, Libby Sass,
Marilyn Scott, Raymond Sontag, Leah Tarbell, Greg
Treverton, Marc Warburton, and Phil Williams, as well as
students of Great Plains National Security Consortium,
James Madison University, Mercyhurst College, the
University of Mississippi, Pennsylvania State University, and
the University of Pittsburgh.
DISCLAIMER
All statements of fact, opinion, or analysis expressed in this
book are those of the authors and do not reflect the official
positions of the Office of the Director of National
Intelligence (ODNI), the Central Intelligence Agency (CIA),
and the Federal Bureau of Investigation (FBI), or any other
US government agency. Nothing in the contents should be
construed as asserting or implying US gove rnment
authentication of information or agency endorsement of the
authors views. The materials in the book have been
reviewed by the ODNI, FBI, and CIA only to prevent the
disclosure of classified material.
xix
Introduction
2 Introduction
Since the pioneering efforts of Heuer to understand and
address common cognitive pitfalls and analytic
pathologies, considerable progress has been made in
developing a variety of new SATs and defining the ways
they may be used. In 2011, Heuer joined one of the authors
of this volume, Randolph H. Pherson, in publishing the
most comprehensive work on this subject to date,
Structured Analytic Techniques for Intelligence Analysis.5
The book describes how structured analysis compares to
other analytic methods, including expert judgment and
quantitative methods, and provides a taxonomy of eight
families of SATs and detailed descriptions of some fiftyfive techniques. By including an in-depth discussion of
how each technique can be used in collaborative team
projects and a vision for how the techniques can be
successfully integrated into analysis done in the
intelligence, law enforcement, and business communities,
Heuer and Pherson challenged analysts from all disciplines
to harness the techniques to produce more rigorous and
informative analysis.
WHY A BOOK OF CASES?
The books published by Heuer and Pherson have helped
analysts become familiar with the range of available
structured analytic techniques and their purposes, but little
work has been done to provide analysts with practical
exercises for mastering the use of SATs. This book is
designed to fill that gap. As such, it is best regarded as a
companion to both Psychology of Intelligence Analysis and
Structured Analytic Techniques for Intelligence Analysis. The
cases in this bookvivid, contemporary issues coupled
with value-added analytic exercisesare meant to bridge
the worlds of theory and practice and bring analysis to life.
They compel readers to put themselves in the shoes of
analysts grappling with very real and difficult challenges.
Readers will encounter all the complexities, uncertainties,
and ambiguities that attend real-life analytic problems and,
in some cases, the pressures of policy decisions that hang in
the balance.
We have chosen a case study approach for several
reasons. First, the techn ique has proved an effective
teaching tool in a wide variety of disciplines, fostering
interactive learning and shifting the emphasis from
instructor-centric to student-centric activity while usually
sparking interest in issues previously unfamiliar to
students.6 The use of the case study approach also allows
Introduction 3
Luna?are designed to teach SATs that challenge prevailing mindsets and develop alternative explanations for
events.
As analysts gain more familiarity with the issues for
which they are respons ible, they often encounter new
developments for which no line of analysis has been
developed. In such circumstances, analysts require
techniques for developing and testing new hypotheses
and for visualizing the data in creative and thoughtprovoking ways. The Assassination of Benazir Bhutto,
Death in the Southwest, The Atlanta Olympics
Bombing, and The DC Sniper are designed with these
goals in mind.
Finally, as analysts master their subjects, they are asked
to tackle problem sets that are arguably the most difficult
analytic challenges: understanding the perceptions and
plans of foreign adversaries and forecasting uncertain future
developments shaped by dynamic sets of drivers. In
Colombias FARC Attacks the US Homeland, Understanding Revolutionary Organization 17 November, and
Defending Mumbai from Terrorist Attack, students put
themselves in the shoes of the adversary and develop a
range of plausible future outcomes, while in Iranian Meddling in Bahrain and Shades of Orange in Ukraine students not only develop scenarios but also actively consider a
range of future outcomes and specific indicators that a particular outcome is emerging. Violence Erupts in Belgrade
rounds out the cases by placing students in a direct decision
support role in which they must not only provide assessments about the forces and factors that will drive events but
also develop a decision framework and troubleshoot their
analysis.
Each of our case studies employs a consistent internal
organization that guides the student through an analytic
process. We begin each case study by listing several
overarching Key Questions. These questions are designed
as general reading guides as well as small-group discussion
questions. The questions are followed by the Case
Narrative, which tells the story of the case. This is
followed by a Recommended Readings section. The final
section, Structured Analytic Techniques in Action, presents
focused intelligence questions and exercises to guide the
student through the use of several structured analytic
techniques and toward self-identification of the value
added by SAT-aided analysis. The turnkey Instructor
Materials, which are available to analysts, students, and
instructors via download, put the learning points for the
4 Introduction
NOTES
1. See Rob Johnston, Analytic Culture in the U.S. Intelligence
Community: An Ethnographic Study (Washington, DC: Center for
the Study of Intelligence, Central Intelligence Agency, 2005), http://
www.fas.org/irp/cia/product/analytic.pdf, 2223. What tends to
occur is that the analyst looks for current data that confirms the
existing organizational opinion or the opinion that seems most
probable and, consequently, is easiest to support....This tendency
to search for confirmatory data is not necessarily a conscious
choice; rather, it is the result of accepting an existing set of hypotheses, developing a mental model based on previous corporate
products, and then trying to augment that model with current data
in order to support the existing hypotheses.
2. See Jack Davis, Introduction: Improving Intelligence
Analysis at CIA; Dick Heuers Contribution to Intelligence
Analysis, in Psychology of Intelligence Analysis, ed. Richards J.
Heuer Jr. (Washington, DC: Center for the Study of Intelligence,
Central Intelligence Agency, 1999, and reprinted in 2007 by
Pherson Associates, LLC, Reston, VA, http://www.pherson.org),
https://www.cia.gov/library/center-for-the-study-of-intelligence/
csi-publications/books-and-monographs/psychology-of-intelli
gence-analysis/PsychofIntelNew.pdf, xvxix.
3. Heuer, ed., Psychology of Intelligence Analysis.
4. Richards J. Heuer Jr., The Evolution of Structured
Analytic Techniques, presentation to the National Academy of
Science, National Research Council Committee on Behavioral and
Social Science Research to Improve Intelligence Analysis for
National Security, Washington, DC, December 8, 2009, http://
www7.nationalacademies.org/bbcss/DNI_Heuer_Text.pdf.
5. Richards J. Heuer Jr. and Randolph H. Pherson, Structured
Analytic Techniques for Intelligence Analysis, 2nd ed. (Washington,
DC: CQ Press, 2015).
6. See Richard Grant, A Claim for the Case Method in the
Teaching of Geography, Journal of Geography in Higher Education
21, no. 2 (1997): 17185; and P. K. Raju and Chetan S. Sankar,
Teaching Real-World Issues through Case Studies, Journal of
Engineering Education 88, no. 4 (1999): 501-8.
Analytic Family
Premortem Analysis
p. 240
Challenge Analysis
Structured Self-Critique
p. 245
Challenge Analysis
Starbursting
p. 113
Idea Generation
The first two steps in the Premortem Analysis are rightbrain-led, creative brainstorming. This process asks analysts to imagine a future in which they have been proved
wrong and work backward to try to identify the possible
causes. In essence, they are identifying the weak links in
their analysis in order to avoid these potential pitfalls prior
to publishing the analysis. Most analysts are more left
brained than right brained, which often makes imagination
techniques like brainstorming challenging. However, when
coupled with the systematic, left-brained checklist that
comprises the second half of the Premortem Analysis,
brainstorming can be the first step toward identifying
sometimes fatal analytic flaws. It is important to encourage
students to be as creative as possible when brainstorming,
keeping all ideas in play.
In this case, a brainstorming session might prompt students to consider the following:
New evidence comes to light that suggests
someone other than the Russians is behind the
poisoning (e.g., her husband, her children, an
acquaintance, a colleague at work, or a case of
mistaken identity).
The toxicology reports were faked. She isnt ill.
The mercury was accidentally placed in the vehicle
(e.g., by her kids, the former owner of the vehicle, or
someone else).
Step 2: Use a brainstorming technique to identify alternative hypotheses for how the poisoning could have occurred.
Keep track of these hypotheses.
6 Chapter 1
In this case, students might identify a number of alternative
perpetrators of the crime. They could include the following:
Karinna Moskalenkos husband.
Moskalenko herself, who staged the poisoning with
or without the assistance of her husband to put the
Russian government on the defensive.
A jealous work colleague.
An acquaintance not connected to her legal work.
Someone connected to a previous or pending case.
Unsupported. There is no
evidence that the Russians
targeted her.
Unsupported. There is no
evidence of intent; there are
other possible explanations.
An accident or fluke.
The alternatives should not include scenarios that contradict known facts in the case. Instructors may advise students that facts such as the presence of mercury in the car
and that Moskalenko and her family are truly suffering from
symptoms of mercury poisoning may be accepted as accurate for the purposes of the case study. As a result, any alternative hypothesis that the Moskalenko family poisoning is a
hoax or that the mercury is not present would be discarded.
Step 3: Identify key assumptions underlying the consensus view. Could any of these be unsubstantiated? Do some
assumptions need caveats? If some are not valid, how much
could this affect the analysis?
The most important aspect of this step is the conversation it produces about the effect of assumption on the analysts confidence level in the mainline judgment itself.
In this case, when assumptions are explicated in this
manner, it becomes apparent that the key assumptions are
unsupported by evidence. This lack of evidence suggests
that analysts should be prepared to track down additional
information, consider alternative explanations, and potentially add a caveat to or revise the mainline judgment.
Some key assumptions and notional assessments are
listed in Table 1.4.
Step 4: Review the critical evidence that provides the
foundation for the argument. Is the analysis based on any
critical item of information? On a particular stream of
reporting? If any of this evidence or the source of the
reporting turned out to be incorrect, how much would this
affect the analysis?
The Moskalenko case is short on hard evidence. Students
should note this dearth, as well as the fact that the direct
evidence in this case is based on two main sources: French
police and Karinna Moskalenkos comments to the press.
Assessment
Other evidence is really historical information, speculation on the part of Moskalenkos friends and colleagues, and
conclusions based on inference.
Step 5: Is there any contradictory or anomalous information? Was any information overlooked that is inconsistent
with the lead hypothesis?
The key pieces of hard evidence in the case are the mercury found in Moskalenkos car and the press reports confirming that she suffered from mercury poisoning. Even
these hard facts, however, are anomalous when examined
more closely. Other information, such as the discrepancy
between press headlines and actual substance of their reports,
is contradictory. A notional analysis is presented in Table 1.5.
Assessment
Mercury found
in car
Moskalenkos
illness
Headline
versus facts
bias, satisficing, premature closure, anchoring, and historical analogy? (See Table 1.7)
Step 9: Based on the answers to the themes of inquiry
outlined, list the potential deficiencies in the argument in
order of potential impact on the analysis.
Analysts should recognize that there are potential deficiencies in most elements of the Premortem Analysis of this
case, including the following:
Unsupported assumptions.
Absence of evidence.
Contradictory information.
Presence of analytic pitfalls.
Analytic Value Added: As a result of analysis, would
you retain, add a caveat to, or dismiss the mainline judgment, and why? Students should seek to dismiss the mainline judgment that the Russians poisoned Moskalenko
because of the unsupported statements by the press and
Moskalenko herself, and the likelihood that analytic pitfalls biased the judgment. They should cite the gaps in
their information base as well as the potential for other,
Assessment
No physical evidence
linking the crime to the
Russians
No other sources of
information other than
Moskalenkos statements,
the mercury found in the
car, and the laboratory
reports confirming that she
has mercury poisoning
Definition
Analytic mindset
Anchoring
Confirmation bias
Historical analogy
Mirror imaging
Premature closure
Satisficing
8 Chapter 1
WHO?
Task 2.
Rewrite the lead judgment of the case so that it reflects any
changes you would incorporate as a result of the Premortem
Analysis.
Important elements that students should use to revise
the judgment include these:
While Moscow has a long history of targeting
its opponents, the involvement of the Russian
government in this case is unclear at this time.
WH
?
AT
Y?
WH
HO
W?
RE
HE
WHEN?
WHO?
Wh
What was the location?
WH
T?
HA
Y?
HO
W?
RE
HE
WHEN?
Wh
What was the substance?
10 Chapter 1
NOTES
1. More Poison: Another Prominent Adversary of Vladimir
Putin Is Mysteriously Exposed to Toxins [editorial], Washington
Post, October 22, 2008, http://www.washingtonpost.com/wp-dyn/
content/article/2008/10/21/AR2008102102342.html.
2. The steps as outlined in this case combine the processes for
a Premortem Analysis and Structured Self-Critique. This combination is particularly helpful in cases that require analysts to think
broadly, imaginatively, and exhaustively about how they might have
been wrong. The Premortem Analysis taps the creative brainstorming process, and the Structured Self-Critique provides a step-by-step
assessment of each analytic element. To aid students learning process, the questions in this case have already been narrowed from the
fuller set of Structured Self-Critique questions found in Richards J.
Analytic Family
p. 56
Premortem Analysis
p. 240
Challenge Analysis
Structured Self-Critique
p. 245
Challenge Analysis
11
12 Chapter 2
that treatment was sought, not by the date the case was
reported in the press. In fact, the FBI used a similar
chronology to illustrate this point in the official
Amerithrax Investigative Summary, noting, the evidence
supports the conclusions that the mail attacks occurred on
two separate occasions.1
Task 1.
Create a Chronolog y of the anthrax attacks and
investigation.
Step 1: Identify the relevant information from the case
narrative with the date and order in which it occurred.
Step 2: Review the Chronology by asking the following
questions:
Event
18 September 2001
Hamilton Township postal worker Richard Morgano scratches his arm while fixing a jammed machine.
19 September 2001
21 September 2001
New York Post employee Johanna Huden notices a bump on her finger that later turns out to be cutaneous anthrax.
25 September 2001
Erin OConnor handles a threatening letter addressed to NBC correspondent Tom Brokaw.
26 September 2001
Hamilton Township postal worker Richard Morgano presents with cutaneous anthrax.
28 September 2001
28 September 2001
29 September 2001
1 October 2001
Ernesto Blanco falls ill in Boca Raton, FL and is diagnosed with inhalation anthrax.
1 October 2001
1 October 2001
1 October 2001
Assistant to CBS News Anchor Dan Rather, Claire Fletcher develops cutaneous anthrax.
2 October 2001
5 October 2001
8 October 2001
The FBI begins a criminal investigation into the anthrax cases. Forty agents search the American Media, Inc. building
where Blanco and Stevens worked.
9 October 2001
At Hamilton Township mail center, a machine jams and a colleague of Norma Wallace shoots compressed air into the
machine, sending dust particles into the air.
14 October 2001
Hamilton Township postal worker Patrick ODonnell develops symptoms of acute cutaneous anthrax.
15 October 2001
Bret Wincup and Grant Leslie open a letter addressed to Senator Daschle and white powder pours out.
15 October 2001
15 October 2001
16 October 2001
16 October 2001
An anonymous Washington, DC Brentwood postal worker called George Fairfax in the press develops inhalation anthrax.
Event
16 October 2001
Washington, DC Brentwood postal worker Thomas Morris, Jr. develops inhalation anthrax.
16 October 2001
17 October 2001
17 October 2001
Hamilton Township postal center accountant Linda Burch develops cutaneous anthrax.
18 October 2001
The Centers for Disease Control confirms that the strains of anthrax in the Daschle and Brokaw letters match, as do the
handwriting in the letters. Also in October, Northern Arizona University microbiologist Dr. Paul Keim pinpoints the strain as
Ames, a strain developed in US government labs. The CDC confirms the find.
19 October 2001
Hamilton Township postal worker Norma Wallace is diagnosed with inhalation anthrax.
19 October 2001
21 October 2001
Hamilton Township postal worker Patrick ODonnell is released from the hospital.
21 October 2001
Washington, DC Brentwood postal worker Thomas Morris, Jr. dies from inhalation anthrax.
22 October 2001
22 October 2001
State Department Mail Center Employee David Hose develops inhalation anthrax.
23 October 2001
New York Post employee Mark Cunningham develops cutaneous anthrax after going through old mail postmarked in
September.
23 October 2001
Hamilton Township postal worker Jyotsna Patel is released from the hospital.
25 October 2001
Manhattan Eye, Ear and Throat Hospital stockroom attendant Kathy Nguyen develops inhalation anthrax.
31 October 2001
Manhattan Eye, Ear and Throat Hospital stockroom attendant Kathy Nguyen dies of inhalation anthrax.
9 November 2001
FBI Press Briefing provides linguistic and behavior assessment of a potential anthrax killer and asks for the publics help.
14 November 2001
15 November 2001
Investigators find an anthrax-laced letter to Senator Leahy in a bag of quarantined mail that was postmarked 9 October.
21 November 2001
June 2002
FBI releases information that radiocarbon dating indicates the spores used in the attacks were made within the last two years.
June 2002
25 June 2002
July 2002
August 2002
Investigators pinpoint a mailbox in Princeton, NJ from which the anthrax letters were sent.
1 August 2002
6 August 2002
11 August 2002
14 Chapter 2
New York
18
19
20
21
22
23
24
25
26
27
29
September
30
10
11
12
10
11
12
October
New Jersey
18
19
20
21
22
23
24
25
26
27
28
29
September
30
October
Ernesto Blanco; inhalation anthrax
Florida
19
20
21
22
23
24
25
26
27
28
29
September
30
10
11
12
10
11
12
10
11
12
October
Washington
18
19
20
21
22
23
24
25
26
27
28
29
September
30
October
Connecticut
18
19
20
21
22
23
24
25
26
27
28
September
29
30
October
Anthrax cases are listed by the victim's name, anthrax type, and illness onset date. Deaths are listed separately.
13
14
15
16
17
18
19
20
21
22
23
Kathy Nguyen;
inhalation anthrax.
24
25
26
27
28
29
30
31
1 - //
14
//
21
//
21
//
21
November
Patrick ODonnell; cutaneous anthrax.
Jyotsna Patel; inhalation anthrax.
Linda Burch; cutaneous anthrax.
Norma Wallace; inhalation anthrax.
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
1 - //
14
November
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
1 - //
14
November
Leroy Richmond; inhalation anthrax. George Fairfax; inhalation anthrax. Thomas Morris, Jr.; inhalation anthrax. Joseph
Curseen, inhalation anthrax.
Thomas Morris, Jr. dies.
Joseph Curseen dies. David Hose; inhalation anthrax.
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
1 - //
14
//
21
November
Ottilie Lundgren; inhalation anthrax.
Ottilie Lundgren
dies.
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
1 - //
14
//
21
November
16 Chapter 2
Map 2.1 Example of a Map Graphic Depicting the Spatial and Temporal Aspects of the Attacks
Derby, CT
14 November; 21 November
New York, NY
21 September
28 September
29 September
1 October
1 October
19 October
23 October
25 October; 31 October
Hamilton Township, NJ
26 September
28 September
14 October
15 October
17 October
Legend
Non-italics = cutaneous case
19 October
Washington, DC
16 October
16 October
treatment sought.
Two dates= onset and death.
16 October; 21 October
16 October; 22 October
22 October
Boca Raton, FL
1 October
2 October; 5 October
Definition
Analytic mindset
Anchoring
Confirmation
bias
Historical
analogy
Mirror imaging
Premature
closure
Satisficing
18 Chapter 2
linking the anthrax to USAMRIID. This lack of
evidence should challenge the level of certainty
that Hatfill should be named as a person of interest
until the circumstantial evidence can be thoroughly
reviewed.
Neither is there evidence, either direct or indirect,
linking Hatfill to NBC or Tom Brokaw, the New York
Post, or Senators Daschle and Leahy.
Step 8: Have you considered the presence of common
analytic pitfalls such as analytic mindsets, confirmation
bias, satisficing, premature closure, anchoring, and
historical analogy?
Confirmation bias. The case against Hatfill could
represent confirmation bias. No physical evidence
links Hatfill to the crime, yet he is publicly named
a person of interest. The evidence against him is
entirely circumstantial and deserves greater scrutiny.
The presence of several pieces of circumstantial
evidence that the government found once it focused
on him as a suspect may have had the unintended
consequence of raising the governments confidence
in Hatfills guilt. As a result, each piece of evidence
deserves greater scrutiny to ensure that the decision
to name Hatfill as a person of interest is not a
result of confirmation bias. For example, are there
alternative explanations for why Hatfill was taking
Cipro in 2001?
Satisficing/Premature Closure. The government
interviewed Hatfill and searched his home on
25 June. No charges were brought against him
at that time. As pressure mounted to identify
the perpetrator, however, the government again
searched his home on 1 August. Pressurewhether
explicit or implicitmay have caused investigators
to come to the first, most plausible explanation
(satisficing) without fully investigating the other
possible suspects or tracking down questions about
circumstantial or anomalous evidence (premature
closure). In law enforcement spheres, this is called
detective myopia.
Step 9: Based on the answers to the themes of inquiry just
outlined, list the potential deficiencies in the argument in
order of potential impact on the analysis.
The lack of physical evidence linking Hatfill to the
crime raises uncertainty about his guilt, even in the
face of other circumstantial evidence.
20 Chapter 2
presumably to throw investigators off his trail; threw out a
book on codes that he may have used to embed codes into
the anthrax letters; and gave the FBI questionable samples
of RMR-1029 in order to conceal his activities from
investigators.10
Investigators also pointed to Ivinss mental health status,
noting his use of alternate identities, his 40-year-long
obsession with the Kappa Kappa Gamma (KKG) sorority
during which he burglarized chapter houses, and his
inability to explain his own suspicious behavior. The task
force found that not only were the anthrax letters sent from
a New Jersey mailbox outside a KKG chapter at Princeton
University, but also Ivins was unable to provide reasonable
or consistent explanations for his behavior, such as his late
night hours and submission of questionable samples of
RMR-1029.11
Still, given Ivinss untimely death, and the fact that the
government could not take the case to trial, not everyone
accepted the governments explanations. Ivinss lawyers
posthumously defended their client, calling the charges
heaps of innuendo and a total absence of proof that he
committed this crime.12 Some of his colleagues accused
the government of hounding an innocent man to
suicide.13 Later, when the government closed the case in
February 2010 and released to the public thousands of
documents related to the case, his colleagues still raised
doubts that he could have perpetrated the crime. In an
email quoted in the documents released by the
government, Ivins posthumously offers his own
explanation for some of his erratic behavior, blaming an
alter ego, Crazy Bruce, who surfaces periodically as
paranoid, severely depressed and ridden with incredible
anxiety.14
Over a decade after the attacks, questions still remain. A
2010 report by the National Research Council found that it
is not possible to reach a definitive conclusion about the
origins of the anthrax in letters mailed to New York City
and Washington, D.C., based solely on the available
scientific evidence. 15 The report specifically calls into
question the RMR-1029 flask, indicating that while the
NOTES
1. Amerithrax Investigative Summary, Department of
Justice, February 19, 2010, www.justice.gov/amerithrax, 3.
2. The steps as outlined in this case combine the processes
for a Premortem Analysis and Structured Self-Critique. This
Analytic Family
p. 47
p. 209
Devils Advocacy
p. 260
Challenge Analysis
3 Cyber H20
Cases in Intelligence Analysis: Structured Analytic Techniques in Action
Instructor Materials
23
24 Chapter 3
via a foreign-based IP address. The implications of this are
far-reaching because it would be the first such reported
incident and could signal a new trend in activity that could
have reverberations across not only the water sector, but
also other sectors that utilize industrial control systems.
Step 4: Has your organization or any other organization
ever answered this question or a similar question before,
and, if so, what was said? To whom was this analysis
delivered, and what has changed since that time?
This is a first for the water sector and for US
infrastructure, but there have been other instances, such as
in Australia, in which an insider has compromised a waste
water system.
Step 5: Who are the principal customers? Are these
customers needs well understood? If not, try to gain a
better understanding of their needs and the style of the
reporting they like.
The customer set includes federal, state, and local
officials, as well as industry. At the federal level, interest will
be high because of the possible implications of such an attack
for other types of infrastructure, the broader economic
impact, and the potential national security implications. At
the state and local level, interests will center on the
implications for the water customers and the economic
effects. Industry will be interested in all of these issues.
Step 6: Are there other stakeholders who would have an
interest in the answer to this question? Who might see the
issue from a different perspective and prefer that a different
question be answered? Consider meeting with others who
see the question from a different perspective.
At the federal level, DHS Cyber Emergency Response
Team (CERT) is an important resource for cyberforensics.
At the industry level, the WaterISAC may have expertise
that could be brought to bear. The Curran-Gardner
employees and contract staff may also be able to provide
more context for analysts regarding the timing, location,
pump type, and SCADA system logs.
Step 7: From your first impressions, what are all the
possible answers to this question? For example, what
alternative explanations or outcomes should be considered
before making an analytic judgment on the issue?
While the initial reports suggest that a hacker caused the
pump failure, other possible explanations could include a
cyber-savvy insider or a mechanical failure.
Cyber H20 25
interpretation of evidence and reasoning about any particular problem. Assumptions are usually a necessary and
unavoidable means of filling gaps in the incomplete,
ambiguous, and sometimes deceptive information with
which the analyst must work. They are driven by the analysts education, training, and experience, including the cul
tural and organizational contexts in which the analyst lives
and works. It can be difficult to identify assumptions,
because many are sociocultural beliefs that are unconsciously or so firmly held that they are assumed to be truth
and not subject to challenge. Nonetheless, identifying key
assumptions and assessing the overall impact should they
be invalid are critical parts of a robust analytic process.
Task 2.
Conduct a Key Assumptions Check of the prevailing judgment that the pump failure was caused by a Russian-based
intrusion using stolen SCADA system log-on credentials.
Step 1: Gather a small group of individuals who are
working on the issue along with a few outsiders. The
primary analytic unit already is working from an established
mental model, so the outsiders are needed to bring other
perspectives.
1. Basically supported
2. Correct with some caveats
3. Unsupported or questionablethe key
uncertainties
Step 7: Refine the list, deleting those assumptions that do
not hold up to scrutiny and adding new assumptions that
emerge from the discussion.
Commentary
Solid
With Caveat
Unsupported
26 Chapter 3
Commentary
Supported
With Caveat
Unsupported
Task 3.
Build the strongest possible case against the prevailing judgment that the pump failure was caused by a Russian-based
intrusion using stolen SCADA system log-on credentials.
Steps: Although there is no prescribed procedure for a
Devils Advocacy, begin with the analytic judgment,
assumptions, and gaps. These can serve as a useful starting
point from which to build the case against the original
judgment that the pump failure was caused by a Russian-based
intrusion using stolen SCADA system log-on credentials.
Next, build a logical argument that undermines each goal.
It is too early to conclude that the pump failure was
caused by a Russian-based intrusion using stolen SCADA
system log-on credentials. The basis for the judgment is an
unsupported assumption that the so-called attack originated
in Russia and was conducted using stolen log-on
credentials. While previous government- and industrysponsored experiments have demonstrated this capability
on the part of hackers, we cannot rule out other possible
explanations at this time. Barring further investigation and
collection of information from the site of the pump failure
and US government cyberforensic specialists, it is just as
likely that the cause of the failure is attributable to an
insider or a simple equipment malfunction.
Analytic Value Added: Which issues could undermine
the analysis, and why? Unsupported assumptions and
Cyber H20 27
28 Chapter 3
the FBI have concluded that there was no malicious or
unauthorized traffic from Russia or any foreign entities,
as previously reported.10
NOTES
1. ICSB-11-327-01Illinois Water Pump Failure Report,
Department of Homeland Security Industrial Control Systems
Cyber Emergency Response Team, November 23, 2011, http://www
.us-cert.gov/control_systems/pdf/ICSB-11-327-01.pdf.
2. Ibid.
3. ICS-CERT Monthly Monitor, Department of Homeland
Security Industrial Control Systems Cyber Emergency Response
Team, December 2011, http://www.us-cert.gov/control_systems/
pdf/ICS-CERT_Monthly_Monitor_Dec2011.pdf.
4. Kim Zetter, Exclusive: Comedy of Errors Led to False
Water-Pump Hack Report, Wired, November 30, 2011, http://www
.wired.com/threatlevel/2011/11/water-pump-hack-mystery-solved.
5. ICSCERT and FBI Statements on Water System Attacks,
InfosecIsland, November 22, 2011, http://www.infosecisland.com/
blogview/18303-ICS-CERT-and-FBI-Statements-on-WaterSystem-Attacks.html.
6. Ibid.
7. Zetter, Exclusive: Comedy of Errors Led to False WaterPump Hack Report.
8. Ibid.
9. Mickey McCarter, Infrastructure Security: DHS, FBI
Dispel Allegations of Illinois Water Pump Attack, Homeland
Security Today, November 30, 2012, http://www.hstoday.us/focusedtopics/infrastructure-security/single-article-page/dhs-fbi-dispelallegations-of-illinois-water-pump-hack.html.
10. ICSB-11-327-01Illinois Water Pump Failure Report,
Department of Homeland Security Industrial Control Systems
Cyber Emergency Response Team, November 23, 2011, http://www
.us-cert.gov/control_systems/pdf/ICSB-11-327-01.pdf.
11. Ibid.
Analytic Family
p. 304
Decision Support
Deception Detection
p. 198
Premortem Analysis
p. 240
Challenge Analysis
Structured Self-Critique
p. 245
Challenge Analysis
29
30 Chapter 4
There is no doubt that Lee moved large quantities of data
from a classified computer to an unclassified computer. The
question is why. Was he told to archive the data? Was he
afraid of losing his job and did he want to keep a copy of his
notes? Did he put the data on tape drives to pass to the
Chinese? Although Lee requested remote access to a
classified system while in Taiwan, he did not do so
surreptitiously. Some would point to his questionable
security practices as evidence that he was trying to conceal
clandestine activities; others would point out that he was
simply absentminded.
The case study does not include information about Lees
financial situation or whether his colleagues at the lab
exhibited similar behavior and security lapses. Neither does
the case contain any information about Wen Ho Lees
attitude toward the management at Los Alamos National
Laboratory (LANL) nor whether he felt denied opportunity
or otherwise disadvantaged. These potential driving forces
would be topics of investigation and analysis and at the very
least represent gaps that should be discussed.
Step 3: Make one list showing the strongest arguments
supporting Wen Ho Lees innocence and another list
showing the strongest arguments showing his guilt.
Step 4: Array the lists in a table like Table 4.2 in the book.
Table 4.5 shows an example response.
Arguments For
Arguments Against
Weight
When visiting LANL, Hu Side hugged Lee and thanked him for
his help.
Financial trouble?
Total
Total
17
20
deception is well done, one should not expect to see evidence of it. There are, however, some indicators that should
alert analysts that they may be the targets of deception, such
as the timing of reporting or the bona fides of a source, or
when there are known and potentially serious consequences
if the source is believed.
For illustrative purposes, we have focused this Deception
Detection example on the provenance of the walk-in
document that catalyzed the case. The same process,
however, could be used to examine the possibility of
deception surrounding any of the actors or evidence in the
case.
Task 2.
Use Deception Detection to determine whether deception
may be occurring in the case of Wen Ho Lee.
Step 1: Using Table 4.3 in the book as your guide,
determine whether Deception Detection should be
conducted. Assuming that the United States and the FBI
would be the target, who would be the most likely
perpetrators of deception? If a case can be made that
someone may have a motive to deceive, state this as a
hypothesis to be proved or disproved. Note which indicators
best apply to this case. Table 4.6 shows a sample response.
Information suggesting
indicators may be true:
(Continued)
32 Chapter 4
Information suggesting
indicators may be true:
Analysts may have assumed
prior to the walk-in that the
Chinese could have received
help from the Russians or could
have developed the warhead
on their own. The walk-in
information would lead them
to consider an espionage
hypothesis more seriously.
The walk-in information
prompted both the Department
of Energy and the FBI to
expend substantial resources
investigating LANL and Wen
Ho Lee.
The Chinese almost certainly
have other sources at DOE and
the National Labsor people
in contact with employees
therewho could report that
an investigation was underway.
To protect a real or more productive spy by casting suspicion on someone else, namely
Wen Ho Lee.
To get rid of Wen Ho Lee if he was becoming a troublesome source.
To confuse any investigation while continuing to procure valuable intelligence.
Double agents feeding information to a known intelligence organization such as the FBI or
the CIA.
Providing the US government with authentic documentation through a walk-in, for
example, a report with drawings that contained more than public information.
Participating in routine scientific exchanges with national lab personnel.
The entire system of Chinese intelligence gathering offers deniability or the option of casting suspicion on multiple actors.
Only basis is the actual documentation provided, but that could be part of the deception
operation.
Little information about the access or background of the source; not a recruited source.
Unlikely the source would be trying to please the collector or obtain more revenue because
there is no established relationship between the source and the collector; it is feasible,
however, that the source may have been promoting a personal agenda.
Shows a high level of detail but not entirely consistent with what we know Wen Ho Lee to
have worked on.
No other sources of information to collaborate what was provided by the walk-in. No
conflicts but also no independent collaboration.
No other sources of information to collaborate what was provided by the walk-in. No
conflicts but also no independent collaboration.
Care was taken to translate the documents well; the sketches speak for themselves.
34 Chapter 4
in order to broaden the range of possible explanations or
estimates that are seriously considered. This process helps
reduce the risk of analytic failure by identifying and analyzing
the features of a potential failure before it occurs.
Task 4.
Conduct a Premortem Analysis and Structured SelfCritique of the reigning view in the case study that Wen Ho
Lee passed nuclear secrets to the Peoples Republic of China.
Step 1: Imagine that a period of time has passed since you
concluded that Wen Ho Lee was guilty of espionage. You
suddenly learn from an unimpeachable source that the
judgment was wrong. Then imagine what could have
happened to cause the analysis to be wrong.
The first two steps comprise the Premortem Analysis.
This right-brain-led, creative brainstorming process asks
analysts to imagine a future in which they have been proved
wrong and work backward to try to identify the possible
causes. In essence, they are identifying the weak links in
their analysis in order to avoid these potential pitfalls prior
to publishing the analysis or, in this case, bringing a case to
prosecution. Most analysts are more left brained than right
brained, which often makes imagination techniques like
brainstorming challenging. However, when coupled with
the Structured Self-Critique, the systematic, left-brained
checklist that comprises steps three through eight,
brainstorming can be the first step toward identifying
sometimes fatal analytic flaws. It is important to encourage
students to be as creative as possible when brainstorming,
keeping all ideas in play.
In this case, a brainstorming session might prompt
students to consider the following:
Was Wen Ho Lees behavior any different than that
of his colleagues? For example, were his security
indiscretions atypical, or did his colleagues often act
in the same way, forgetting to report meetings or
revealing controlled but not classified information to
foreign nationals without permission?
Was it suspicious or insignificant that Wen Ho Lee
entered the lab at 3:30 a.m. Christmas Eve? Was he a
Christian who celebrated Christmas? Did he and his
colleagues often work late hours?
Was Wen Ho Lee a member of a broader network
that was exploited by Chinese intelligence but did not
provide any actual secret information to the Chinese?
If so, who else might be in this network? Who else
Assessment
On the other hand, the fact that Wen Ho Lee did not
download computer manuals is inconsistent with the
alternative hypothesis that he was only archiving nuclear data
he worked on.
36 Chapter 4
consistent with both hypotheses: that the walk-in was
legitimate or that the Chinese decided to provide detailed
information to make the walk-in look credible in the eyes of
US government officials.
Step 7: Is there an absence of evidence, and does it
influence the key judgment? Table 4.9 shows an example
response.
Confirmation
bias
Satisficing
Historical
analogy
Assessment
Assessment
Being a Spy. He also completed a textbook on applied physics, which he began writing while he was in prison.4
KEY TAKEAWAYS
Application of structured analytic techniques to the Wen
Ho Lee case underscores the need to:
Always challenge inherited assumptions. The
Department of Energy presented the FBI with the
findings of an administrative inquiry that was based
on several keyand unchallengedassumptions.
Before launching the investigation of Wen Ho
Lee, it is important to critically examine the key
assumptions upon which the DOE case was based.
Be open to alternative hypotheses. When data are
inconsistent with the lead hypothesis, stop and ask
yourself if there are alternative and more compelling
explanations for the behavior being observed.
Make time to reflect, especially at the start of a new
project or investigation. When operating under
major time constraints and substantial pressure
from above to produce, avoid the temptation to
plunge in. The need to employ structured analytic
techniques, like a Key Assumptions Check, is greatest
when the stakes are high. A quick answer will satisfy
your customer for the moment, but you will have to
live with a wrong answer for the rest of your life.
NOTES
1. The steps as outlined in this case combine the processes
for a Premortem Analysis and Structured Self-Critique. This combination is particularly helpful in cases that require analysts to
think broadly, imaginatively, and exhaustively about how they
might have been wrong. The Premortem Analysis taps into the
creative brainstorming process, and the Structured Self-Critique
provides a step-by-step assessment of each analytic element. To aid
students learning process, the questions in this case have already
been narrowed from the fuller set of Structured Self-Critique
questions found in Richards J. Heuer Jr. and Randolph H. Pherson,
Structured Analytic Techniques for Intelligence Analysis, 2nd ed.
(Washington, DC: CQ Press, 2015).
Table 5.1 Case Snapshot: Jousting with Cuba over Radio Marti
Structured Analytic Technique Used
Analytic Family
p. 56
Deception Detection
p. 198
p. 175
p. 181
1982
(Continued)
39
40 Chapter 5
Step 2: Select the most relevant information from the case
narrative. Consider how best to array the data along the
Timeline. Can the information be organized by category?
Construct a Timeline of the Radio Marti case.
A Timeline that contrasts US actions with Cuban actions
is provided in Figure 5.3.
Step 3: Review the Timeline by asking the following questions: Should any underlying assumptions about the evidence be taken into consideration? Do the duration and
sequence of events suggested by the data make sense? Are
there data gaps? Could any events outside the Timeline have
influenced the activities?
A review of the Timeline suggests four major
observations:
Ronald
Reagan
elected
US
President
US
Establishes
Presidential
Commission
US
Government
Actions
Timeline
Cuban
Government
Actions
Jan
Jan
1979
1980
Cuba
announces
plans for two
500 kW
transmitters.
Struggling as
economic
crisis spawns
popular
discontent.
US urged
by Florida
broadcasters
to jam
Cuban
radios
US
Senate
Committee
approves
legislation
US
Senate
opts not
to take
up bill
Aug Sep
1981
US officials
tell
commercial
broadcasters
major countermeasures are
being
considered
Aug Sep
1982
During technical
discussions,
Cuba says it will
broadcast over
two 500 kW
stations using
Radio Martis
1040 kHz
frequency
US Senate
passes bill
with VOA
standards
but House
amends it
to make it a
surrogate home
broadcasting
service
Dec Jan
1983
Cuba
disrupts
broadcasts
of WHO
and
several
other radio
stations
Havana
agrees to
engage in
radio
interference
talks
May
President
Reagan signs
bill to
establish
Radio Marti
SepOct
1984
Cuba refuses
to continue
radio
interference
talks
because
Radio
Marti bill
signed
into law
42 Chapter 5
cannot be rejected simply because there is no evidence of it;
if deception is well done, one should not expect to see evidence of it. There are, however, some indicators that should
alert analysts that they may be targets of deception, such as
the timing of reporting, the bona fides of a source, or when
believing what a source says could have known and potentially serious consequences.
Cuba had been engaged in adversarial relations with the
United States for two decades before the Reagan administration came into office. Both sides had employed the full range
of diplomatic and military tactics, including the threat posed
by nuclear missiles on Cuban soil. The Soviet Union and its
external intelligence service (the KGB) had mentored and
supported the Cuban service. The KGB had a long history of
using perceptions management and deception. Given these
background circumstances, analysts need to be alert to the
possibility that the opposition would employ perceptions
management and/or deception to help achieve its goals.
Task 2.
Using Deception Detection techniques, determine whether
Cuba might be employing perceptions management and/or
deception against the United States.
The Cuban governmentas well as its Soviet allyhas a long history of employing
deception.
Cuban threats and actions were often received in response to critical congressional
actions on Radio Marti. Both public and private statements suggested that the Cuban
government believed it had much to lose if the United States began broadcasting to
Cuba. It was concerned that Radio Marti programming would publicize the failures of the
revolutionary government and help foment discontent with the regime.
Accepting reports that Cuba was preparing to jam or otherwise interfere with US
radio broadcasting could prompt the US Congress to decide not to initiate broadcasts,
anticipating the commotion this might generate in the business community.
Accepting reports that Cuba was preparing to jam or otherwise interfere with US radio
broadcasting prompted Washington to develop costly countermeasures.
The Cubans had a timely, accurate feedback channel throughout this period in the
form of congressional reaction to its various threats and the access to questions about
Radio Marti received by its double agents. In addition, its own penetrations of the US
government, discovered or undiscovered, may have been able to provide additional
reporting.
Table 5.7 Radio Marti: Assessing the Likelihood of Cuban Deception with MOM, POP, MOSES, and EVE
Motive, Opportunity, and Means (MOM):
In the case of Radio Marti, the Cuban goal was clear: prevent Radio Marti from broadcasting to
Cuba as a surrogate radio service providing a source of internal news not controlled by the Castro
regime. To thwart the US administrations plan, Cubas best tactic was to prevent passage of the
legislation in the US Congress, or cause Congress to modify the broadcast content of Radio Marti
so that it would not cause internal problems for the Cuban government. Threats to disrupt US
broadcasts if Radio Marti began broadcasting were a tactic designed to encourage opposition of
powerful US commercial interests and their representatives in Congress to oppose Radio Marti.
The United States was receiving information about Cubas intentions through multiple channels.
Open sources included public statements by Cuban diplomats and other officials. Diplomatic
exchanges in multiple forums provided additional information. Cubas demonstration of the
power of its transmitters to disrupt US broadcasts provided both open information and data for
technical analysis of the capabilities of the transmitters. In addition, if Cuba could control some
or all of the oppositions clandestine collection of intelligence about Cuban intentions, it could
influence US perceptions of its intentions.
Given the Cubans objective of thwarting the Reagan administrations plans for Radio Marti,
if the deception failed or was detected and failed, the worst that could happen would be that
Radio Marti would start up, probably sooner rather than later because the administration would
not need to prepare countermeasures and would not be running the political risks involved with
Cuba disrupting US radio broadcasting. Detection of a deception operation also runs the risk
that the opposition will identify the means by which the deception is being conducted. The risk
to the Cubans would be calculated in terms of the value of those means.
Castros intentions were the critical information in this case. If Castro were providing that
information as part of the deception or perceptions management campaign, no sensitive
information would be lost and there would be no cost.
The Cubans had rich sources of feedback on a potential deception. The response of the main
target, the US Congress, and various interest groups provided an excellent means of monitoring
the impact of a deception and its continuing credibility. If the Cubans controlled some or all of
the clandestine information, they could gain some insights about how the opposition assessed
the information and its impact on their analysis by evaluating the follow-up questions asked of
their controlled sources.
Past Opposition Practices (POP):
The clandestine introduction of Soviet nuclear missiles into Cuba represented one of the great
strategic deceptions of the 20th century. The Cubans were partners and enablers in that deception.4
Deception is often used by a weak or weaker power against a stronger adversary. In that sense,
the possibility of Cuban deception would fit a well-established universal pattern of deception.
The specifics of this case indicate that Cuba would have a motive for deceiving the United
States about its intentions to disrupt radio broadcasting. However, no specific information was
available at the time to indicate whether or not they would disrupt broadcasts.
The Cuban Missile Crisis provides a robust historical precedent for attempting to deceive the
United States.
The Cubans had the potential to manipulate all of the open sources providing information about
their position on Radio Marti. Furthermore, they had the ability to coordinate their open source
information with any controlled clandestine collection.
Open sources could be manipulated at will. Technical information derived from open sources would
be much more difficult to manipulate. Specifically, the capabilities of the Cuban transmitters to
disrupt US radio broadcasts were subject to standard technical analytic techniques. Clandestine
human sources can always be manipulated if controlled. In addition to standard counterintelligence
tradecraft used to vet sources, the specific sources reporting on Radio Marti could be evaluated, in
part, by the consistency of their reporting with other sources of information.
(Continued)
44 Chapter 5
Table 5.7 Radio Marti: Assessing the Likelihood of Cuban Deception with MOM, POP, MOSES, and EVE
(Continued)
Does the source have direct access or only
indirect access to the information?
In this case, whether sources had direct access to the information or not would not provide the
analysts with any means to judge whether Castro knew what he would do at the end of the day,
was telling the truth to the source, or was manipulating the source.5
Even if the source had been reporting for a substantial period of time, the question is whether
the source was controlled, and, if so, at what point was he controlled.
Not applicable.
In this case, analysts had a substantial body of sources derived from open, clandestine, human,
and technical means of collection.
The critical unknown was how Fidel Castro would respond when and if Radio Marti began to
broadcast to Cuba; that could only be determined at the last minute. The United States would
likely learn of that final decision by listening to US radio stations.
No. But analytically, this could be a sign of deception. Conflicts and inconsistencies are the norm
in intelligence collection.
No. However, as noted, no evidence could answer the ultimate questionwhat would Fidel do
when he heard Radio Marti in Havana?
TECHNIQUE 3: MULTIPLE
HYPOTHSIS GENERATION:
QUADRANT HYPOTHESIS GENERATION
Many techniques can be used to help generate a set of
hypotheses, including basic brainstorming, Simple
Hypothesis Generation using the Structured Brainstorming
technique, Quadrant Hypothesis Generation using a 2 2
matrix to structure the process, and the Multiple Hypotheses
GeneratorTM. The Multiple Hypotheses GeneratorTM is a
software tool that applies the journalists classic set of questions (Who? What? How? When? Where? and Why?) to
develop a set of mutually exclusive hypotheses by generating
permutations of the lead hypothesis.6
Of the four techniques just mentioned, basic brainstorming is the least rigorous because it simply involves listing
what first comes to mind. Such an unstructured process
usually fails the key test of hypothesis generation: that the
set of hypotheses generated should be comprehensive and
mutually exclusive. The other three techniques are more
likely to pass this test if performed correctly.
In this case study, Quadrant Hypothesis Generation
would be a good choice because the analytic challenge can
be defined along two key dimensions: what range of options
the Cubans might consider and how serious the impact
might be on the United States. By creating four mutually
exclusive quadrants, each defined by different endpoints of
the two key dimensions, the Quadrant Hypothesis Generation process reframes the question in four different ways,
spurring more creativity and ensuring a more comprehensive analytic approach.
Task 3.
Use the Quadrant Hypothesis Generation technique to
develop a set of three to five hypotheses that address the
question: How will Cuba respond to the launch of Radio
Marti broadcasts?
Step 1: Identify two key dimensions or drivers influencing Cubas decision making about how to respond
using Structured Brainstorming or drawing from expert
analysis.
The two primary actors in this case study are Cuba and
the United States. In determining a set of key drivers or key
dimensions of the issue, this is the best place to start. With
regard to Cuba, the key question is: What is Castros underlying objective? Is he determined to prevent Radio Marti
from broadcasting regardless of the consequences, or would
46 Chapter 5
Impact on the
United States
SEVERE
DELAY
or
MODIFY
Cuban Objectives
PREVENT
MILD or NONE
Description
Comment
DELAY
or
MODIFY
Disrupt US AM
broadcasting to
prevent launch of
Radio Marti
Impact on
United States
Damage or destroy
Radio Marti by sabotaging its facilities
in Florida
Cuban Objectives
Threaten to disrupt US
broadcasts or punish
the Cuban American
community in Florida
PREVENT
MILD or NONE
48 Chapter 5
Task 4.
Use the ACH software to identify which hypotheses provide
the most credible explanation in answering this question:
How will Cuba seek to delay or prevent Radio Marti from
broadcasting? The basic ACH software is available at http://
www.globalytica.com or from the Palo Alto Research
Center at http://www2.parc.com. A collaborative version of
ACH called Te@mACH can be accessed at http://www
.globalytica.com.
Step 1: Select three to five hypotheses based on the results
of Quadrant Hypothesis Generation exercise, striving for
mutual exclusivity.
The principal concern of the US stakeholders was that
Cuba would disrupt commercial radio broadcasts across the
country. However, posing the intelligence question in a
broader form, How will Cuba seek to delay or prevent
Radio Marti from broadcasting? includes other possible
responses by the Cubans. So the first step in structuring the
analysis is to pose the question properly to ensure that the
full range of possible outcomes is considered.
A hypothesis is essentially a persons best guess to answer
a question. According to Heuer and Pherson, in an ACH
exercise, Hypotheses should be mutually exclusive; that is, if
one hypothesis is true, all others must be false. The list of
hypotheses should include all reasonable possibilities. Include
a deception hypothesis if that is appropriate.9 In the case of
hypotheses related to Radio Marti, some of the hypotheses
would be mutually exclusive only because of the intent of the
Cubans, not their capabilities to disrupt US broadcasts. A set
of hypotheses to consider is provided in Table 5.9.
Hypothesis
1.
2.
Cuba Sabotages Radio Marti facilities to delay or prevent Radio Marti broadcasts
3.
4.
5.
Despite Cubas signing of the North American Radio Broadcasting (NARB) Agreement in 1950, Cuban interference on the AM band begins
to grow in the 1960s after Castro comes to power; by the 1970s, it is a serious problem.
2.
In 1979, Cuba submits an inventory to ITU that includes plans for two radio stations transmitting with 500 kW of powera volume ten
times the limit permitted to any US radio station.
3.
The collapse of the Soviet Union and its economic subsidies severely damages the Cuban economy, resulting in an explosion of popular
discontent.
4.
In August 1981, Cuba says it intends to shift the frequencies of its 500 kW stations to 1040 kHz and 1160 kHz.
5.
In 1982, the Board of Directors of the Florida Association of Broadcasters adopts a resolution urging the United States to jam Cuban radio
broadcasts until illegal interference from Cuba ends.
6.
Technical intelligence sources confirm the location of the Cuban broadcasting stations.
7.
The Federal Communications Commission (FCC) estimates that, at full power, the two 500 kW transmitters could be heard as far away as
Alaska and Hawaii.
8.
On 30 August, the Cuban transmitter broadcasts on 1040 kHz for several hours at 150 kW (three times the US legal maximum), causing
significant interference with WHOs broadcasting and several other US radio stations.
9.
The National Association of Broadcasters, citing the broadcasts, lobbies Congress on behalf of farmers and truckers to delay implementation of Radio Marti, and the Senate decides not to take up the legislation.
10.
The New York Times reports in May 1983 that senior US officials have told commercial broadcasters that a list of some forty US countermeasures are being considered if Cuba interferes with US radio stations, including destruction of offending Cuban transmitters.
11.
An amended version of Radio Marti legislation passes the US House of Representatives, stating that Radio Marti must adopt Voice of
America (VOA) standards.
12.
Congress finally passes Radio Marti legislation in September 1983, with a legislative history that enables Radio Marti to become a surrogate home broadcasting service for Cuba.
13.
14.
Radio Marti is set to broadcast from Florida at 50 kW on 1040 kHz, which will not interfere with the signal of radio station WHO in Des
Moines, Iowa.
50 Chapter 5
52 Chapter 5
Marti. That said, after Castro did not disrupt US
AM broadcasting, some hard questions about the
reliability of the key sources could have been asked.13
KEY TAKEAWAYS
Structured analytic techniques provides one of
the best mechanisms for overcomingor, at least,
mitigating the effects ofcognitive traps and
mental mindsets that lead to making poor analytic
NOTES
1. Kenneth N. Skoug Jr., The United States and Cuba Under
Reagan and Shultz: A Foreign Service Officer Reports (Westport,
CT: Praeger, 1996), 17.
2. E.O. 12323. The Federal Register.
3. Skoug, The United States and Cuba Under Reagan and
Shultz, 19.
4. For a detailed treatment of the Cuban Missile Crisis case,
see Graham Allison and Philip Zelikow, Essence of Decision:
Explaining the Cuban Missile Crisis (New York: Longman, 1999).
5. Skoug, The United States and Cuba Under Reagan and
Shultz, 27; Michael Wines and Ronald J. Ostrow, Cuba Exults
That CIAs Men in Havana Were Double Agents; In a Television
Series, Alleged Spies-Turned-Heroes Tell How They Duped
American Agency, LA Times, August 12, 1987.
6. For more information on the Multiple Hypotheses
GeneratorTM, go to http:// www.globalytica.com.
7. See Gary King, Robert O. Keohane, and Sidney Verba,
Designing Social Inquiry (Princeton, NJ: Princeton University
Analytic Family
p. 209
Devils Advocacy
p. 260
Challenge Analysis
Strengths-Weaknesses-Opportunities-Threats
p. 308
Decision Support
53
54 Chapter 6
Step 2: Ideally, participants should be asked to bring a
list of assumptions when they come to the meeting. If not,
start the meeting with a silent brainstorming session. Ask
each participant to write down several assumptions on 3 5
cards.
Step 3: Collect the cards and list the assumptions on a
whiteboard for all to see. A simple template can be used, as
in Table 6.4 in the book.
An initial list of brainstormed Key Assumptions for this
case might include several higher-order assumptions such
as the following:
Commentary
Supported
With Caveat
Will they feel safe using the road? Perhaps while the US
military is there, but Soviet history suggests an ongoing
security presence will be necessary.
Unsupported
Commentary
Supported
With Caveat
Unsupported
The road will benefit anyone who can and does use it;
this includes the Taliban, which may be interested in
using the road for its own purposes.
56 Chapter 6
Step 7: Refine the list, deleting those assumptions that do
not hold up to scrutiny and adding new assumptions that
emerge from the discussion.
This process reveals that it is important to amend
assumptions to capture important nuances, such as by
disaggregating the assumption that the local populace wants
and needs the road. This process also reveals new
assumptions that underpin initial assumptions. One
example is the assumption that the road will improve
commerce in the region and, in turn, that the Afghan
government has the capacity to use it to promote commerce.
Step 8: Consider whether key uncertainties should be
converted into collection requirements or research topics.
In this case, several key uncertainties stem from the
assumption that the road will improve voter participation,
security, commerce, and the central governments reach.
Other key uncertainties are that a functioning road will
benefit the Afghan government, locals, and US/NATO
forces more than the Taliban and that the Taliban will
continue to oppose US/NATO presence at its current,
manageable level. Both of these warrant additional
research into how much permanent security presence
(US, NATO, or Afghan) will be required for the roads
continued use.
Analytic Value Added: What impact could
unsupported assumptions have on the decision to build the
road? How confident should military decision makers be
that the benefits of building the road will outweigh the
risks? Much of the strategy is premised on assumptions that
may be valid in the Western context but are questionable
when applied to Pashtun culture. As a result, it cannot be
assumed that the locals will be grateful for the road and will
express that gratitude through participation in a democratic
process. Neither can it be assumed that the locals
including the Talibanintend to use the road in the ways
envisioned by the United States.
Another key factor in this analysis is the behavior of the
Taliban forces in the region. If the Taliban increases the
magnitude of its campaign against the United States and
cooperative locals, it could significantly affect the ability of
the United States to build the road in a timely and secure
manner and the roads impact on local opinion. The
decision to pursue construction is based in part on the
assumption that Taliban operations will remain at their
current level and that the United States can suppress any
change in that level.
TECHNIQUE 3: STRENGTHS-WEAKNESSESOPPORTUNITIES-THREATS
Task 3.
Conduct a SWOT analysis of the pending decision to spur
economic development, promote central governance, and
improve security in the region by building a road connecting Kandahar City to Tarin Kowt prior to the September
election.
Step 1: Clearly define the objective.
58 Chapter 6
US Weaknesses
Sufficient funding.
Taliban could exploit finished road to finance and support its own
operations at the expense of the United States.
Taliban could use the road for propaganda purposes to turn locals
against the project.
The US engineers will be blamed for any errors or accidents during
construction.
Supply line is threatened by the remote environment and by
insurgents.
Successful construction could saddle Afghan government with
expensive upkeep.
Improve Weaknesses
Construct logistic bases along road route and preposition needed
supplies.
Use local national interpreters and cultural advisors to identify tribal
leaders.
Establish small civil affairs units to work with local population.
Request infantry and air assets in support of the mission.
Rotate in new equipment or work at less hot times of the day.
Exploit Opportunities
Use early outreach to discuss and vet the route with local village
elders.
Use air superiority to deliver supplies.
Use local construction forces when possible.
Mitigate Threats
Empower the village elders so that they see the benefits of the road and
will be more inclined to accept any unforeseen problems that arise in
construction.
Use locals to deliver supplies and augment this with air supply.
Use US Infantry units to flush out Taliban forces from surrounding mountains.
Use of locals on construction teams could slow the process, but could
redound to US advantage if it helps establish a workforce knowledgeable
about road upkeep and capable of providing needed information about
surrounding local and insurgent positions.
60 Chapter 6
development as well as security and stability. While
some have noted that reconstructed roads contribute positively to economic and social conditions in
Afghanistan, there is currently little evidence based
on sound impact assessments that these projects
have resulted in expected benefits. . . . 7
Figure 6.1 V
oter Turnout by Election in Afghanistan, 20042010
308,896
171,470
71,783
61,043
34,283
23,646
85,835
13,611
KEY TAKEAWAYS
An effective Red Team approach can include a range
of techniques and is an essential part of any process
aimed at uncovering hidden weaknesses in a course
of action. In this case, the approach helps to identify a
misalignment of strategic, operational, and tactical goals.
Even without an abundance of time or specialized
knowledge, analysts can use these structured analytic
techniques to identify the right questions to ask and to
outline an approach that can mitigate weaknesses before
they have deleterious effects on mission outcome.
NOTES
1. Laura M. Walker, Task Force Pacemaker Constructing
a Road to Democracy, Army Engineer, SeptemberOctober
2005, 20.
2. Captain Claudia Crossland, US Army, interview with the
authors, Virginia, July 67, 2010.
3. Elizabeth Wannstedt, Meeting of the Blades, Army
Engineer, SeptemberOctober 2005, 3031.
4. David Galula, Counterinsurgency Warfare: Theory and
Practice, Westport, CT: Praeger Security International, 1964, 4.
5. Crossland, interview.
6. US Government Accountability Office, Afghanistan
Reconstruction Progress Made in Constructing Roads, but
Assessments for Determining Impact and a Sustainable Maintenance
Program Are Needed (GAO-08689), July 8, 2008, 5. Available at
http://www.gao.gov/products/GAO-08689.
7. Ibid., 38.
8. Ibid., 3.
9. Ibid., 47.
10. Ibid., 26.
11. Ibid.
Analytic Family
p. 56
Simple Hypotheses
p. 171
p. 173
p. 181
graphically; and identify possible gaps, anomalies, and correlations. In addition, these techniques pull the analyst out
of the evidentiary weeds to view a data set from a more strategic vantage point. Chronologies and Timelines can be
paired with mapping software to create geospatial products
that display multiple layers of information such as time,
location, terrain, weather, and other travel conditions.
The details of this case make an annotated Timeline and
Map particularly useful in identifying key pieces of
evidence, confidence levels in the reporting, and gaps in the
information.
Task 1.
Create a Timeline of Lunas last hours.
Step 1: Identify the relevant information from the case
narrative with the date and order in which it occurred.
Consider how best to array the data along the Timeline.
Can any of the information be categorized?
There are many ways to present the data in this case in a
timeline. A full timeline of the case will reflect a period
from Lunas youth in New York through his death and into
the present day. It will include all references in the case to
Lunas activities prior to his death and new information
uncovered in the investigation. This new information
should be reflected on the timeline at the time it allegedly
occurred. A more sophisticated timeline would also include
a separate line for when the information was reported.
Doing so not only helps an analyst see events as they
unfolded but also understand when information became
available. This allows analysts to look for any anomalies in
the pattern of the reporting that might be associated with a
deception hypothesis.
63
64 Chapter 7
The timeline in Figure 7.1 is excerpted from a longer
timeline of the case and illustrates how relevant information
can be displayed along a two-sided timeline in order to
reflect evidence and analysis, including assumptions and
gaps. It also shows how color coding can be used to reflect
categories of activities. In this timeline, the evidence is
broken into three categories: Lunas known movements, the
cars movements, and his bank card activities.
Task 2.
0900
1800
1730: Ravenelle
sees Luna at the
courthouse after
negotiating the
plea agreement.
1700
Morning:
Courthouse,
Baltimore, MD.
Fined $25
late fee
by judge.
Work Day
2000
1900
17302100 gap: It is
unclear when exactly
Luna left for home or if
he went directly
home.
Analysis
Evidence
2200
2300
Sometime after
2300: Luna receives
cell phone call at
home and leaves
for office.
2100
0200
Sometime
before 0330:
Luna possibly
at Elverson
Roy Rogers.
0320: King
of Prussia,
PA. Credit
card used
to buy gas
for two
cars.
0300
0237: Car
enters
Turnpike at
Interchange
6A from
NJ Route
130.
0100
0057: Delaware,
I-95 Exit 3, Travel
Plaza. Lunas
debit card used
for $200 ATM
withdrawal.
0046: E-Z
Pass at
Delaware
line toll.
Perryville,
MD, toll.
0028: E-Z
Pass at
0000
0500
Penknife
discovered
near where
body
found.
February
0404: Car
exits PA
Turnpike at
Reading/
Lancaster
with paper
ticket.
0400
04000530 gap:
Luna alive when car
entered parking lot
and pulled up to
creek, according to
coroner. Lunas
activities during this
hour and a half
unknown.
66 Chapter 7
Lebanon
Hershey
270
Rockville
495
Kin
Ki
Kin
inngg of
o Prussiaa
King
95
Philadelphia
Phi
Ph
hiilad
hil
ladelphi
delph
elphiaa
West Chester
Burlington
Burlingto
Burli
Bur
gton
ton
onn
206
Ca enn
Camden
202
70
Ch ster
Chester
543
833
Newark
r
NEW
NEWJERSEY
JERSEY
Glassboroo
95
Pennsville
l
40
Hammonton
Ham
Hammo
mm ntoon
55
40
30
MARYLAND
Bel Air
95
97
Washington, DC
50
Bridgeton
13
Millville
49
50
Smyrna
20
347
34
Chestertown
Cheste
Che
C
he rtown
w
300
DELAWARE
Delaware
Bay
Dover
Severna Park
50
Bowie
47
Vinelandd
Middletown
Midd
dletown
Edgewood
dgewood
Glen Burnie
49
213
3
Aberdeen
A
Abe
berdeen
29
322
27
97
76
Wilmington
Wilm
mington
i
n
28
Norristown
wn
Norristown
PENNSYLVANIA
Che
sap
eak
Coatesvillee
TTrenton
rento
e o
en
Warminster
30
74
Reisterstown Owings
win
At 0404 Lunas car exits
Mills
Mi
ills
Timonium
turnpike. 27
Mount
Airy
Towson
T
on
Randallstown
Lunas body found at 0530
70 Road in
Baltimo
B
imo
im
m
off Dry Tavern
Lancaster
County, PA.
Damascus
40
Phoenixville
Ph
322
Lansdale
202
100
222
476
176
Columbia
York
nv
Denver
82
Lancasterr
94
Ephrata
222
15
76
Middletown
M
dlet
l town
Pottstown
47
Centreville
Annapolis
13
0
113
10
10
20 miles
20 kilometers
Thursday
4 December
Time
Location
Activity
2338
391713.21N
76372.43W
2349
Baltimore, MD
391539.12N
763438.87W
0028
Perryville, MD
393515.68N
76 424.15W
0046
393842.39N
754552.56W
0057
Lunas debit card was used for a $200 ATM withdrawal from
Exxon at Travel Plaza.
393945.30N
754125.71W
0237
40 65.78N
744721.25W
0247
40 718.18N
745046.90W
0320
King of Prussia, PA
Lunas debit card was used at a Sunoco Station to buy gas and
possibly for another ATM withdrawal.
40 522.03N
752215.61W
0330
PA Turnpike, Elverson, PA
40 858.46N
754959.85W
0404
401258.97N
76 429.27W
After
0530
Denver, PA
401237.45N
76 330.58W
Geo-coordinates
His wife had him killed because she found out he was cheating.
68 Chapter 7
hypothesis), a lover, a hit man, Lunas colleagues, etc.
Alternatively, grouping by Why (debt, work-related issues,
jealousy/envy, and random violence), for example, can help
considerably with achieving mutual exclusivity and can help
consolidate the Who list later.
Step 4: Use problem restatement and consideration of the
opposite to develop new ideas.
Problem Restatement: Why did Jonathan Luna take such
a circuitous and late-night trip toward Philadelphia?
Opposite: Luna was not suicidal; he was a victim of
someone elses rage. This could include a random act of
violence or a murder by a lover, colleague, criminal he had
previously prosecuted, or creditor.
This process illuminates the possibility of a random act
of violence. Luna had allegedly traveled to Philadelphia
numerous times. His circuitous route that night took him
first directly toward Philadelphia. Only after the anomalous
two-hour period from the 0057 ATM withdrawal to 0247
did his car take a turn westward. Could he have been
headed to Philadelphia and fallen victim to a random act
of violence on his trip? Lunas key witness in the case he
had been prosecuting that day, who had reversed himself
on the stand, had been in custody in Philadelphia. Could
Luna have been returning to Philadelphia for work-related
purposes?
Step 5: Update the list of alternative hypotheses.
Problem restatement augments the list of hypotheses by
including the possibility of a random act of violence.
Step 6: Clarify each hypothesis by asking, Who? What?
How? When? Where? and Why?
Make a list of each of the categories above. Step back and
consider how each list could be augmented. The Who list
includes colleagues, stranger, lover, creditors, criminal he
had prosecuted in the past. Refine this list to make the
categories more mutually exclusive. This helps clarify the
hypotheses. For example, creditors, criminals, and
colleagues could all have employed a hit man.
Step 7: Select the most promising hypotheses for further
exploration.
Luna was murdered by those he was negotiating a plea
bargain for, his creditors, or his lover; Luna committed
suicide; Luna was killed in a random act of violence.
Who?
Why?
Lead Hypothesis
Suicide (Luna)
Debt
Brainstormed
Alternatives
Adversary/Hit Man
Lover
Random Attacker
Work-Related
Problem
Jealousy/Envy
Accident
Table 7.5 Luna Multiple Hypotheses GeneratorTM: Example of Permutations and Credibility Scoring
Who?
Suicide
Adversary/
Hit Man
Lover
Random
Attacker
Why?
Permutations
Credibility Score
Debt
Work-related
Jealousy/envy
Accident
Debt
Work-related
Jealousy/envy
Accident
Debt
Work-related
Jealousy/envy
Accident
Debt
Work-related
Jealousy/envy
Accident
70 Chapter 7
Table 7.6 Luna Multiple Hypotheses GeneratorTM: Example of Sorted and Scored Hypotheses
Permutation
Credibility
Table 7.7 Luna Multiple Hypotheses GeneratorTM: Example of Hypotheses for Further Exploration
Hypotheses for Further Exploration
Reasoning
The main motivation for such an accidental suicide has been reported as being an effort
to garner sympathy and/or stave off taking a polygraph in connection with an ongoing
investigation.
His profession makes him a possible target of many individuals. Whether the death was a
hit or an attack by a known acquaintance, the work-related adversary hypothesis should be
explored further.
Luna had credit card debt. Were there any other debts that could have prompted an adversary to
intentionally or unintentionally take his life?
The so called personal nature of the attack, including wounds to the genitals, could point to a
lovers involvement.
Given stops along the roundabout route and gaps in information concerning the route itself
after the 0057 withdrawal, must consider a random attacker.
72 Chapter 7
Figure 7.2 Jonathan Luna Case: Basic List of Evidence for ACH
FBI says Luna alone all night.
G
as station attendee says saw Luna late at night about
once a month over six-month period.
C
urrently negotiated plea agreement that resulted
in lesser charges for defendants.
Died of drowning.
Signs of restraint.
Injuries to genitals.
Figure 7.3 Luna PARC ACH and Te@mACH Coding Differences in Matrix View
74 Chapter 7
attack by an adversary. However, separate reporting cites
inside sources saying that DiBiagio had lied about Lunas
work status to protect Lunas family. If, however, DiBiagios
public and alleged private comments are removed from the
matrix, the suicide hypotheses remain the most
inconsistent with the data. As a result, this piece of
evidence is not as crucial as initially thought, because
while DiBiagios comments are highly applicable to the
suicide hypotheses, they are not applicable to the other,
more likely hypotheses.
Another piece of highly diagnostic evidence is the FBIs
statement that Luna was alone all night. For the purposes of
the ACH matrix, this evidence can be treated as an
assumption. If it is assumed that this is true, it becomes a
critical piece of evidence because it is highly inconsistent with
all of the hypotheses except suicide. As a result, it is important
to track down the underlying evidence that would support
this assumption. The FBI did not make this evidence public,
so analysts should consider what indicators would raise or
lower their confidence in the veracity of this assumption.
Continue this process until all diagnostic evidence is
reviewed.
Step 8: Report the conclusions by considering the relative
likelihood of all the hypotheses.
The sensitivity analysis reveals areas for further scrutiny,
but in the absence of additional information, the tentative
conclusions about the relative likelihood of the hypotheses
hold. However, any written analysis should include a full
accounting of conflicting information, gaps, and
assumptions upon which the analysis is based and what new
information might change the likelihood of the hypotheses.
Step 9: Identify indicators or milestones for future
observation.
The ACH process suggests that analysts should pay
careful attention to new information that either
corroborates or discredits Coronor Walps assessment, the
FBIs assertion that Luna was alone, or information about
blood from a second person in the car. These pieces of
information would differentiate further between the suicide
and other hypotheses. Information about possible workrelated problems, adversaries, recent contacts, extramarital
activities, and previous threats could serve as important
evidence that would discriminate between the lover and
work-related hypotheses. These pieces of information could
significantly affect the likelihood of the hypotheses and
NOTE
1. Eric Rich and Allan Lengel, US Prosecutors Death Still
Puzzling, Washington Post, December 3, 2004, http://www.washing
tonpost.com/wp-dyn/articles/A297452004Dec2.html.
Analytic Family
p. 56
Mind Maps
p. 86
p. 181
77
78 Chapter 8
uncovered in the investigations. This new information should
be reflected on the Timeline at the time it allegedly occurred.
In some cases, it might be preferable to include a separate
citation for when the information was reported. Doing so not
only helps an analyst see events as they unfolded but also to
understand when information became available. This allows
analysts to look for any anomalies in the pattern of the
reporting that might support a deception hypothesis.
The Timeline in Figure 8.2, excerpted from a longer
Timeline of the case, illustrates how relevant information
can be displayed along several parallel tracks illustrating
four dimensions of the event: Bhuttos activities, the
governments actions and statements, the actions of the
attackers and the Taliban, and the role of the media.
Step 2: Review the Timeline by asking the following
questions:
Are there data gaps?
The key issue that emerges from the Timeline is the
apparent dispute over what actually caused Bhuttos death.
The Timeline helps analysts sort through this issue by
allowing them to compare known facts with the various
statements of government officials and others cited by the
media. Most of the initial reporting stated that she died
of gunshot wounds. In subsequent days, the government
declared that the actual cause of death was a head trauma
caused by a major explosion that went off near Bhuttos SUV.
Many have argued that the government was too quick to
clean up the crime site and that a more methodical search
might have revealed additional critical items of evidence.
Some controversy also erupted over whether one or more
assassins were involved in the plot. The only reference to
a second bomber was the speculation prompted by the
release of a grainy video that showed a man with a white
scarf standing just behind the purported gunman. No other
reference to this man appears in the case, and the Scotland
Yard investigators contended that only one gunman was
involved, who detonated his explosive vest after firing
several shots. In contrast, the intercepted communication
indicates that the purported perpetrators, the Pakistani
Taliban, had intended to engage up to five assassins in the
plot. Lastly, some would question the husbands decision
not to demand an autopsy, expecting that a proper autopsy
could have revealed more information.
Do the duration and sequence of events suggested by
the data make sense?
Party
departs for
Rawalpindi.
1400
Holds rally
in Liaquat
Bagh.
SUV
departs
rally.
1530-1710 1712
Government
has 1,000
police officers
on duty with Police
escort
snipers on
provided.
roofs.
Bhutto
meets
with
US Reps.
27
Dec Morn- Early
2007ing Afternoon
Government
battles
with rooftop
sniper who
kills 4 Sharif
supporters.
Head
raised
above
sunroof.
1714
Man raises
gun to
shoot,
suspected
bomber
in background.
ATTACKER
GOVERNMENT
BHUTTO
Head
lowered
just before
explosion.
1715
Gunman
looks
down.
1716
Bomb
explodes
near SUV,
probably
triggered
by gunman.
TALIBAN
GOVERNMENT
BHUTTO
MEDIA
Pronounced
dead; cause
of death
to be
determined
on autopsy.
TV reports
that Bhutto
was shot in
the head; AFP
reports she
was killed by
a suicide
bomber.
UPI reports
husband told
GEO TV Bhutto
was shot in the
neck; cites reports
that a gunman
fired at Bhutto
and then set off
explosive vest.
1816
Bhutto is
received
at Rawalpindi
General
Hospital.
1735
TIME reports
that doctors
who attended
Bhutto
immediately
after attack
say she died
of gunshot
wounds.
(approx.)
1845
Senior police
officer orders
wash of crime
scene with
fire hoses.
AFP cites
unnamed
police source
who
confirms
Bhutto was
shot before
bomb
exploded.
Friends and
witnesses
who take
Bhutto to
hospital
claim she
was shot.
27 Dec
Late
Evening
Musharraf
blames
terrorists for
Bhuttos
death,
appeals for
national
unity.
Newpaper Dawn
speculates Bhutto
was killed by sniper
fire; Daily Times
cites several bullet
wounds to head
and neck;
eyewitnesses
heard three gunshots.
Bhutto is buried
in her familys
mausoleum in
the afternoon.
28 Dec
1700
BG (ret.) Cheema
announces at Min.
of Interior press
conference that
Bhutto died of a
head injury
caused by
explosion; links
attack to Mehsud.
AM
Bhuttos remains
are transferred to
her husband and
flown to Larkana
for burial.
28 Dec
Taliban leader
Mehsud
congratulates killers
in purported
intercepted
communication.
Dawn releases
grainy photos
of alleged
attacker(s).
29 Dec
Punjab Joint
Investigation
Team reports no
blood or tissue
on the hatch
where Bhutto
allegedly
struck her head.
TIME reports
doctors who
attended
Bhutto
released new
findings over
the weekend
that the cause
of death was
head trauma.
2 Jan
Musharraf
asks Scotland
Yard to
investigate
Bhuttos death.
80 Chapter 8
What additional information should you seek? Key
topics to pursue would include information on any plotting prior to the incident, any indications of government or
ISID collusion with Baitullah Mehsud or other individuals
who might target Bhutto, and any concrete evidence that the
police were ordered to clean up the site prematurely.
How confident are you in the sources of information?
The timeline suggests that careful scrutiny should be given
to press reporting and eyewitness reports. In addition, the
motives of all reporting sources should be evaluated with an
eye toward determining if there was intent to deceive investigators or the public.
TECHNIQUE 2: MIND MAPS
Mind Maps are visual representations of how an individual
or a group thinks about a topic of interest. A Mind Map
diagram has two basic elements: the ideas that are judged
relevant to whatever topic one is thinking about and the
lines that show and briefly describe the connections
between these ideas. Whenever you try to put a series of
thoughts together, that series of thoughts can be represented
visually with words or images connected by lines that rep
resent the nature of the relationships between them. Any
thinking for any purp ose, whether about a personal
decision or analysis of an intelligence issue, can be
diagrammed in this manner. In fact, Mind Mapping was
originally developed as a fast and efficient way for students
to take notes during briefings and lectures.
In cases such as this, where initially there is little solid
evidence and much speculation, it is particularly
important to cast the net wide to make sure that nothing is
excluded. This is especially so because the Pakistani
government immediately leaped to a conclusion, blaming
the so-called Pakistani Taliban operating in Pakistans
tribal belt. Although the hypothesis offered by the
Pakistani government appears credible, the more
important question is whether it is the only hypothesis
worth considering.
Task 2.
Generate a Mind Map to explore who could have been
behind Benazir Bhuttos assassination.
Step 1: Identify the focal question or the logical starting
point for an investigation. Write the focal question down in
the center of the page and draw a circle around it.
(Husband)
Asif Ali
Zardari
Hindi
Nationalists
Family Members
Qari Saifullah
Akhtar
Islamic
Militants
Pakistani
Taliban
India*
(Niece)
Fatima
Bhutto
Lone
Wolf
al-Qaeda
Aitezaz
Shah
Baitullah
Mehsud
United States*
Nation-States
(President)
Pervez
Musharraf
Rogue
Elements
(former ISID)
Ijaz Shah
(Intelligence
Bureau Chief)
China*
Senior Officials
(Minister of
Religious Affairs)
Ejaj ul-Haq
Imran Khan
(former cricketer/
politician)
Political Rivals
Pakistani
Government
Intel Services
Gen.
Hamid Gul
(former Prime
Minister)
Nawaz Sharif
(former Chief
Minister of
Punjab)
Chaudhry
Pervez Elahi*
(former Chief
Minister of
Sindh)
Arbab
Ghulam Rahim*
(Pakistani Muslim
League)
Chaudhry
Hussein
82 Chapter 8
several entities and/or individuals associated with it. For
example, two of Bhuttos relatives (her niece and husband)
are connected to the Family Members category. The
Pakistani government category is more complex, with one
individual (President Musharraf ) linked to it as well as two
entitiesIntelligence Services and Senior Officials. Each of
these entities has several names associated with it, which
can be extracted from the case study.
Step 7: While building the Mind Map, consider the
possibility of cross-links from one issue to another. Show
directionality with arrows pointing in one or both
directions.
Several connections may be worth noting on the Mind
Map, especially the link between President Musharraf and
the Pakistani Taliban headed by Mehsud. The link between
Pakistani Intelligence Chief Hamid Gul and the Taliban is
also worth noting. These connections suggest that Mehsud
could have acted either alone or with the support of the
Pakistani government. Mehsuds links to al-Qaeda should
be depicted as well, suggesting that this link could provide
another reason for suspecting Mehsud. Lastly, Aitezaz
Shahs reported links to the Pakistani Taliban require noting
and possible further discussion.
Step 8: While building the Mind Map, consider the
possibility of conflicting evidence or conflicting concepts. If
they appear, label them differently by color, written name,
or shape, or by putting an asterisk or other icon inside the
circle or box.
In this case, it would be useful to color code linkages or
hypotheses that could have been surfaced based on weak
data or information that may have been provided with
intent to deceive. Benazir Bhuttos message accusing four
current and former Pakistani officials of having motive to
kill her is not substantiated by any other information in
the case. Similarly, a case can be made for nation-states
such as India, China, or the United States being possible
suspects given histories of past tensions, but such
allegations are not substantiated by any information
presented in the case study. It is a good idea to include
such potential suspects in the Mind Map in order to
generate a comprehensive list of suspects, but it is also
helpful to indicate with color coding or an icon that the
evidence supporting these suspects is weak.
Step 9: Reposition, refine, and expand the Mind Map
structure as appropriate.
Possible Motive
Possible Motive
al-Qaeda
China
United States
India
Possible Motive
India
China
al-Qaeda
United States
84 Chapter 8
Analytic Value Added: Does the creation of the Mind
Map prompt you to consider a much broader array of
potential explanations or hypotheses? The act of drawing
the Mind Map prompts analysts to think about a larger
range of alternatives at the outset of a project. For example,
once the analyst decides to list Fatima Bhutto as a potential
mastermind, the question that immediately comes to mind
is whether other family members, such as the husband,
should be added to the Mind Map. The Mind Map approach
also makes it easier to array a large number of alternatives
in a simple display that is easy to embellish and refine.
Does it help you drill down for each hypothesis to
consider second- and third-level questions? In this exercise, the Mind Map approach prompts the analyst to consider possible linkages between the groups and individuals
depicted and to come up with the names of specific people
who could have been the mastermind behind the operation.
In considering the Islamic Militants category, for example,
creating the Mind Map prompts one to explore several questions such as these:
Which key Pakistani militant groups, such as the
Harkat-ul-Jihad-al-Islami (HUJI), deserve attention,
apart from the Pakistani Taliban?
How are these various actors linked?
Would they combine forces in an attempt to
assassinate Bhutto?
Did they have the capability to launch the attack that
killed Bhutto?
Does it help you identify potential gaps in knowledge? The Mind Map approach not only reveals key gaps in
knowledge but helps open the door to considering the possibility that several entities might simultaneously have been
attempting to kill Bhutto and that more than one plot may
have been playing out at the time of her death.
TECHNIQUE 3: ANALYSIS OF
COMPETING HYPOTHESES
Analysts face a perennial challenge of working with
incomplete, ambiguous, anomalous, and sometimes
deceptive data. In addition, strict time constraints and the
need to make a call often conspire with a number of
natural human cognitive tendencies to result in inaccurate
or incomplete judgments. Analysis of Competing
Hypotheses (ACH) improves the analysts chances of
86 Chapter 8
The current set of five hypotheses are sufficiently distinct
from each other to argue against combining any into a
single hypothesis. Given the strength of the Taliban
hypothesis, thought should be given to exploring whether
other hypotheses from the Islamic Militants category should
be considered, such as a lone wolf, HUJI, or an al-Qaeda
operative.
Step 6: Draw tentative conclusions about the relative
likelihood of each hypothesis. An inconsistency score will
be calculated by the software; the hypothesis with the
lowest inconsistency score is tentatively the most likely
hypothesis. The one with the most inconsistencies is the
least likely.
The two hypotheses with the highest inconsistency
scores are Rogue ISID elements and Musharraf and his
government. Some of the most compelling arguments for
discarding these hypotheses are the fact that a suicide
bomber was employed, the government had provided
heavy security, Bhutto had stopped short of attacking
Musharraf directly, and up to this point most of the
suicide bombings had been targeted at the ISID and the
military. The primary reason for dismissing Political
Rival Sharif and Bhuttos Niece Fatima is the finding
that Bhutto was killed by a suicide bombing, not bullets
from a gun. Neither Sharif nor Fatima are likely candidates
to have used a suicide bomber.
Step 7: Analyze the sensitivity of your tentative conclusion
to a change in the interpretation of a few critical items of
evidence by using the software to sort the evidence by
diagnosticity.
The analysis would change dramatically if it were
determined that the intercepted communication or the
teenagers confession was not authentic or if new evidence
emerged that one of the other suspects was involved in a
plot to assassinate Bhutto that day. Also of concern would
be a finding that the Scotland Yard report included the
caveat that restrictions placed on its investigation by the
Pakistani government may have precluded it from
conducting a thorough inquiry.
Step 8: Report the conclusions by considering the relative
likelihood of all the hypotheses.
The ACH software automatically moves the hypothesis
or hypotheses that are the most credible to the left side of
the matrix. The least likely hypothesis will appear on the
far right. The most credible hypotheses are those with the
surrounding the killing and its aftermath. The threemember commission conducted more than 250 interviews
in Pakistan with government officials and private citizens
who had knowledge of the assassination. The commissions
investigative team also examined the Scotland Yard report
and reviewed hundreds of documents, photographs, and
other documentary material provided by Pakistani and
British officials. Following are some of the key findings of
the report, published on 30 March 2010:
Ms. Bhuttos assassination could have been prevented if
adequate security measures had been taken....The
federal government under General Musharraf...[was]
not proactive in neutralizing [threats] and/or ensuring
that the security provided was commensurate to those
threats.1
She died when a 15 and a half year-old suicide bomber
detonated his explosives near her vehicle, [but] no one
believes that this boy acted alone.2
Ms. Naheed Khan recalled that immediately after she
had heard the three gunshots, Ms. Bhutto fell down into
the vehicle onto her lap. Ms. Khan said that she felt the
impact of the explosion immediately thereafter....Ms.
Khan saw that Ms. Bhutto was not moving and saw that
blood was also trickling from the ear.3
Five persons were arrested by [Pakistani officials]:
Aitezaz Shah, Sher Zehman, Husnain Gul, Mohamad
Rafaqat, and Rasheed Ahmed. In addition, [Pakistani
officials] charged Nasrullah, Abdullah, Baitullah
Me h s u d , a n d Mau l v i S a h i b a s pro c l a i m e d
offenders....The accused are alleged to have served as
handlers and logistics supporters of the suicide bomber,
or as persons who were knowledgeable about the plans
to assassinate Ms. Bhutto.4
The investigation into Ms. Bhuttos assassination, and
those who died with her, lacked direction, was
ineffective, and suffered from a lack of commitment to
identify and bring all of the perpetrators to justice.5
The [Joint Investigation Team]...did nothing to build a
case against Mr. Mehsud, treating the contents of the
intercept presented to the public by Brigadier Cheema as
determinative of his culpability. AIG Majeed told the
Commission that he saw no need to establish the
authenticity of the intercept or the basis for its analysis,
including the voice identification and the interpretation
of the conversation as a reference to Ms. Bhuttos
assassination.6
88 Chapter 8
The UN report shed light on several key aspects of the
investigation. It noted that no blood or tissue was found
on the trucks escape hatch lever, drawing into question
whether Bhutto had hit her head on the lever when she fell
into the cab.7 The report also dismissed reports that doctors
had deliberately altered their initial findings that Bhutto had
suffered gunshot injuries. More significant, the commission
said it had not found any credible, new information showing
that Bhutto had received bullet wounds.8
The report noted that numerous people may have wished
Bhutto harm, including local jihadi groups, the Pakistan
Taliban, al-Qaeda, and members of the Pakistani
government and political elite.9 After the Karachi attack,
Bhuttos attorney said that he had received a handwritten
letter from someone claiming to be the head of suicide
bombers and a friend of al-Qaeda who threatened to
assassinate Bhutto in a gruesome manner. An al-Qaeda
spokesperson, Mustafa Abu al Yazid, had also claimed
responsibility for her assassination in an interview with the
Asia Times Online.10
According to the UN report, many senior Pakistani
officials believed Baitullah Mehsud was part of a larger
conspiracy to assassinate Bhutto, but the report observes
that many of these same officials would have had a motive
to eliminate Bhutto because they were threatened by the
possibility of her regaining power.11,12 The true story of
Mehsuds involvement may never be known because he was
killed in a drone attack in August 2009.13
The commission took the police to task for focusing the
investigation on lower-level operatives and not exploring
whether any higher-level officials may have been involved
in the planning, financing, or execution of the
assassination.14 It attributed police reluctance in part to a
concern that Pakistani intelligence services may have had a
role in the assassination.15
KEY TAKEAWAYS
The tendency to plunge in should always be tempered
by a process designed to identify all the relevant
information and evaluate all possible explanations.
Chronologies and Timelines are invariably some of
the best ways to begin an analysis; they not only help
the analyst organize the data but can reveal key gaps,
inconsistencies, and correlations in the data.
Employing a more systematic process, such as a
Mind Map, at the start of the investigation helps
frame the issue. It also helps analysts identify a more
comprehensive set of hypotheses early on.
Consider a full range of hypotheses against all the
relevant information and return to this analysis
over time. There could be several, intertwined
explanations, or the hypotheses could change
over time as more information comes to light. Be
prepared to evaluate each piece of new information
against all the possibilities.
NOTES
1. United Nations, Report of the United Nations Commission
of Inquiry into the Facts and Circumstances of the Assassination of
Former Pakistani Prime Minister Mohtarma Benazir Bhutto, March
30, 2010, http://www.un.org/News/dh/infocus/Pakistan/UN_
Bhutto_Report_15April2010.pdf, 2.
2. Ibid., 2.
3. Ibid., 28.
4. Ibid., 41.
5. Ibid., 2.
6. Ibid., 41.
7.
8.
9.
10.
11.
12.
13.
14.
15.
Ibid., 40.
Ibid., 3233.
Ibid., 3.
Ibid., 48.
Ibid., 50.
Ibid., 51.
Ibid., 41.
Ibid., 3.
Ibid.
Analytic Family
Structured Brainstorming
p. 102
Idea Generation
Starbursting
p. 113
Idea Generation
p. 209
p. 173
p. 181
89
90 Chapter 9
92 Chapter 9
Step 12: Present the results, describing the key themes or
dimensions of the problem that deserve investigation.
The group should end up with a set of three to five
hypotheses that best explain why the young Navajo couple
died suddenly. At this stage of the exercise, the hypotheses
can be fairly general so as not to rule out a viable alternative.
Some sample hypotheses include these:
The couple came in contact with a highly toxic
chemical or biological substance.
The two young Navajos were the victims of a
deliberate hate crime targeting the Navajo Nation.
The two young Navajos were collateral damage in a
terrorist plot that for the first time involved the use of
biological weapons.
The couple succumbed to a particularly virulent,
naturally occurring pathogen.
The two young people had other health problems
that made them more susceptible to the common
flu.
Analytic Value Added: Did we explore all the
possible forces and factors that could explain why the
young Navajo couple died? Did our ideas group
themselves into coherent affinity groups? Structured
Brainstorming is a powerful tool for generating a diverse
number of ideas; it taps the expertise and past experiences
of everyone in the group and gives them equal opportunity
to provide their input. The requirement to place all the
ideas into affinity groups forces the group to critically
examine the underlying forces and factors that might have
caused the deaths while avoiding the cognitive trap of
satisficing, wherein one generates a short list of ready
answers to the question without any underlying rigor to
the process.
The silent, structured brainstorming approach is a powerful technique to pull out new and often never previously
considered ideas and concepts. It avoids the trap of deferring to the most knowledgeable person in the room by
giving all participants an equal, but silent, opportunity to
surface their ideas.
Did our ideas group themselves into coherent affinity groups? How did we treat outliersthat is, the sticky
notes that seemed to belong in a group all by themselves?
Did the outliers spark new lines of inquiry? Did the
labels we generated for each group accurately capture
the essence of that set of sticky notes? While conducting
TECHNIQUE 2: STARBURSTING
Starbursting is a form of structured brainstorming that
helps analysts generate as many questions as possible. It is
particularly useful in developing a research project, but it
can also help to elicit many questions and ideas to challenge
conventional wisdom. This process allows the analyst to
consider the issue at hand from many different perspectives,
thereby increasing the chances that the analyst will uncover
a heretofore unconsidered question or idea that will yield
new analytic insights.
Task 2.
Construct a Starbursting diagram to explore the Who?
What? How? When? Where? and Why? questions relating
to the untimely death of a healthy young Navajo couple.
Step 1: Use the template in Figure 9.1 in the book or draw
a six-pointed star and write one of the following words at
each point of the star: Who? What? How? When? Where?
and Why?
Step 2: Start the brainstorming session, using one of the
words at a time to generate questions about the topic. Do
not try to answer the questions during the brainstorming
session; just focus on generating as many questions as
possible.
Students should be able to develop at least two to four
questions per point in the star, as reflected in example
Figure 9.2.
Step 3: After generating questions that start with each of
the six words, the group should either prioritize the
questions to be answered or sort the questions into logical
categories.
WHO?
WH
?
AT
Y?
WH
HO
W?
WHEN?
RE
HE
W
What was the cause of death?
W
What toxins have they been exposed to?
W
What chemical toxins could cause these symptoms?
W
What natural pathogens could cause these symptoms?
W
What has changed in the environment?
H
How did they become ill?
D
Did they inhale harmful fumes?
D
Did they experiment with illegal substances?
When did they become ill; how quickly did they die?
When did others show the same symptoms; when did they die?
Does time of year matter?
94 Chapter 9
Are there known toxic waste sites that all the victims
might have visited?
Are the symptoms consistent with any other viruses or
diseases that are more lethal than the common flu?
TECHNIQUE 3: KEY ASSUMPTIONS CHECK
The Key Assumptions Check is a systematic effort to make
explicit and question the assumptions that guide an analysts
interpretation of evidence and reas oning about any
particular problem. Such assumptions are usually necessary
and unavoidable as a means of filling gaps in the incomplete,
ambiguous, and sometimes deceptive information with
which the analyst must work. They are driven by the ana
lysts education, training, and experience, including the
organizational context in which the analyst works. It can be
difficult to identify assumptions, because many are
sociocultural beliefs that are held unconsciously or so firmly
that they are assumed to be truth and not subject to
challenge. Nonetheless, identifying key assumptions and
assessing the overall impact should conditions change are
critical parts of a robust analytic process.
Task 3.
Conduct a Key Assumptions Check of the initial theory that
the young Navajo couple died from a particularly virulent
common flu virus.
Step 1: Gather a small group of individuals who are
working the issue along with a few outsiders. The primary
analytic unit already is working from an established mental
model, so the outsiders are needed to bring other
perspectives.
In this instance, the Navajo tribal healers and experts
from CDC in essence played the role of outsiders. The
historical perspective provided by the tribal healers turned
out to be critical to solving the case.
Commentary
Supported
With Caveat
Unsupported
Commentary
Supported
9. D
ead Navajos were victims of a hate
crime.
With Caveats
Unsupported
96 Chapter 9
major resource decisions given the fact that caregivers are
not coming down with the illness. The assumption that
Navajos are deliberate targets is mere speculation
unjustified by any known data.
Step 8: Consider whether key uncertainties should be
converted into collection requirements or research topics.
The Key Assumptions Check should inspire the analysts
to focus their attention on the Unsupported assumptions
that have emerged as Key Uncertainties. Analysts could
focus their assessment on those questions that are most
likely to move the investigation forward. These might
include the following:
Are people who do not belong to the Navajo Nation
dying as well?
What are the indications that the illness is
contagious?
Are the symptoms consistent with any other viruses
or diseases that are far more virulent than the
common flu?
Are there any reports of tourists contracting the
disease or spreading it to other parts of the country
when they return home?
Are any Internet sites or blogs posting information
critical of the Navajo Nation?
What similarities can we detect among those who
have become ill?
Are there known toxic waste sites that all the victims
might have visited?
Can any link be established between Fort Wingate
and those who have fallen ill or died of this disease?
Can a link be established between a mushrooming
rodent population and Navajos suddenly becoming
ill? What would the tribal healers and history tell us
about a potential link?
Analytic Value Added: When CDC investigators
arrived on the scene and interviewed doctors, did they
inherit any key assumptions that would have had an
impact on how effectively they organized their
investigation? CDC investigators were careful to review all
the information provided by the on-site caregivers and to
initiate new research to establish patterns and look for
similarities. More important, they reached outside their
normal circles to seek input from Navajo tribal healers in
hopes of gaining additional perspectives on the case. This
Who?
What?
How?
Act of
Nature
Brainstormed
Alternative
Components
Unknown Disease
(Natural Pathogen)
Intentional
Act of Man
Chemical Toxin
Accidental
Exposure
Anyone
98 Chapter 9
Table 9.7 Multiple Hypotheses GeneratorTM: Death in the Southwest Permutation Tree
Who?
What?
Only Navajos
Unknown Disease
(Natural Pathogen)
Chemical Toxin
Anyone
Unknown Disease
(Natural Pathogen)
Chemical Toxin
Why?
Permutations
Credibility Score
Act of Nature
Only Navajos are dying from a virulent form of the common flu.
discard
discard
Accidental Exposure
discard
Act of Nature
Accidental Exposure
discard
Act of Nature
Accidental Exposure
Act of Nature
discard
Accidental Exposure
discard
Act of Nature
Accidental Exposure
Act of Nature
Accidental Exposure
discard
4
Table 9.8 Multiple Hypotheses GeneratorTM: Death in the Southwest Hypotheses Re-sorted by Credibility
Permutations
Credibility Score
People are dying from accidental exposure to a new, unknown natural pathogen.
Only Navajos are dying from a virulent form of the common flu.
discard
discard
Only Navajos are dying from accidental exposure to a virulent form of the common flu.
discard
Only Navajos are dying from accidental exposure to a new, unknown natural pathogen.
discard
discard
discard
People are dying from accidental exposure to a virulent form of the common flu.
discard
Credibility Score
100 Chapter 9
Analytic Value Added: Which hypotheses should be
explored further? Additional medical tests should be
conducted to help determine if a new virus might be the
cause of the problem. Researchers also need to investigate
how t he vic t ims acquire d t he p at hogen. What
commonalities exist in terms of where the victims worked,
where they played, what locations they all might have
frequented, or what work practices they might all share? If
domestic radical extremists or terrorists were to blame,
then research is needed to investigate why they would be
targeting the Four Corners region or, more specifically,
members of the Navajo Nation. For example, are there any
recent postings on the Internet by such groups that would
suggest that an attack on members of the Navajo Nation
was justified? The chances that Fort Wingate is the source
of the problem would be greatly increased if most of those
who became ill worked at the fort or had relatives or
acquaintances who worked there. Almost certainly, there
would be press reports and a major buzz in the local
community if Fort Wingate were the actual source of the
problem.
Which of the six key components (Who? What? How?
When? Where? and Why?) can be set aside because they
are givens, and why? The case study is challenging because
many of the answers to these questions overlap. For example, the answer to Where? would indicate a natural cause if
the Where turned out to be pastureland or farmland and,
alternatively, an act of man if a specific location was identified that all the victims have frequented in recent weeks. The
Why component poses similar challenges; at a minimum it
focuses attention on what specific groups would have motive
to launch an attack aimed at the Navajo Nation or the Four
Corners region.
Which hypotheses from the original list were discarded, and why? Most of the hypotheses that were discarded were dropped because the internal logic of the
permutation did not stand up to scrutiny. For example, a
terrorist is not likely to use the common flu to cause a largescale panic, nor would the use of the common flu be likely
to generate large numbers of casualties.
TECHNIQUE 5: ANALYSIS OF
COMPETING HYPOTHESES
Analysts face a perennial challenge of working with
incomplete, ambiguous, anomalous, and sometimes
deceptive data. In addition, strict time constraints on analysis
and the need to make a call often conspire with a number
of natural human cognitive tendencies to result in inaccurate
102 Chapter 9
tentative conclusions about the relative likelihood of each
hypothesis would include the following observations:
The Common Flu hypothesis is likely to have the
most Inconsistents and is the easiest to dismiss.
The Hate Crime hypothesis also has several
Inconsistents and is not likely to be correct.
The remaining two hypotheses have the fewest
Inconsistents and appear worthy of serious
consideration and further investigation.
It is just as important to critically examine the Inconsistent
items of relevant information for the most likely hypotheses
as well. If many Inconsistents are associated with all the
most likely hypotheses, this could signal that there is a
missing hypothesis. However, if the inconsistent evidence
can be described at best as a squishy Inconsistent, then the
hypothesis probably is the most likely explanation.
104 Chapter 9
How long had the virus been present in the area? Tribal
elders knew the presence of rodents in tribal homes put
people at risk because it potentially exposed them to rodent
feces and urine.4 To avoid sickness, the elders recommended
burning affected clothing and isolating food supplies. Tests
on tissue samples collected and preserved by Sevilleta
Wildlife Refuge ecologists showed that the now-termed Sin
Nombre or Without a Name virus had been present in
the rodent population for at least ten years before the 1993
epidemic. Based on the Navajo tribal healers oral histories,
epidemiologists suspected that rodent-transmitted disease
had been present in the Four Corners Region since the early
part of the twentieth century.5
In 1993, when precipitation plummetedactually
returned to normaland available vegetative food
sources were depleted, the increased rodent population
began searching for food in new environments, such as
barns and peoples homes. The virus, which does not
cause illness in the rodent host, was transferred from
rodents to humans via saliva, urine, or fecal matter.
Human infection occurs when the materials are inhaled
as aerosols or introduced onto broken skin, similar to an
anthrax infection. The disease was concentrated in the
Navajo population simply because environmental
conditions in the local area and agricultural cultivation
increased contact between man and infected rodents.
Visitors who had hiked or camped in the Navajo Nation
area also became victims because of their exposure to the
deer mouse.6,7
Research on the outbreak later determined that 50
percent of the infections were acquired in or around the
home, 10 percent at the workplace, 5 percent during
recreation, and the remainder for mixed or unknown
reasons. A frequent antecedent of contracting the virus was
opening and inhabiting a long unused cabin. This may be
related to several factors: entry disturbs deer mice, which
often urinate as they flee; the closed cabin lacks ventilation;
and the roof prevents inactivation of the virus by the
ultraviolet component of sunlight.8
Hantaviruses often bring death quickly. Usually 30 to 40
percent of patients die within twenty-four to forty-eight
hours after admission to a hospital, even in well-run
intensive care units (ICUs). The best indicator that a
hantavirus is present is a finding of decreasing or
abnormally low platelet counts. Approximately 40 percent
of patients do not require the placement of a plastic tube
into the trachea to protect the patients airway and provide a
NOTES
1. David Perlin and Ann Cohen, Hantavirus: Four Corners,
United States, 2002, http://www.infoplease.com/cig/dangerousdiseases-epidemics/hantavirus-four-corners-united-states.html.
2. Tom Paulson, Doctor on Trail of Another Deadly Virus,
Seattle Post-Intelligencer, April 9, 2003, http://www.seattlepi.com/
default/article/Doctor-on-trail-of-another-deadly-virus-1111862
.php.
Analytic Family
p. 209
Pros-Cons-Faults-and-Fixes
p. 330
Decision Support
p. 173
olice investigators were under severe pressure to discover who placed the bomb in Centennial Park and to
bring that person or persons to justice. One person had
been killed by the bomb and over a hundred were injured,
and the public was justifiably concerned about safety at the
Olympic Games. In such circumstances, the investigating
team is under extreme pressure to come to closure quickly
and to identify a prime suspect. Such dynamics make analysts and investigators vulnerable to groupthink and more
likely to adopt satisficing strategies that will please all key
stakeholders.
The best way to cope with such pressure is to employ
structured techniques that allow investigators and analysts
supporting them to take a few moments to reflect on what
they know and what they need to know before plunging in to
resolve the case. In this case study, we explore how three
structured analytic techniquesthe Key Assumptions Check,
Pros-Cons-Faults-and-Fixes, and the Multiple Hypotheses
GeneratorTMcan be employed to better frame the problem
and avoid going down unnecessarily time-consuming investigative blind alleys. Each technique takes relatively little time
to employusually only an hour or twobut can save investigators much time over the long run by avoiding nonproductive leads. The techniques also can make the investigation
more efficient by focusing attention on key information gaps
and what types of additional information could prove the
most compelling in helping to solve the case.
TECHNIQUE 1: KEY ASSUMPTIONS CHECK
The Key Assumptions Check is a systematic effort to make
explicit and question the assumptions that guide an analysts
interpretation of evidence and reasoning about any particular problem. Such assumptions are usually necessary and
unavoidable as a means of filling gaps in the incomplete,
ambiguous, and sometimes deceptive information with
which the analyst must work. They are driven by the analysts education, training, and experience, including the
organizational context in which the analyst works. It can be
difficult to identify assumptions because many are sociocultural beliefs that are held unconsciously or so firmly that
they are assumed to be true and not subject to challenge.
Nonetheless, identifying key assumptions and assessing the
overall impact should conditions change are critical parts of
a robust analytic process.
Task 1.
Assume you are a member of the FBI team investigating the
bombing. Piedmont College President Cleere has called the
FBI office in Atlanta to present his rationale for making
Richard Jewell a prime suspect in the case. Following consultations with Washington, D.C., your team has decided to
do just that. To help kick off the investigation, you have
been asked to conduct a Key Assumptions Check with your
teammates to go over what assumptions the team is making
about Jewell and the bombing in Centennial Park. Your task
is to guide the team through the following eight steps for
conducting a Key Assumptions Check.
Step 1: Gather a small group of individuals who are
working the issue along with a few outsiders. The primary
analytic unit already is working from an established mental
model, so the outsiders are needed to bring other
perspectives.
107
108 Chapter 10
In this case, the FBI team of investigators would benefit
from including some local or state law enforcement officials
in the brainstorming process.
Step 2: Ideally, participants should be asked to bring their
lists of assumptions when they come to the meeting. If not,
start the meeting with a silent brainstorming session. Ask each
participant to write down several assumptions on a 3 5 card.
Supported
2. M
any more people would have died or been injured if Richard Jewell had not
alerted authorities to the knapsack.
With Caveats
6. Jewell would have known how to place the bomb without being seen.
9. J ewell intended the bomb to explode in fewer than 30 minutes because his
intent was to clear the area of people and ambush police and security officers.
10. R
ay Cleeres statements were truthful and not motivated by his holding a
grudge against Jewell.
Unsupported
14. J ewells personality fit the profile of someone who would create an incident so
he could emerge a hero.
15. J ewells personality fit the profile because he sought out publicity after the
bombing.
16. J ewell might be the bomber because he appeared uncomfortable talking about
the victims out of guilt.
17. J ewells statement that he wanted to get a position on the Atlanta police
department was inappropriate and could indicate he had a motive for planting
the bomb.
110 Chapter 10
a large group of law enforcement officers would converge
on the site fairly quickly? How would Jewell have acquired
this information? Would this suggest that Jewell might have
been surveilling the site for several days? If so, would such
activity show up on the security video cameras? If so,
wouldnt Jewell be concerned that the cameras would catch
him planting the bomb? Would Jewell have known about
the security cameras?
Analytic Value Added: What assumptions, if any, did
law enforcement analysts and officials make as they began
the investigation? Law enforcement officials fairly quickly
focused on a single, lead hypothesis that Jewell had planted
the bomb with the intent of revealing it to the authorities
and taking credit for minimizing the number of casualties.
They assumed motive and capability and, as new information surfaced, decided how it could be made to fit the lead
hypothesis. Information inconsistent with this lead hypothesis, such as the impossibility of both making the 911 call
and alerting authorities in Centennial Park to its presence
one minute later, was ignored.
Were they influenced by key assumptions of others,
including the press and the experts they interviewed, who
wanted to assist their work? FBI investigators initially
responded to the call from Piedmont College President
Cleere, appropriately treating this hypothesis as worthy of
further investigation, but nothing in the public record
shows that they challenged the assumption that Cleere was
truthful and not carrying a grudge against Jewell.
As colleagues generated other examples of the wannabe
hero syndrome, however, they fell into the trap of satisficing, whereby a proposed explanation or theory of the case
quickly gains acceptance because it fits with most of the key
facts and the explanation satisfies the needs of ones supervisors and the public.
Did the investigators fall into the trap of groupthink,
or did they have sufficient cause to focus on Jewell as a
suspect? The investigators quickly fell into the trap of
groupthink, allowing a tip from President Cleere and a few
anecdotesof people having taken credit for incidents to
make themselves appear as heroesto dominate their
thinking. In reviewing Jewells past history in law enforcement, they were quick to confuse correlation with causality.
Moreover, the case study notes that Jewell was charged with
impersonating a police officer but does not reveal if he was
actually convicted. Although Jewell had a history of employment problems, there was nothing in his case history to
suggest that he would go to the extreme of constructing an
Cons
1. He could not have made 911 call and alerted police to the presence of
the knapsack.
2. He would not have treated other police officers as his prime target.
112 Chapter 10
Pros
Cons
Fixes
He had an accomplice.
Table 10.8 Atlanta Olympics Bombing Multiple Hypotheses GeneratorTM: Brainstormed Alternatives Example
Lead Hypothesis: Richard Jewell planted the bomb to make himself a hero and help obtain a job.
Components
Lead Hypothesis
Who?
Richard Jewell
What?
Antipersonnel bomb
When?
27 July 1996
Why?
To get a job
Where?
How?
Alternatives
International terrorists
Disgruntled contractors
To inflict harm
Centennial Park
Prepositioned explosive
Task 3.
Step 6: Evaluate the credibility of the remaining hypotheses on a scale of 1 to 5, where 1 is low credibility and 5 is
high credibility.
The three hypotheses rated 0 in Table 10.9 can be discarded
because they make little sense. For example, it makes no sense
that terrorists would plant bombs to protest being laid off.
Step 7: Re-sort the remaining hypotheses, listing them
from most to least credible, as shown in Table 10.10.
Step 8: Restate the permutations as hypotheses.
The permutations above are stated as hypotheses.
Step 9: Select from the top of the list those alternative
hypotheses most deserving of attention and note why these
hypotheses are most interesting (see Table 10.11).
The four most plausible hypotheses with a credibility
score of 3 or higher are these:
Richard Jewell planted the bomb to make himself a
hero and obtain a job.
International terrorists planted the bomb to inflict
harm on America.
114 Chapter 10
Table 10.9 Atlanta Olympics Bombing Multiple Hypotheses GeneratorTM: Permutations and Credibility Scoring
Example
Who?
International
Terrorists
Domestic Violent
Extremists
Disgruntled Workers
Why?
Permutations
Credibility Score
To inflict harm
To inflict harm
To inflict harm
Credibility
Table 10.11 Atlanta Olympics Bombing Multiple Hypotheses GeneratorTM: Hypotheses for Further Exploration
Example
Hypotheses for Further Exploration
Reasoning
Richard Jewell planted the bomb to make himself a hero and obtain a job. Jewells past employment history makes him a candidate for a
wannabe attack.
International terrorists planted the bomb to inflict harm.
Security guards who had recently been laid off were angry about
losing their jobs.
116 Chapter 10
THE HUNT FOR ERIC RUDOLPH
Over a two-year period after the bombing, special agents on
the Southeast Bomb Task Force interviewed thousands of
witnesses and traced nearly every component of the bomb.
The task force was comprised of the FBI; Bureau of Alcohol,
Tobacco, and Firearms (ATF); Georgia Bureau of
Investigation; Alabama Bureau of Investigation;
Birmingham Police Department; and prosecutors from the
Justice Department. In addition, many local and state law
enforcement units supported the task force.6
On 14 October 1998, federal authorities charged Eric
Rudolph with conducting the fatal bombing at Atlantas Centennial Park on 27 July 1996. Rudolph became a serious target
of investigation in part because a Tennessee couple identified
him as the man to whom they sold the smokeless powder
believed to have been used in the Atlanta bomb device.7
Federal authorities also charged Rudolph with a double
bombing at a health clinic in the Sandy Springs Professional
Building in North Atlanta on 16 January 1977 and with the
bombing of a gay night club, the Otherside Lounge, in Atlanta
on 21 February 1997.8 In the Sandy Springs bombing, the first
bomb caused significant damage at the back of the building.
The second bomb was designed to kill and maim rescuers,
paramedics, firefighters, and police officers who rushed to the
scene to help, according to the Director of the ATF.9 A second
bomb was also found at the scene of the Otherside Lounge
bombing, but the area was cleared before it exploded.
In addition, Rudolph was charged with the bombing at the
New Woman All Woman Health Care Clinic in Birmingham,
Alabama, on 29 January 1998, which killed Birmingham
police officer Robert Sanderson and severely injured the clinics head nurse, Emily Lyons. In announcing the charges
against Rudolph, the government said it would pay a reward
of $500,000 for information leading to a conviction of
Rudolph and a reward of up to $1,000,000 for information
leading to Rudolphs arrest.10
Rudolph became one of Americas top ten most wanted
fugitives from justice. 11 A sizeable law enforcement
contingent, supported by infrared-equipped helicopters and
tracking dogs, was dispatched to comb the 517,000-acre
Nantahala Forest in western North Carolina to look for any
sign of Rudolph.12,13
After more than five years on the run, Rudolph was
captured in May 2003 when police spotted him near a trash
bin in Murphy, North Carolina, apparently scavenging for
food.14 He was brought to trial in July 2004 and charged
with the bombings of the health clinic and the Otherside
KEY TAKEAWAYS
When under severe pressure to find a culprit
or generate an analytic conclusion quickly, an
alarm should go off telling you that these are the
circumstances where the use of structured analytic
techniques is most justified.
The use of techniques like the Key Assumptions
Check or Pros-Cons-Faults-and-Fixes only take a few
hours but can save investigators days, if not weeks,
of energy they would otherwise waste tracking down
low-priority leads or working from assumptions that
upon close inspection prove invalid.
Considering multiple credible hypotheses (or
suspects) at the start of an investigation often proves
much more efficient and less time-consuming
overall than conducting the investigation in a serial
fashion by first going after a prime suspect, and
then a second suspect if the first does not pan out,
and then a third suspect, etc. Considering multiple
suspects also helps focus attention on the most
diagnostic evidence.
INSTRUCTORS READING LIST
Federal Bureau of Investigation, Counterterrorism Division,
Counterterrorism Threat Assessment and Warning Unit,
National Security Division. Terrorism in the United States:
1996. http://www.fbi.gov/stats-services/publications/
terror_96.pdf.
Ostrow, Ron. Richard Jewell and the Olympic Bombing:
Case Study. Pew Research Centers Project for Excellence
in Journalism. February 15, 2003. http://www.journalism
.org/node/1791.
NOTES
1. BBC, 1996: Bomb Rocks Atlanta Olympics, http://news
.bb c.co.uk/onthis day/hi/dates/stories/july/27/ne wsid_
3920000/3920865.stm.
2. Iver Peterson, Head of FBI Says It Cant Trace Disclosure
in Olympic Bombing Case, New York Times, December 20, 1996,
http://www.nytimes.com/1996/12/20/us/head-of-fbi-says-itcanttrace-disclosure-in-olympic-bomb-case.html.
3. Harry R. Weber, Former Olympic Park Guard Jewell
Dies, Washington Post, August 30, 2007, http://www.washington
post.com/wp-dyn/content/article/2007/08/30/AR2007083000324
.html.
4. David Kohn, Falsely Accused, 60 Minutes II, CBS
Worldwide, June 26, 2002, http://w w w.cbsnews.com/
stories/2002/01/02/60II/main322892.shtml.
5. Kevin Sack, Richard Jewell, 44, Hero of Atlanta Attack
Dies, New York Times, August 30, 2007, http://www.nytimes
.com/2007/08/30/us/30jewell.html?n=Top/Reference/Times%20
Topics/Subjects/O/Olympic%20Games.
6. Department of Justice, Eric Rudolph Charged in
Centennial Olympic Park Bombing [press release], October 14,
1998, http://www.fas.org/irp/news/1998/10/477crm.htm.
7. BBC, 1996: Bomb Rocks Atlanta Olympics.
8. Department of Justice, Eric Rudolph Charged in
Centennial Olympic Park Bombing.
9. Ibid.
10. Ibid.
11. Key Dates in Hunt for Eric Rudolph, Fox News, June 2,
2003, http://www.foxnews.com/story/0,2933,88269,00.html.
12. Search for Rudolph Continues 5 Years After Bombing,
CNN, July 23, 2001, http://articles.cnn.com/20010723/justice/
rudolph.search_1_emily-lyons-eric-robert-rudolph-double
bombing.
13. Paul Nowell, Search for Bombing Suspect Resumes,
Washington Post, July 12, 1999, http://www.washingtonpost.com/
wp-srv/national/longterm/rudolph/rudolph.htm.
14. Associated Press, Raw Data: Timeline in Eric Rudolph
Case, Fox News, June 2, 2003, http://www.foxnews.com/
story/0,2933,88269,00.html.
15. BBC, 1996: Bomb Rocks Atlanta Olympics.
16. Mike Lopresti, A Decade Later, Atlanta Olympic Bombing
Overshadowed, USA Today, July 23, 2006, http://www.usatoday
.com/sports/columnist/lopresti/20060723-lopresti-atl-10years_x.htm.
17. BBC, 1996: Bomb Rocks Atlanta Olympics.
18. Henry Schuster, Why Did Rudolph Do It? CNN, April
15, 2005, http://www.cnn.com/2005/US/04/11/schuster.column/
index.html.
19. Associated Press, Eric Rudolph Gets Life Without Parole,
Fox Ne w s , Ju ly 1 8 , 2 0 0 5 , http : / / w w w. fox ne w s . c om /
story/0,2933,162790,00.html.
Analytic Family
p. 209
p. 173
p. 122
Idea Generation
11 The DC Sniper
Cases in Intelligence Analysis: Structured Analytic Techniques in Action
Instructor Materials
119
120 Chapter 11
In the early days of the investigation, the lead hypothesis
had four key components:
LoneOnly one shooter was involved in the
multiple shootings.
WhiteSerial killers are almost always Caucasian.
MaleSerial killers are almost always male.
Military experienceThe shooter must have had
military experience in order to shoot so well and may
have even been a sharpshooter.
Step 4: Elicit additional assumptions. Work from the prevailing analytic line back to the key arguments that support
it. Use various devices to help prod participants thinking.
Ask the standard journalist questions: Who? What? How?
When? Where? and Why? Phrases such as will always,
will never, or would have to be suggest that an idea is
not being challenged and perhaps should be. Phrases such
as based on or generally the case usually suggest that a
challengeable assumption is being made.
For the purposes of this case study, it works best to
focus the conversation on the lone, white male theory. At
the time, other explanations were considered, including the
possibility that the shooter was a foreign terrorist; a domestic extremist, and possibly a white supremacist because
Commentary
Supported
With Caveats
1. Lone
2. White
3. Male
4. Military experience
Unsupported
122 Chapter 11
credibility score is meant to illuminate new, credible hypotheses for further examination. And although the process
encourages analysts to focus on the hypotheses with the
highest credibility scores, hypotheses with low credibility
scores should not be entirely discarded because new evidence
could emerge that could make a hypothesis more credible.
Task 2.
Use the Multiple Hypotheses GeneratorTM (see Table 11.3)
to create and assess alternative hypotheses. Contact
Globalytica, LLC at [email protected] or go to
http://www.globalytica.com to obtain access to the software
if it is not available on your system.
Step 1: Identify the lead hypothesis and its component
parts.
In this example, the Who, Why, and What have been
explored. The lead hypothesis could best be articulated as
follows: A white male is driving a white van and killing to
extort money. The key components are white male, white
van, and killing to extort money. Since it is a fact that
shootings are happening and that the ballistic tests have
resulted in the identification of the type of weapon used,
these aspects can be considered to be static and need not be
included in the permutations.
Steps 2 & 3: Identify plausible alternatives for each key
component and strive to keep them mutually exclusive.
Discard any given factors such as the How (shooting) that
will be the same for all hypotheses. Table 11.7 shows the
results of a brainstorming session on alternatives.
The students are likely to suggest additional alternatives,
but the two alternatives listed above have generally proven
most effective in illustrating the technique. For example,
other alternatives to White Male could be Hispanic or
Lead Hypothesis
Alternative/Brainstormed
Who?
White Male
Black Male
White Female
What?
White Van
Sedan
On Foot
Why?
To Extort Money
Seek Fame
Cause Terror
What?
White Van
White Male
Sedan
On Foot
White Van
On Foot
White Van
Black Male
Sedan
On Foot
Why?
Permutations
Credibility Score
Extort Money
Terrorize
Seek Fame
Extort Money
Terrorize
Seek Fame
Extort Money
Terrorize
Seek Fame
Extort Money
Terrorize
Seek Fame
Money
Terrorize
Seek Fame
Money
Terrorize
Seek Fame
Extort Money
Terrorize
Seek Fame
Extort Money
Terrorize
Seek Fame
Extort Money
Terrorize
Seek Fame
124 Chapter 11
Credibility
Score
Permutations
Credibility Score
1
1
white males. The exact motive is less important than knowing the Who and What, but examining the potential reasons
may assist investigators in how they approach the investigation and potential future communication with the sniper.
Using the Multiple Hypotheses GeneratorTM allowed each
aspect of the alternative hypotheses to be evaluated in a
robust manner that explicitly detailed the facts and assumptions underlying each credibility score. These conversations
are often enlightening and may not happen if the technique
is not used.
TECHNIQUE 3: CLASSIC QUADRANT
CRUNCHINGTM
Classic Quadrant CrunchingTM combines the methodology
of a Key Assumptions Check with Multiple Scenarios
Generation to generate an array of alternative scenarios or
stories. This process is particularly helpful in the DC Sniper
case because of embedded assumptions in the FBI profile,
witness reports of white vans, and the contents of the
demand note. This technique allows the user to look at and
challenge those key assumptions. When combined with the
Multiple Hypotheses GeneratorTM, this technique provides
a strong basis for developing and considering alternative
explanations and scenarios.
Task 3.
Use Classic Quadrant Crunching to challenge the key
assumptions in the case that is listed below.
TM
Contrary Assumption
Contrary Dimensions
A. Lone Attacker
Multiple Attackers
Team
Copycat Killers
B. White
Other Race
Black
Hispanic
C. White Van
Sedan
On Foot
D. To Extort Money
Other Motivation
Seek Fame
Cause Terror
126 Chapter 11
Multiple Attackers/Race
Team
Team
Black
Hispanic
Copycat Killers
Copycat Killers
Black
Hispanic
A/C
1
2
Multiple Attackers/Transport
Team
Team
Sedan
On Foot
Copycat Killers
Copycat Killers
Sedan
On Foot
A/D
1
2
Multiple Attackers/Motivation
Team
Team
Seek Fame
Cause Terror
Copycat Killers
Copycat Killers
Cause Terror
Seek Fame
B/C
1
2
Race/Transport
Black
Black
Sedan
On Foot
Hispanic
Hispanic
Sedan
On Foot
B/D
1
2
Race/Motivation
Black
Black
Seek Fame
Cause Terror
Hispanic
Hispanic
Cause Terror
Seek Fame
C/D
1
2
Transport/Motivation
Sedan
Sedan
Seek Fame
Cause Terror
On Foot
On Foot
Seek Fame
Cause Terror
Quadrant A/B-1 is a team of black snipers that is conducting attacks in multiple locations across the metropolitan
Washington, D.C., area. The snipers formed a team sometime over the past year and set their well-practiced plan in
motion after several months of planning and training. The
circumstances surrounding the formulation of their group
and the exact number of members in the cell are unknown.
As a result, if this team is quite small, they could be conducting the attacks one at a time. If the team is larger and
dispersed, they could be conducting coordinated attacks at
preappointed times.
128 Chapter 11
three children. There...Mr. Muhammad planned to create a training ground for 140 young homeless men whom
he would send out to wreak similar havoc and to shut
things down in cities across the United States.15
KEY TAKEAWAYS
Decision making based on faulty assumptions can
impede an investigation. Always explicitly identify
and assess the effect implicit assumptions may have
on an investigation.
The tendency to plunge in should always be
tempered by a process designed to identify all
evidence and evaluate all possible explanations.
Failure to consider alternative explanations from the
start can slow an investigation and let the real killer
avoid prosecution.
Employing a more systematic process at the start
of the investigation to better frame the issue helps
analysts identify unproductive blind alleys early on
and avoid them.
INSTRUCTORS READING LIST
Horwitz, Sari, and Michael E. Ruane. Sniper: Inside the Hunt
for the Killers Who Terrorized the Nation. New York:
Random House, 2003.
NOTES
1. James Alan Fox and Jack Levin, An Anatomy of Serial
Murder, chap. 3 in Extreme Killing: Understanding Serial and Mass
Murder (London: Sage, 2005), 38. Available at http://www.sagepub
.com/upm-data/5396_Fox_Final_Pages_Chapter_3.pdf.
2. Ibid.
3. Ibid.
4. A Byte Out of History: The Beltway Snipers, Part 1, FBI
Online, October 22, 2007, http://www.fbi.gov/news/stories/2007/
october/snipers_102207.
5. Closing the Net: How They Cracked the Case, CNN,
October 25, 2002, http://edition.cnn.com/2002/US/South/10/24/
sniper.case.cracked/index.html.
6. A Byte Out of History: The Beltway Snipers, Part 1, FBI
Online.
7. Bushmaster .223: Accurate, Inexpensive, CNN, October
24, 2002, http://articles.cnn.com/20021024/us/sniper.bushmaster
.rifle_1bushmaster-semi-automatic-rifle-weapon.
8. Closing the Net: How They Cracked the Case, CNN.
9. Sari Horwitz and Michael E. Ruane, Sniper: Inside the Hunt
for the Killers Who Terrorized the Nation (New York: Random
House, 2003), 170, 188.
10. Ibid., 16365.
11. A Byte Out of History: The Beltway Snipers, Part 1, FBI
Online.
12. Horwitz and Ruane, Sniper: Inside the Hunt for the Killers
Who Terrorized the Nation, 234.
13. Ibid., 235.
Analytic Family
p. 144
Indicators
p. 149
Indicators Validator
p. 157
in the internal affairs of Colombia, and its leaders have concluded that the time has come. In this fictitious scenario,
members of the Secretariat and top military commanders
gather in the Amazon jungle to formulate a strategy for a
retaliatory strike in the United States.
The challenge for US analysts is to forecast how an attack
is most likely to be launched and, in so doing, help federal,
state, local, and tribal officials prevent or mitigate the damage of such an attack. When confronted with this challenge,
the first reaction of many students is to propose that the US
government issue a general alert to all state, local, and tribal
officials that a FARC attack on the homeland may be imminent, and ask them to look out for any suspicious activity
that would indicate a FARC attack is being planned or
implemented. Unfortunately, such guidance is so unspecific
as to lack value for law enforcement officials. The purpose
of this exercise is to show that with the use of structured
analytic techniques, analysts can generate a plausible set of
attention-deserving scenarios and create tailored lists of collection requirements that provide operational value to
headquarters, FBI field offices, and fusion centers.
Task 1.
The major victory of the Colombian army and its US military supporters in Colombia against the FARC has created a
new situation wherein the FARC sees itself substantially
weakened, increasingly desperate, and determined to demonstrate that it is not a spent force. The FARC had threatened to
retaliate against the United States in the past for interfering
129
130 Chapter 12
included, often the groups perspective is limited to the
stream of reporting it reads every day; as a result, key
assumptions may remain unchallenged, and historical analogies may be ignored.
Step 2: Pass out sticky notes and marker-type pens to all
participants. Inform the team that there is no talking during
the sticky-notes portion of the brainstorming exercise.
Use different color sticky notes and encourage the participants to write down short phrases consisting of three to
five words, not long sentences.
Step 3: Present the team with the following question: If
you were in the FARC Secretariat, what are all the things
you personally would think about when planning an attack
on the US homeland? The reason for first asking group
members how they would react is to establish a baseline for
assessing whether the adversary is likely to react differently.
Keep the question as general as possible so as not to inadvertently restrict the creative brainstorming process. It also
helps to ask the group if they understand the question and
whether they believe it should be worded differently. Spending a few minutes to ensure that everyone understands what
the question means is always a good investment.
Ask them to put themselves in the FARCs shoes and
simulate how its leaders would respond. Emphasize the
need to avoid mirror imaging. The question is not What
would you do if you were in their shoes? but How would
the FARC leadership approach this problem, given their
background, past experience, and the current situation? It
is important to emphasize the importance of avoiding mirror imaging. In a classroom situation, many students may
not know much about the FARC; this is why it is important
to ensure that all participants read the case study with the
relevant background material carefully. They should also
have the case study at hand for quick reference.
Step 4: Ask the group to write down responses to the
question using a few key words that will fit on a sticky note.
After a response is written down, the participant gives it to
the facilitator, who then reads it out loud. Marker-type pens
are used so that people can easily see what is written on the
sticky notes when they are posted on a wall or whiteboard.
Give the students a few minutes to think about the issue
and jot down a few ideas. Then go around the room and
collect the sticky notes. Read the responses slowly and stick
them on the wall or the whiteboard as you read them. Some
sample sticky notes might address topics such as financing,
type of weapon, target, deniability, need for contacts in the
United States, escape plan, motive, logistic support, infiltration, partners, and access to technology.
Step 5: Post all the sticky notes on a wall in the order in
which they are called out. Treat all ideas the same.
Encourage participants to build on one anothers ideas.
Usually there is an initial spurt of ideas followed by pauses
as participants contemplate the question. After five or ten
minutes there is often a long pause of a minute or so. This
slowing down suggests that the group has emptied the barrel of the obvious and is now on the verge of coming up
with some fresh insights and ideas. Do not talk during this
pause, even if the silence is uncomfortable.
Remind the group not to talk during this part of the
exercise. It is important for them to hear what others are
suggesting, as this might stimulate new ideas for them to jot
down. Also take care not to spend too much time talking
yourself. The participants need quiet time to think, and it is
very important for the instructor not to interrupt their
thought processes. Often when it is the quietest, the best
thinking is taking place.
Step 6: After two or three long pauses, conclude this
divergent thinking phase of the brainstorming session.
Step 7: Ask all participants (or a small group) to go up to
the wall and rearrange the sticky notes by affinity groups
(groups that have some common characteristics). Some
sticky notes may be moved several times; some may also be
copied if the idea applies to more than one affinity group.
If only a subset of the group goes to the wall to rearrange
the sticky notes, then ask those who are remaining in their
seats to form into small groups and come up with a list of key
drivers or dimensions of the problem based on the themes
they heard emerge when the instructor was reading out the
sticky notes. This keeps everyone busy and provides a useful
check on what is generated by those working at the wall.
Step 8: When all sticky notes have been arranged, ask the
group to select a word or phrase that best describes each
grouping.
Four or five themes usually emerge from this part of the
exercise.
A variety of potential targets, including US military
installations and particularly USSOUTHCOM
in Miami; FBI and DEA facilities, mostly in
Washington, D.C., and along the US southern
border; and senior US officials, who could be targets
of assassinations or kidnappings.
any of the groups. Consider whether such an outlier is useless noise or the germ of an idea that deserves further
attention.
Often one or two outlier sticky notes are worth pointing out to the class because they provide a fresh perspective
or suggest a potentially valuable new line of inquiry. Here
are some examples:
A note that says heroin could open the door to
a discussion of whether the FARC would consider
operations to corrupt heroin currently being supplied
in the United States to force drug addicts to switch to
cocaine as a safer drug of choice.
A note that says attack the US embassy in Bogot
might be initially rejected as outside the scope of
the original question, but the instructor should
note that by raising the question of an attack on
the US embassy, the participant has, in effect,
challenged a key assumption of the exercise (that
the attack would take place on US soil), and
perhaps in the real world this might prompt the
group to conduct a key assumptions check and
subject this particular assumption to more careful
scrutiny.
Step 12: Assess what the group has accomplished. Can
you identify four or five key factors, forces, themes, or
dimensions that are most likely to influence how the FARC
leadership would mount an attack?
Work with the group to develop a consensus on four
themes that emerge as the most important drivers for this
topic. Write the candidate drivers on the board and draw a
line under each driver. The line represents the spectrum for
that driver. Label the end points of the spectrum for each
dimension or driver being considered. For example, if one
driver is sophistication of the weapon, then at the right
end of the line you would write CBRN or WMD and at
the left end of the line you would write small arms or
simple weapons or rifle.
The themes that most often are generated by this stage of
the exercise are as follows:
Sophistication of weapons (simple such as a rifle or
an assassination to highly sophisticated such as a
CBRN-type attack).
Motive (straightforward revenge to terrorizing US
population).
Target (tactical such as a US military base to strategic
such as the Pentagon or senior Washington officials).
132 Chapter 12
Partners (a do it alone operation to partnering with
other terrorist groups such as the IRA or ETA or
obtaining the support of drug distribution networks
in the United States).
Other themes that might emerge but usually do not work
as well when conducting a Multiple Scenarios Generation
exercise include these:
Cost/benefit (minimal or major commitment of
resources and personnel).
Infiltration/exfiltration (whether to infiltrate FARC
operatives or contract out to drug networks or
radical extremists already operating in the United
States).
Willingness to accept risk (Are FARC leaders willing
to consider a spectacular operation that could spur
the United States to launch a major retaliatory
strike in Colombia, or would they opt for a more
modest attack that sends a message but reduces
the prospects of a retaliatory strike against their
forces?).
Timing (Will the attack be a quick response easily
tied to recent events in Colombia or a much better
planned and more sophisticated attack that could
take months or even years to pull off?).
Target security (Will the FARC go after hard or soft
targets?).
Step 13: At this point, the group should ask, Does the
FARC Secretariat share our values or motives or methods of
operation? If not, then how do those differences lead them
to act in ways we might not have anticipated before engaging in this exercise?
Step 14: Present the results, describing the alternatives
that were considered and the rationale for selecting the path
the group believes the FARC Secretariat is most likely to
take. Consider less conventional means of presenting the
results of the analysis, such as the following:
Describing a hypothetical conversation in which the
Secretariat leaders would discuss the issue in the first
person.
Drafting a document (set of instructions, military
orders, or directives) that the FARC Secretariat
would likely generate.
In most cases, the group should end up with a presentation that defines some version of the following four key
drivers and associated spectrums: type of weapon, motive
for the attack, target of the attack, and whether any outside
assistance is sought.
Students should be encouraged to present their key findings by speaking in the first person, as if they were actual
FARC members planning the attack.
Analytic Value Added: The silent structured
brainstorming approach is a powerful technique to pull out
new and often never previously considered ideas and
concepts. It avoids the trap of deferring to the most
knowledgeable person in the room by giving everyone an
equal, but silent, opportunity to surface ideas. While
conducting the structured brainstorming exercise, it is
useful to note whether particularly useful and creative ideas
are generated after long pauses when everyone is thinking;
if this does occur, it is important to alert the entire group to
the phenomenon.
Were we careful to avoid mirror imaging when we
put ourselves in the shoes of the FARC Secretariat?
By putting themselves in the shoes of the FARC, analysts are more likely to focus on attack scenarios the
FARC would be best positioned to implement successfully
and thus be the most likely. By conducting a Red Hat
Analysis, they usually focus not only on how to launch an
attack but the extent to which the plan they choose
could make them vulnerable to retaliation. Often exfiltrating forces is as important as infiltrating them into the
United States.
Did we explore all the possible forces and factors that
could influence how the FARC might launch an attack on
the US homeland? The sticky notes should capture a broad
spectrum of forces and factors, including logistical preparations, financing, preferred target, type of weapon to employ,
ability to maintain operational security, mechanisms for
infiltrating and exfiltrating forces, and whether to seek the
assistance of or partner with other groups.
Did our ideas group themselves into coherent affinity groups? How did we treat outliers or sticky notes
that seemed to belong in a group all by themselves? Did
the outliers spark new lines of inquiry? Placing like ideas
into affinity groups can be a challenging task; asking those
not at the wall to come up with their own categories often
provides a useful sanity check. Always take time to give
outlier ideas their due attention. Invariably a structured
Task 2.
Use Multiple Scenarios Generation to identify the most
plausible attack scenarios the FARC would consider in
launching a retaliatory attack on the US homeland.
Step 1: Clearly define the focal issue and the specific goals
of the futures exercise.
When you have little intelligence on a specific threat but
substantial information on the potential perpetrator, Multiple Scenarios Generation is a useful tool to scope the
problem, think creatively about potential attack scenarios,
and generate actionable intelligence. In this case, the focal
question is What are the most plausible ways the FARC
would mount an attack on the US homeland? The goal of
the exercise is to use the four key drivers selected in the Red
Hat/Structured Brainstorming Exercise first to generate a
multitude of possible attack scenarios and then to select the
scenarios that seem the most plausible, thus deserving the
attention of those responsible for thwarting or mitigating
the consequences of such an attack.
Step 2: Brainstorm to identify the key forces, factors, or
events that are most likely to influence how the issue will
develop over a specified time period. In this case, use the
four or five key drivers, themes, or dimensions that emerged
from Task 1, the Red Hat/Structured Brainstorming exercise.
In Task 1, four key drivers emerged: the type of weapon,
the motive for the attack, the most likely target of an attack,
and whether outside assistance will be sought.
Step 3: For each of these key drivers, define the two ends
of the spectrum.
For the purposes of illustration, the spectrums can be
defined as follows:
A. Weapon (simple weapon such as a rifle to a highly
sophisticated CBRN attack).
B. Motive (retaliation for recent military operation in
Colombia to much broader aim to terrorize the US
population).
C. Target (tactical attack on a US military base to the
strategic targeting of a senior Washington official).
D. Partners (a do it alone operation or partnering with
the IRA).
Step 4: Pair the drivers in a series of 2 2 matrices. If
you have four drivers, they can be combined into six pairs,
134 Chapter 12
generating six different matrices. Five drivers would generate ten different matrices.
In this case study, the pairs used to form the six matrices
would be: AB (weapon/motive), AC (weapon/target), AD
(weapon/partner), BC (motive/target), BD (motive/partner), and CD (target/partner). The class usually is broken
into smaller groups to work each 2 2 matrix. With six
matrices, it usually works best to assign two matrices to
each of three groups. Be careful in assigning the matrices to
give each group the opportunity to think about all of the
drivers. This can be accomplished by assigning the matrices
as follows: Group 1 (AB and CD), Group 2 (AC and BD),
and Group 3 (AD and BC).
Step 5: Develop a story or two for each quadrant of each
2 2 matrix.
For example, Group 2 was asked to come up with
four stories (one story for each quadrant of the matrix)
for AC (weapon/target). Their work might look like
Figure 9.2, in which the x-axis represents a tactical versus a strategic target and the y-axis represents the spectrum of simple to sophisticated weapons. In each matrix,
the students have brainstormed a potential attack scenario. For example, a tactical attack using weapons of
mass destruction could involve a biological attack on the
water supply of a military base that was supporting US
military operations in Colombia. In another quadrant, a
simple attack designed to terrorize the US population
could be the kidnapping of the son or daughter of a chief
of police of a major metropolitan area such as Miami.
The students opted to propose the kidnapping of a child
because it was assumed a child would be a soft target
unlikely to have security protection.
If one group works more quickly than the others, the
instructor can ask the group to start putting together lists of
indicators for their favorite scenarios.
Students should present similar matrices for all six combinations of drivers. Once all the matrices have been presented and discussed, the class should look for themes that
emerge or seem to repeat in several of the matrices. These
may be more deserving of attention if similar ideas were
generated by different groups independently. Students
should also discuss which of the scenarios are most deserving of the attention of US policy makers and law enforcement officials and provide reasons to support their choices.
Step 6: From all the scenarios generated, select three or
four that are the most deserving of attention because they
Figure 12.3 Multiple Scenarios Generation: Sample Matrix of FARC Attack on the US Homeland
Military/
Police Post
OF WEAPON
Biological attack
on military base
water supply
Introduction of contaminated
drugs into domestic
supply chain
US Population
SOPHISTICATION
TARGET SELECTION
Mortar attack
on military base
guard post
Kidnapping of
police chiefs
son or daughter
Rifle/Handgun
Figure 12.4 Multiple Scenarios Generation: Selecting the Most Attention-Deserving Scenarios of a FARC Attack on the US
Homeland
Selecting Scenarios
Weapon/Motive
Scenario A
Story 3
Story 2
Scenario B
Target/Partner
Weapon/Target
Story 5
Story 6
Scenario B
Story 8
Motive Partner
Story 9
Story 10
Story 13
Story 14
Nightmare
Scenario
Story 12
Story 15
Scenario C
136 Chapter 12
Some possible wildcard or nightmare scenarios that
might be generated from this exercise would be these:
A decision by the FARC leadership to pay drug
distributors within the United States to spike illegal
drugs with a highly toxic substance and distribute
them in communities that surround US military
bases that have deployed troops to Colombia.
An attempt by FARC members to assassinate the
administrator or assistant administrator of the Drug
Enforcement Administration.
Analytic Value Added: Did the technique help us
generate a robust set of potential scenarios to consider?
The Multiple Scenarios Generation technique can be a
powerful tool to generate new ideas and attack scenarios
that might never have been considered as part of a
traditional analysis.
Did we discover new scenarios that we probably would
not have imagined if we had not used this particular
technique? The technique forces analysts to reframe the
question in many different ways; often the combinations
prompt totally new ways of defining the threat environment. The approach should give analysts more confidence
that they have captured the entire threat space and some
assurance that they are less likely to be surprised by how
events actually play out.
Did similar themes emerge from different matrices
even though different pairs of drivers were being considered? When similar themes emerge from more than one
matrix, analysts can be more confident that a key dimension
has been captured that may require the attention of the decision makers.
Were the final scenarios selected both plausible and the
most deserving of attention? The exercise helps analysts
avoid the frequent trap of coming to premature closure and
focusing on the one or two plausible scenarios that first come
to mind. In selecting the most attention-deserving scenarios, it
is always helpful to work from a previously agreed upon set of
key criteria.
TECHNIQUE 3: INDICATORS
Indicators are observable or deduced phenomena that can be
periodically reviewed to help track events, distinguish
between competing hypotheses, spot emerging trends, and
warn of unanticipated change. An indicators list is a pre
established set of actions, conditions, facts, or events whose
simultaneous occurrence would argue strongly that a
phenomenon is present or a hypothesis is correct. The
Indicator
Scenario A: FARC poisons cocaine to terrorize US population.
A-1
DEA chemists see increase in reports of cocaine laced with toxic substance in several major cities.
A-2
A-3
A-4
A-5
A-6
The FARC posts statements on the Internet saying it will retaliate against the United States for supporting Colombian military strikes
against FARC guerrillas.
A-7
Urban drug treatment centers receive queries about what substances are most often mixed with cocaine to increase volume and
profits.
A-8
A-9
A-10
Local US law enforcement reports increased bulk purchases of poisonous substances such as arsenic.
Scenario B: FARC uses rompas to launch mortar attack on USSOUTHCOM headquarters in Miami.
B-1
USSOUTHCOM security reports suspicious cars seen loitering on streets in vicinity of headquarters.
B-2
Analysts looking at FARC Internet site report claims that FARC will make the US military pay for its misdeeds.
B-3
Hispanic males are observed taking photos of USSOUTHCOM headquarters from a distance.
B-4
Suspicious purchases of liquid petroleum gas containers are noted in Miami hardware stores.
B-5
US government sources report that Venezuela has provided documents and passports to FARC operatives to facilitate their
international travel.
B-6
Recent FARC guerrilla defectors mention a mock-up building in the Amazon is being used for target practice with rompas.
B-7
USSOUTHCOM employees tell their supervisors that they are being approached by strangers and asked who works where in the complex.
B-8
C-1
There are reports of FARC meetings and communications with the IRA.
C-2
FARC publishes open letter to the US president stating that FARC will not be intimidated by actions of the US military.
C-3
C-4
There are intelligence reports of IRA hit squads being dispatched to North America.
C-5
C-6
Colombians in New York report suspicious persons loitering outside the mission offices.
C-7
FARC Internet site claims that FARC will make the US military pay for its misdeeds.
C-8
Suspected FARC members entering the United States are found in possession of Colombian military uniforms.
C-9
A FARC informant reports that a special squad is being formed for a major operation.
Scenario D: Marijuana laced with poison kills many in the vicinity of US military bases.
D-1
Street informants report a buzz in the Hispanic community that the FARC is planning a special operation in the United States.
D-2
Local drug dealers say they are being surveyed by people up their distribution chain asking for details on their user populations.
D-3
D-4
DEA chemists report an increase in marijuana laced with arsenic and other toxic substances.
D-5
Street informants report that their suppliers are talking about making easy money.
D-6
A new theme emerges on Facebook that marijuana consumption may be more dangerous than most suspect.
D-7
Analysts note postings by FARC on its Internet site stating that the United States will pay dearly for violating Colombian sovereignty.
D-8
Drug users become increasingly anxious that the drugs they might purchase could be contaminated.
138 Chapter 12
D. Pay drug distributors within the United States to
lace marijuana sold mostly to teenagers with a
highly toxic, lethal substance and distribute it to
communities that surround US military bases that
have deployed troops to Colombia.
A brainstorming session generated the indicators shown
in Table 12.5 for each scenario.
Step 2: Review and refine each set of indicators, discarding any that are duplicative within any given scenario and
combining those that are similar.
In this example, C-5 and C-9 are similar and merit combination into a new indicator: FARC informants or defectors report that a special squad is being formed for a major
operation up north. Similarly, C-2 and C-7 should be combined to state: FARC warns the United States publicly that
it will no longer tolerate American interference in Colombias internal affairs, particularly with its military forces.
Step 3: Examine each indicator to determine whether it
meets the following five criteria. Discard those that are
found wanting.
1. Observable and collectible. There must be some
reasonable expectation that, if present, the indicator
will be observed and reported by a reliable source.
If an indicator will be used to monitor change over
time, it must be collectible over time.
2. Valid. An indicator must be clearly relevant to the
endstate the analyst is trying to predict or assess, and
it must be inconsistent with all or at least some of the
alternative explanations or outcomes. It must accurately
measure the concept or phenomenon at issue.
3. Reliable. Data collection must be consistent when
comparable methods are used. Those observing
and collecting data must observe the same things.
Reliability requires precise definition of the indicators.
4. Stable. An indicator must be useful over time to
allow comparisons and to track events. Ideally, the
indicator should be observable early in the evolution
of a development so that analysts and decision
makers have time to react accordingly.
5. Unique. An indicator should measure only one
thing and, in combination with other indicators,
should point only to the phenomenon being studied.
Valuable indicators are those that are not only
consistent with a specified scenario or hypothesis
but are also inconsistent with all other alternative
scenarios.
Indicator
Scenario A: FARC poisons cocaine to terrorize US population.
A-1
DEA chemists see increase in reports of cocaine laced with toxic substance in several major cities.
A-2
A-3
A-4
A-5
A-6
The FARC posts statements on the Internet saying it will retaliate against the United States for supporting Colombian military
strikes against FARC guerrillas.
A-7
Urban drug treatment centers receive queries about what substances are most often mixed with cocaine to increase volume
and profits.
A-8
New communications are identified between FARC leaders and drug distributors in the United States.
A-9
Local US law enforcement reports increased bulk purchases of poisonous substances such as arsenic.
Scenario B: FARC uses rompas to launch mortar attack on USSOUTHCOM headquarters in Miami.
B-1
USSOUTHCOM security reports suspicious cars seen loitering on streets in vicinity of headquarters.
B-2
Analysts looking at FARC Internet site report claims that FARC will make the US military pay for its misdeeds.
B-3
Hispanic males are observed taking photos of USSOUTHCOM headquarters from a distance.
B-4
Known FARC sympathizers are reported purchasing suspicious quantities of liquid petroleum gas canisters.
B-5
US government sources report that Venezuela has provided documents and passports to FARC operatives to facilitate their
international travel.
B-6
Recent FARC guerrilla defectors mention a mock-up building in the Amazon is being used for target practice with rompas.
B-7
USSOUTHCOM employees tell their supervisors that they are being approached by strangers and asked who works where in the
complex.
B-8
C-1
There are reports of FARC meetings and communications with the IRA.
C-2
FARC warns the United States publicly that it will no longer tolerate American interference in Colombias internal affairs,
particularly with its military forces.
C-3
C-4
There are intelligence reports of IRA hit squads being dispatched to North America.
C-5
FARC informants or defectors report that a special squad is being formed for a major operation up north.
C-6
Colombians in New York report suspicious persons loitering outside the mission offices.
C-7
Suspected FARC members entering the United States are found in possession of Colombian military uniforms.
Scenario D: Marijuana laced with poison kills many in the vicinity of US military bases.
D-1
Street informants report a buzz in the Hispanic community that the FARC is planning a special operation in the United States.
D-2
Local drug dealers say they are being surveyed by people up their distribution chain asking for details on their user populations.
D-3
D-4
DEA chemists report an increase in marijuana laced with arsenic and other toxic substances.
D-5
Street informants report that their suppliers are talking about making easy money.
D-6
A new theme emerges on Facebook that marijuana consumption may be more dangerous than most suspect.
D-7
Analysts note postings by FARC on its Internet site stating that the United States will pay dearly for violating Colombian
sovereignty.
D-8
Informants report that drug users are complaining that the drugs they are purchasing are contaminated.
140 Chapter 12
scenario were beginning to emerge or that particular hypothesis were true. A critical question that is not often asked is
whether a given indicator would appear only for the scenario
or hypothesis to which it is assigned or also in one or more
alternative scenarios or hypotheses. Indicators that could
appear under several are not considered diagnostic, suggesting
that they are not particularly useful in determining whether a
specific scenario is beginning to emerge or a particular hypothesis is true. The ideal indicator is highly likely for the scenario
to which it is assigned and highly unlikely for all others.
Could appear
Is unlikely to appear
Is highly unlikely to appear
Indicators developed for their particular scenario, the
home scenario, should be either highly likely or likely.
If the software is unavailable, you can do your own scoring. If the indicator is highly likely in the home scenario,
then in the other scenarios,
Highly likely is 0 points.
Task 4.
Use the Indicators ValidatorTM to assess the diagnosticity of
your indicators.
Step 1: Create a matrix similar to that used for Analysis of
Competing Hypotheses. This can be done manually or by
using the Indicators Validator TM software. Contact
Globalytica, LLC at [email protected] or go to
http://www.globalytica.com to obtain access to the
Indicators ValidatorTM software if it is not available on your
system. List the alternative scenarios along the top of the
matrix and the indicators that have been generated for each
of the scenarios down the left side of the matrix.
Step 2: Moving across the indicator rows, assess whether
the indicator for each scenario
Is highly likely to appear
Likely is 1 point.
Could appear is 2 points.
Unlikely is 4 points.
Highly unlikely is 6 points.
If the indicator is likely in the home scenario, then in the
other scenarios,
Highly likely is 0 points.
Likely is 0 points.
Could appear is 1 point.
Unlikely is 3 points.
Highly unlikely is 5 points.
Step 3: Tally up the scores across each row, as shown in
Table 12.7, and then rank order all the indicators.
Is likely to appear
Indicator
Scenario A
Scenario B
Scenario C
Scenario D
Score
HL
HU (6)
HU (6)
C (2)
14
A-2
HU (5)
HU (5)
L (0)
10
A-3
HL
HU (6)
HU (6)
C (2)
14
A-4
HL
HU (6)
HU (6)
HL (0)
12
A-5
HU (5)
HU (5)
C (1)
11
A-6
HL
HL (0)
HL (0)
HL (0)
Indicator
Scenario A
Scenario B
Scenario C
Scenario D
Score
A-7
HU (5)
HU (5)
C (1)
11
A-8
U (3)
U (3)
L (0)
A-9
HU (5)
HU (5)
L (0)
10
Scenario B: FARC uses rompas to launch mortar attack on USSOUTHCOM headquarters in Miami.
B-1
C (1)
C (1)
L (0)
B-2
HL (0)
HL
HL (0)
L (1)
B-3
U (4)
HL
C (2)
C (2)
B-4
HU (5)
U (3)
U (3)
11
B-5
C (2)
HL
HL (0)
C (2)
B-6
U (3)
C (1)
U (3)
B-7
U (4)
HL
L (1)
C (2)
B-8
HU (6)
HL
C (2)
HU (6)
14
U (4)
C (2)
HL
U (4)
10
C-2
L (1)
HL (0)
HL
L (1)
C-3
U (4)
C (2)
HL
U (4)
10
C-4
U (4)
C (2)
HL
U (4)
10
C-5
U (3)
L (0)
U (3)
C-6
U (4)
U (4)
HL
U (4)
12
C-7
U (4)
U (4)
HL
U (4)
12
Scenario D: Marijuana laced with poison kills many in the vicinity of US military bases.
D-1
C (1)
C (1)
U (3)
(Continued)
142 Chapter 12
Table 12.7 FARC Attack on the US Homeland: Indicators ValidatorTM Scoring (Continued)
Scenario A
Scenario B
Scenario C
Scenario D
Score
D-2
Number
Indicator
C (1)
U (3)
U (3)
D-3
L (0)
U (3)
U (3)
D-4
C (2)
U (4)
U (4)
HL
10
D-5
L (0)
U (3)
U (3)
D-6
C (2)
U (4)
U (4)
HL
10
D-7
HL (0)
HL (0)
HL (0)
HL
D-8
HL (0)
U (4)
U (4)
HL
Note: HL = highly likely to appear; L = likely to appear; C = could appear; U = unlikely to appear; HU = highly unlikely to appear.
Table 12.8 FARC Attack on the US Homeland: Rank Ordering of the Indicators on the Basis of Diagnosticity
Number
Indicator
Scenario A
Scenario B
Scenario C
Scenario D
Score
A-1
HL
HU (6)
HU (6)
C (2)
14
A-3
HL
HU (6)
HU (6)
C (2)
14
B-8
HU (6)
HL
C (2)
HU (6)
14
A-4
HL
HU (6)
HU (6)
HL (0)
12
C-6
U (4)
U (4)
HL
U (4)
12
C-7
U (4)
U (4)
HL
U (4)
12
A-5
HU (5)
HU (5)
C (1)
11
A-7
HU (5)
HU (5)
C (1)
11
B-4
HU (5)
U (3)
U (3)
11
A-2
HU (5)
HU (5)
L (0)
10
A-9
HU (5)
HU (5)
L (0)
10
Indicator
Scenario A
Scenario B
Scenario C
Scenario D
Score
C-1
U (4)
C (2)
HL
U (4)
10
C-3
U (4)
C (2)
HL
U (4)
10
C-4
U (4)
C (2)
HL
U (4)
10
D-4
C (2)
U (4)
U (4)
HL
10
D-6
C (2)
U (4)
U (4)
HL
10
B-3
U (4)
HL
C (2)
C (2)
D-8
HL (0)
U (4)
U (4)
HL
B-6
U (3)
C (1)
U (3)
B-7
U (4)
HL
L (1)
C (2)
D-2
C (1)
U (3)
U (3)
A-8
U (3)
U (3)
L (0)
C-5
U (3)
L (0)
U (3)
D-3
L (0)
U (3)
U (3)
D-5
L (0)
U (3)
U (3)
D-1
C (1)
C (1)
U (3)
B-5
C (2)
HL
HL (0)
C (2)
B-1
C (1)
C (1)
L (0)
C-2
L (1)
HL (0)
HL
L (1)
B-2
HL (0)
HL
HL (0)
L (1)
A-6
HL
HL (0)
HL (0)
HL (0)
D-7
HL (0)
HL (0)
HL (0)
HL
Note: HL = highly likely to appear; L = likely to appear; C = could appear; U = unlikely to appear; HU = highly unlikely to appear.
144 Chapter 12
highly likely to appear in all scenarios. Most indicators
will fall somewhere in between.
Step 5: The indicators with the most highly unlikely and
unlikely ratings are the most discriminating and should be
retained.
Step 6: Indicators with no highly unlikely or unlikely ratings
should be discarded.
Step 7: Use your judgment as to whether you should
retain or discard indic ators that score fewer points.
Table 12.9 FARC Attack on the US Homeland: Rank Ordering of the Indicators on the Basis of Diagnosticity by
Scenario
Number
Indicator
Scenario A
Scenario B
Scenario C
Scenario D
Score
HL
HU (6)
HU (6)
C (2)
14
A-3
HL
HU (6)
HU (6)
C (2)
14
A-4
HL
HU (6)
HU (6)
HL (0)
12
A-5
HU (5)
HU (5)
C (1)
11
A-7
HU (5)
HU (5)
C (1)
11
A-2
HU (5)
HU (5)
L (0)
10
A-9
HU (5)
HU (5)
L (0)
10
A-8
U (3)
U (3)
L (0)
Scenario B: FARC uses rompas to launch mortar attack on USSOUTHCOM headquarters in Miami.
B-8
HU (6)
HL
C (2)
HU (6)
14
B-4
HU (5)
U (3)
U (3)
11
B-3
U (4)
HL
C (2)
C (2)
B-6
U (3)
C (1)
U (3)
B-7
U (4)
HL
L (1)
C (2)
Indicator
Scenario A
Scenario B
Scenario C
Scenario D
Score
U (4)
U (4)
HL
U (4)
12
C-7
U (4)
U (4)
HL
U (4)
12
C-1
U (4)
C (2)
HL
U (4)
10
C-3
U (4)
C (2)
HL
U (4)
10
C-4
U (4)
C (2)
HL
U (4)
10
C-5
U (3)
L (0)
U (3)
Scenario D: Marijuana laced with poison kills many in the vicinity of US military bases.
D-4
C (2)
U (4)
U (4)
HL
10
D-6
C (2)
U (4)
U (4)
HL
10
D-8
HL (0)
U (4)
U (4)
HL
D-2
C (1)
U (3)
U (3)
D-3
L (0)
U (3)
U (3)
D-5
L (0)
U (3)
U (3)
D-1
C (1)
C (1)
U (3)
Note: HL = highly likely to appear; L = likely to appear; C = could appear; U = unlikely to appear; HU = highly unlikely to appear.
146 Chapter 12
U (4)
HL
U (4)
C (2)
10
B-10
U (4)
HL
C (2)
C (2)
Note: HL = highly likely to appear; L = likely to appear; C = could appear; U = unlikely to appear; HU = highly unlikely to appear.
Do these indicator lists provide useful leads for alerting FBI field offices and state and local fusion centers of
plausible, potential emerging threats? Yes, the indicators
are sufficiently specific to provide operationally useful guidance to field offices or fusion centers.
Are they focused enough to generate specific collection requirements, giving federal, state, local, and tribal
officials a more concrete idea of what to look for? Yes, the
technique has generated a robust set of concrete indicators
that provide effective guidance to the field.
KEY TAKEAWAYS
When analysts have little data and a mandate to
anticipate a potential terrorist attack, often the
NOTES
1. The description of Red Hat Analysis in this case was taken
from the first edition of Structured Analytic Techniques for
Intelligence Analysis. A more robust approach for conducting Red
Analytic Family
Simple Hypotheses
p. 171
p. 250
Challenge Analysis
p. 122
Idea Generation
147
148 Chapter 13
A consolidated set of hypotheses might look like Table 13.4.
Step 3: Aggregate the hypotheses into affinity groups and
label each group.
Consider multiple ways to display the affinity groups. In
this case, the hypotheses may be grouped by the issue of
autonomy, addressing the question of whether 17N worked
alone or in collaboration with other violent groups active in
Greece and Europe. Another important consideration is
motive, and whether 17N was truly a manifestation of radical politics or whether it was alsoor insteada criminal
enterprise.
Step 4: Use problem restatement and consideration of the
opposite to develop new ideas.
Problem Restatement: Why did it take twenty-seven
years to capture the members of 17N?
Consideration of the Opposite: 17N benefitted
from official protection. 17N benefitted from the
limitations of Greek police and security services.
17N evaded detection because its attacks were
so low-tech. All of these ideas have implications
about 17Ns identity and motive and help expand
explanations for what the group might have
been. Also consider whether 17Ns longevity
might be due to its evolutionary nature. Was 17N
consistently the same thing for the length of its
period of activity? Might its motives, composition,
and objectives have changed over time?
Step 5: Update the list of alternative hypotheses.
Problem restatement augments the list of hypotheses by
including the possibility of government collusion or protection. It also raises the possibility that the groups motive,
objectives, and identity evolved over time.
Step 6: Clarify each hypothesis by asking Who? What?
How? When? Where? and Why?
Make a list of each of the categories. Step back and consider how each list could be augmented. Who and What
suggest possible identities: an autonomous group of Greek
violent extremists, a criminal enterprise, or a subgroup of a
larger regional violent extremist movement? When
addresses the issue of whether 17N had a consistent identity, composition, and objectives over the years, or whether
it evolved. Where addresses the theater of operations: All
claimed attacks were in Athens, but could there have been
activity elsewhere not credited to the group? How
addresses the longevity of the groups success. If it evaded
detection for so many years because of the low-tech nature
of its attacks, what does that also say about what it was?
Why addresses motive: to inspire political revolution, to
make money, to advance political goals of invested officials?
Refine this list to make the categories as mutually exclusive
as possible. This helps clarify the hypotheses.
Step 7: Select the most promising hypotheses for further
exploration.
17N is a Greek violent far-left group that, for a period
of time, worked in collaboration with other violent
groups, Greek and/or foreign, to inspire a Marxist
revolution.
17N is a Greek violent extremist group working in
conjunction with criminal enterprises, in Greece and
regionally, both for monetary gain and to advance a
political agenda.
17N is a group manipulated by or influenced by
Greek political officials to engage in dirty politics in
Athens.
Analytic Value Added: Did using the technique help
you challenge conventional wisdom about the group and
its motives? The technique generated several new ways to
think about the group, suggesting different motives in
particular. This is important because the analyst now will be
looking for additional indicators that can prove or disprove
each of the hypotheses.
Did it reveal ideas or concepts that you might have
missed if you had engaged in conventional brainstorming
only? The technique raised the possibility that 17N might
be operating entirely or partially for criminal motives and
may have evolved over timeideas that certainly would
require more research.
Was it difficult to select those hypotheses that
deserved the most attention? As themes emerged from the
Structured Brainstorming process, it was helpful to use
Figure 13.1 What If? Analysis Scenario: 17N Shoots US Military Officer
It is 1999, the peak of the NATO campaign in the Balkans. The majority of Greeks feel a religio-ethnic affinity with the
Serbs, and vehemently oppose the strikes and any overt support given to the Bosnians and Kosovars by the West. Popular
protests make it clear that this is an issue that resonates with a large swath of the Greek people. 17N sees an opportunity
to advance its agenda and decides to target a US military officer with NATO ties. Senior US military officers or defense
attachs affiliated with the embassy and stationed in Athens are afforded careful security protection by both DoD and
Diplomatic Security. They have armored vehicles and, sometimes, security escorts, and their drivers carefully vary their
routes. All vehicles entering the embassy compound are screened for explosives, and the building itself is inaccessible to
outsiders. Their residences and families are similarly protected. Lower-level officers also receive security training and are
instructed to report any signs of surveillance or unusual behavior. All local embassy hires are carefully screened.
Despite this high security, 17N is still focused on targeting an American military officer and making a statement about
what the group perceives to be immorality of a US-backed NATO campaign. It decides to monitor the major restaurants
and tourist venues in central Athens, where American Embassy personnel are known to congregate, but finds that there
are too many people and it is too hard to distinguish which Americans might have military affiliations. It surveils all cars
coming and going from the embassy compound and finds that some lower-level officers with less security detail are not
always careful about varying their commutes to and from work, especially after several months at post.
One young man in particular, who drives an old model Honda, takes the same major thoroughfare to the embassy from
his residence every day. His short haircut suggests he might have a military affiliation. 17N decides it is their best shot
and plots a drive-by shooting timed for the peak morning rush hour. It prepares the proclamation in advance, accusing
the nameless American of being centrally involved in the incursion into Serbian sovereign space.
150 Chapter 13
For the military officer scenario, the killing would signal that 17N was still active, and security would be heightened not only for US officials but also other for diplomatic
posts in Athens and the Greek government and private
sector.
Step 5: Rank the scenarios in terms of which deserves the
most attention by taking into consideration the difficulty of
implementation and the potential severity of the impact.
Depending on how the other scenarios are constructed,
a likely ranking in descending order of difficulty of implementation would be:
Scenario C: 17N assassinates US political counselor
near US Embassy
Scenario A: 17N shoots US military officer en route
to work
Scenario B: 17N bombs US Embassy vehicle in Athens
Analytic Value Added: Did the technique help you
generate new ways of thinking about the problem? The
technique moved the conversation beyond the debate over
whether 17N is still a viable terrorist organization, but it did
not generate new ideas regarding what type of attack might
be launched. It did, however, provide insight into the
likelihood of a particular type of attack based on degree of
difficulty.
Did it help you assess how difficult each scenario
would be to carry out? By working ones way step by step
through each scenario, it is easier to assess how 17N is most
likely to launch each attack and assess what is required for
each to succeed.
The Foresight Quadrant CrunchingTM technique differs from Classic Quadrant CrunchingTM in that the focus is
on all of the ways something could happen, not just what
might be different. In this version of the technique, the lead
hypothesis dimensions are included in the analysis. Foresight Quadrant CrunchingTM is similar to Classic Quadrant
CrunchingTM, however, in that both use contrary dimensions versus spectrums to define the endpoints of the xand y-axes.
To use our previous example again, the analyst begins
with a lead hypothesis (A criminal group has penetrated a
large corporate database to steal Personal Identity Information [PII]), breaks the lead hypothesis into its component
parts (criminal group/to steal PII); flips the assumption
inherent in each segment (noncriminal group/alternative
motives); brainstorms contrary dimensions (usually from
one to three) consistent with the flipped assumption (busi
ness competitor or foreign country, to download corporate
data or to alter corporate information); and then lists all
possible combinations, comprising nine different attack
scenarios:
The Foresight Quadrant CrunchingTM technique is particularly applicable to the 17N case because (1) little was
known about the identity of the group members or their
plans while they were active, and (2) in several cases only
one credible alternative dimension merited the analysts
152 Chapter 13
Lead Hypothesis
Contrary Dimension
Who? (target)
US official
What? (tactics)
Assassination
How? (weapon)
Remote-control bomb
When? (timing)
Where? (location)
In metropolitan Athens
Why? (motives)
Rockets
US official
In metropolitan Athens
2
Tourists at Olympics
US official
Outside Athens
In metropolitan Athens
Tourists at Olympics
Outside Athens
Target/Tactics
5
US official
Assassination
6
US official
Hostage taking/kidnapping
Tourists at Olympics
Assassination
Tourists at Olympics
Hostage taking/kidnapping
Location/Tactics
9
In metropolitan Athens
11
Assassination
10
Outside Athens
Assassination
In metropolitan Athens
Hostage taking/kidnapping
12
Outside Athens
Hostage taking/kidnapping
154 Chapter 13
Alternative Scenario
Rating
10
11
12
CONCLUSION
On June 29, 2002, a botched attempted bombing by one of
the core members of 17N led to his arrest, confession, and
the subsequent unraveling of the group. Savvas Xiros, a
name new to Greek police, was seriously injured when a
homemade explosive device he had placed behind a Flying
Dolphin ferry ticket kiosk in Piraeus exploded prematurely.
Xiros, a largely self-taught bomb maker, lost several fingers
and suffered permanent damage to his eyes. The port
police who responded to the blast discovered a second
bomb and, more significantly, a bag containing a gun that
linked to a 17N bank robbery in 1984 in which a police
officer had been killed.1 After Savvass photo was placed on
Greek television, an anonymous caller provided information connecting him to a safehouse.2 Two apartments were
discovered, chock full of all the materials 17N used to carry
out its attacks: stolen license plates, keys, forging materials,
pvc pipes, guns, bullets, costumes, proclamations, surveillance notes, and perhaps most interesting of all, a detailed
ledger that chronicled the members pay and expenses per
operative alias.3
Savvas awoke in the hospital under heavy police guard,
and spent the next few weeks being interrogated. Police
aggressively pursued all leads stemming from Savvass
(a)
(b)
(c)
156 Chapter 13
Using techniques such as Foresight Quadrant
CrunchingTM, analysts can better anticipate the
unanticipated and create alternative stories or
bins that could prove useful when newly obtained
information does not fit comfortably within
established investigative categories.
NOTES
1. Tamara Makarenko and Daphne Biliouri. Is this the end
of 17N? Janes Intelligence Review 14 (2002): 9.
2. Ibid.
3. Kiesling, Brady, Greek Urban Warriors: Resistance and
Terrorism 19672012, Athens: Lycabettus Press (forthcoming).
4. Shawn Choy, In the Spotlight Revolutionary Organization
17 November, CDI Terrorism Project, August 5, 2002. www.cdi
.org/terrorism/17N-pr.cfm
Analytic Family
Structured Brainstorming
p. 102
Idea Generation
p. 223
p. 122
Idea Generation
Indicators
p. 149
Indicators Validator
p. 157
157
158 Chapter 14
Step 10: Look for sticky notes that do not fit neatly into
any of the groups. Consider whether such an outlier is useless noise or the germ of an idea that deserves further
attention.
Step 11: Assess what the group has accomplished. How
many different ways have you identified that the assailants
could transport a team to Mumbai?
Step 12: Present the results, describing the key themes or
dimensions of the problem that were identified. Consider
less conventional means of presenting the results by engaging in a hypothetical conversation in which terrorist leaders
discuss the issue in the first person.
Over the course of the exercise, students should generate between twenty and fifty ideas. Groups familiar with
the region or with terrorist activity are likely to generate
more ideas. The most obvious ways to group the
responses would be to distinguish efforts to access Mumbai by sea, by land, or by air. If the students are having
trouble coming up with ideas or their ideas are too general, ask them to drill down on specific ways the terrorists
would come to Mumbai using different modes of transport. Table 14.4 provides a sampling of likely responses.
Encourage the students to be creative, as this usually
builds energy within the group. Some groups, for example, have proposed using gliders, parachutes, and even
Segways. Other seemingly out-of-the-box ideas that could
merit attention are bicycle tours and the use of humantrafficking networks.
Analytic Value Added: Were we careful to avoid
mirror imaging when we put ourselves in the shoes of
Muslim terrorist planners? While a regular citizen
might use commercial air or a border crossing to enter
India, we cannot assume that terrorists would do the
same. The risks of apprehension are too high. Also, some
of the ideas generated may not prove practical if the terrorists need to transport weapons and explosives with
them to Mumbai. Crossing the border or transiting
through an airport might prove impractical, suggesting
that ideas such as using commercial aircraft for transit are
unlikely.
By Land
By Air
If two-staged transit:
If two-staged transit:
If two-staged transit:
160 Chapter 14
TECHNIQUE 2: RED HAT ANALYSIS
Analysts frequently endeavor to forecast the actions of an
adversary or a competitor. In doing so, they need to avoid
the common error of mirror imaging, the natural tendency
to assume that others think and perceive the world in the
same way as they do. Red Hat Analysis is a useful technique
for trying to perceive threats and opportunities as others see
them, but this technique alone is of limited value without
significant understanding of the cultures of other countries,
groups, or people involved. There is a great deal of truth to
the maxim that where you stand depends on where you
sit. By imagining the situation as the target perceives it, an
analyst can gain a different and usually more accurate perspective on a problem or issue.
Reframing the problem typically changes the analysts
perspective from that of an analyst observing and forecasting an adversarys behavior to that of someone who must
make difficult decisions within that operational culture.
This reframing process often introduces new and different
stimuli that might not have been factored into a traditional
analysis.
Task 2.
Use Red Hat Analysis to prioritize the list of various modes
of transport the terrorists might use to enter Mumbai.1
Step 1: Gather a group of experts with in-depth knowledge of the target, operating environment, and the terrorist
groups motives and style of thinking. If at all possible, try to
include people who are well grounded in Mumbais culture,
speak the language, share the same ethnic background, or
have lived extensively in the region.
Step 2: Ask group members to develop a list of criteria
that they would most likely use when deciding which
modes of transport they personally would choose to enter
Mumbai. The reason for first asking the group how it would
act is to establish a baseline for assessing whether the terrorists are likely to act differently.
Key criteria would include the following:
Minimizing the chances of detection prior to
implementing the plan.
Minimizing the chances of detection while in transit.
Minimizing the chances of detection during the
attack.
Rating
Analytic Value Added: Was your list of criteria comprehensive? The list provided in Table 14.4 is fairly comprehensive, but challenging the students to come up with a
few more ideas is always recommended. Terrorist groups
can be very innovative, and surprise will work to their
advantage.
Did some criteria deserve greater weight than others?
Did you reflect this when you rated the various ideas?
The process of rating each idea allows the students to reflect
on the criteria they have developed. In this case, the concept
of a staged transit appears to have the most utility. If traveling by sea, the assailants would need a larger ship that is
ocean-worthy but then would have to transfer to some less
visible mode of transit upon arriving in the vicinity of
Mumbai.
162 Chapter 14
Usually the students will propose to add criteria to the
list. In this instance, one question would be whether the
possibility of renting trucks (as has been done in the United
States) or stealing them would be a viable option in India or
Pakistan. Another issue that might arise is what strategy the
terrorists have decided to adopt. If the intent is to launch a
suicide bombing, then options using aircraft might be rated
higher.
TECHNIQUE 3: CLASSIC QUADRANT
CRUNCHINGTM
Classic Quadrant CrunchingTM combines the methodology
of a Key Assumptions Check 3 with Multiple Scenarios
Generation4 to generate an array of alternative scenarios or
stories. This process is particularly helpful in the Mumbai
case because little is known about the actual plans and
intentions of the attackers. This technique helps the analyst
identify and challenge key assumptions that may underpin
the analysis while generating an array of credible alternative
scenarios to help law enforcement focus on the most likely
types of attacks to anticipate.
Task 3.
Use Classic Quadrant CrunchingTM to brainstorm all the
possible ways terrorists might launch an attack on Mumbai.
List the scenarios from most to least likely.
Table 14.6 Defending Mumbai Classic Quadrant CrunchingTM: Contrary Dimensions Example
Key Components
Lead Hypothesis
Who?
(attacker)
Lakar-e-Taiba (LeT)
Jaish-e-Mohammed (JEM)
What?
(weapon)
Small explosives
Large explosives
Where?
(targets)
How?
(tactics)
A single event
An extended event
Why?
(motives)
When?
(timing)
On a significant date
164 Chapter 14
Small explosives
Small explosives
Multiple events
Extended event
Large explosives
Large explosives
Multiple events
Extended event
Weapon/Locations
5
Small explosives
Small explosives
Transit locations
Religious locations
Large explosives
Large explosives
Transit locations
Religious locations
Tactics/Locations
9
10
Multiple events
11
Extended event
Transit locations
Transit locations
Multiple events
12
Extended event
Religious locations
Religious locations
Alternative Scenario
Rating
LeT launches simultaneous attacks using small arms and explosives targeting several hotels, the train
station, and several restaurants.
LeT attacks the Taj Hotel with small arms and grenades and takes hostages; it also uses small explosives to
set fire to the hotel.
10
LeT orchestrates a series of simultaneous attacks using small arms and grenades against Hindu temples and
a Jewish synagogue, taking hostages at two of the locations.
LeT attacks the main train station, a bus depot, and people congregating at bus stops, throwing small
explosives from motorcycles and setting small bombs in the train station.
LeT orchestrates a series of cascading attacks, beginning with small-arms fire and escalating to increasingly
large bomb attacks targeting bus stops, bus depots, trains, and train stations.
11
LeT attacks the train station, takes hundreds of hostages, and sets up a defensive perimeter, leading to an
extended siege.
LeT explodes several large suicide car bombs at hotels, the train station, and several restaurants.
LeT suicide bombers with vests attack several Hindu temples, a Jewish synagogue, and a Christian church.
12
LeT attacks a Jewish religious center or synagogue and takes hostages, leading to an extended siege.
Large bombs are detonated at a train station and the airport, causing major casualties.
LeT, with the support of insiders, explodes large preset bombs at various religious sites and then ambushes
the first responders.
166 Chapter 14
awareness that prepares the mind to recognize and prevent
a bad scenario from unfolding or help a good scenario to
come about.
Task 4.
Create separate sets of indicators for the most attentiondeserving scenarios, including those that were generated in
Task 3, the Classic Quadrant CrunchingTM exercise.
Step 1: Create a list of the most attention-deserving scenarios to track for this case.
Students should be encouraged to select the most attention-deserving scenarios, realizing that time is of the essence
and the list should be kept short, preferably to no more than
five scenarios. Usually that will require combining some
scenarios that share similar characteristics. Table 14.9 provides an illustrative list of attention-deserving scenarios.
Quadrants
Represented
Lead Hypothesis
1, 5, 9, 10
2, 7
3, 10, 11, 12
Attention-Deserving Scenario
Scenario 1, Simple Armed Assault: LeT conducts an armed assault with AK-47s and grenades launched from the sea against the Taj Hotel.
1-a
1-b
Suspicious people are only observed surveilling the Taj Mahal Palace.
1-c
People renting rooms at the Taj Mahal Palace for several weeks appear suspicious.
1-d
1-e
1-f
Reports tell of LeT purchases of assault rifles, grenades, and ammunition in Pakistan.
1-g
Sources report that the attack team is small (five or fewer people).
1-h
1-i
Documents captured in LeT possession show sketches of only the Taj Hotel.
Scenario 2, Simultaneous Attacks: LeT launches simultaneous attacks from the sea using small arms and
explosives targeting several hotels, a train station, religious sites, and restaurants.
2-a
Sources report LeT is providing training in small arms, portable bombs, preset bombs, and grenades at camps in Pakistan.
2-b
Suspicious people are observed surveilling a large number of prominent public sites in Mumbai.
2-c
2-d
Reports tell of LeT purchases or acquisition of assault rifles, grenades, and ammunition.
2-e
Reports tell of LeT purchases or acquisition of RDX and other bomb materials.
2-f
Sources report the attackers are formed into several teams and number more than five.
2-g
2-h
2-i
3-a
3-b
Sources report LeT is providing training in the use of suicide vests or it is practicing deploying suicide car or truck bombs.
3-c
3-d
Suspicious people are observed surveilling a large number of prominent public sites in Mumbai.
3-e
LeT posts virulent anti-Indian rhetoric on its website justifying the use of suicide bombers.
3-f
3-g
3-h
3-i
4-a
4-b
4-c
(Continued)
168 Chapter 14
Table 14.10 Mumbai Indicators for Most Attention-Deserving Scenarios Examples (Continued)
Number
Attention-Deserving Scenario
4-d
4-e
4-f
4-g
4-h
Sources report that LeT operatives will carry handcuffs, tape, phones in their packs.
4-i
Sources report that LeT is scouting for locations that can be easily defended.
4-j
Sources report that LeT camps are providing training in defending fixed positions.
Task 5.
Use the Indicators ValidatorTM to assess the diagnosticity of
your indicators.
Step 1: Create a matrix similar to that used for Analysis
of Competing Hypotheses.6 This can be done manually
or by using the Indicators ValidatorTM software. Contact
Globalytica, LLC at [email protected] or go
to http://www.globalytica.com to obtain access to the
Indicators Validator TM software if it is not available on
your system. List the alternative scenarios along the top
of the matrix and the indicators that have been generated for each of the scenarios down the left side of the
matrix.
Step 2: Moving across the indicator rows, assess whether
the indicator for each scenario
Is highly likely to appear
Is likely to appear
Could appear
Is unlikely to appear
Is highly unlikely to appear
Indicators developed for their particular scenario, the
home scenario, should be either highly likely or likely.
If the software is unavailable, you can do your own scoring. If the indicator is highly likely in the home scenario,
then in the other scenarios,
Highly likely is 0 points.
Likely is 1 point.
Could is 2 points.
Unlikely is 4 points.
Highly unlikely is 6 points.
If the indicator is likely in the home scenario, then in the
other scenarios,
Highly likely is 0 points.
Likely is 0 points.
Could is 1 point.
Unlikely is 3 points.
Highly unlikely is 5 points.
Step 3: Tally up the scores across each row and then rank
order all the indicators.
Table 14.11 shows how each indicator was rated for each
scenario. The number beside the rating is the score. It is
important to remind the students that the scoring for
home scenario indicators rated likely is different from the
scoring for home scenario indicators rated highly likely.
Indicator
Scenario 1
Scenario 2
Scenario 3
Scenario 4
Score
Scenario 1, Simple Armed Assault: LeT conducts an armed assault with AK-47s and grenades launched from the sea against the Taj Hotel.
1-a
HL
HL (0)
L (1)
HL (0)
1-b
HL
HL (0)
HL (0)
HL (0)
1-c
HL
L (1)
L (1)
HL (0)
1-d
HL
HL (0)
HL (0)
HL (0)
1-e
L (0)
L (0)
L (0)
1-h
L (0)
C (1)
L (0)
1-i
HL
U (4)
U (4)
C (2)
10
Scenario 2, Simultaneous Attacks: LeT launches simultaneous attacks from the sea using small arms and
explosives targeting several hotels, a train station, religious sites, and restaurants.
2-a
C (2)
HL
U (4)
HL (0)
2-b
U (3)
L (0)
L (0)
2-c
L (0)
L (0)
L (0)
(Continued)
170 Chapter 14
Indicator
Scenario 1
Scenario 2
Scenario 3
Scenario 4
Score
HU (6)
HL
L (1)
L (1)
2-e
2-f
U (4)
HL
C (2)
C (2)
2-h
U (3)
C (1)
C (1)
2-i
U (3)
HL
C (2)
C (2)
Scenario 3, Suicide Attacks: LeT orchestrates several simultaneous attacks launched from the sea using suicide
bombers to target several public places, including hotels, the train station, and religious sites.
3-a
U (4)
U (4)
HL
HU (6)
14
3-b
HU (6)
HU (6)
HL
HU (6)
18
3-c
HU (6)
HU (6)
HL
HU (6)
18
3-d
U (3)
HL (0)
HL (0)
3-e
U (4)
L (1)
HL
C (2)
3-h
HU (5)
HU (5)
HU (5)
15
3-i
U (3)
U (4)
U (4)
11
Scenario 4, Hostage Taking: LeT attacks the Taj Hotel and possibly other sites from the sea,
including those frequented by foreigners, with small arms and takes hostages.
4-a
HL (0)
HL (0)
L (1)
HL
4-b
L (0)
HL (0)
L (0)
4-c
HL (0)
HL (0)
L (1)
HL
4-e
U (2)
HL (0)
C (1)
4-f
U (4)
HL (0)
HL (0)
HL
4-g
U (4)
C (2)
HL (0)
HL
4-h
U (2)
U (2)
U (2)
4-i
U (2)
C (1)
U (2)
4-j
U (2)
U (2)
U (2)
Note: HL = highly likely to appear; L = likely to appear; C = could appear; U = unlikely to appear; HU = highly unlikely to appear.
Indicator
Scenario 1
Scenario 2
Scenario 3
Scenario 4
Score
3-b
HU (6)
HU (6)
HL
HU (6)
18
3-c
HU (6)
HU (6)
HL
HU (6)
18
3-h
HU (5)
HU (5)
HU (5)
15
3-a
U (4)
U (4)
HL
HU (6)
14
3-i
U (3)
U (4)
U (4)
11
1-i
HL
U (4)
U (4)
C (2)
10
2-e
HU (6)
HL
L (1)
L (1)
2-f
U (4)
HL
C (2)
C (2)
2-i
U (3)
HL
C (2)
C (2)
3-e
U (4)
L (1)
HL
C (2)
2-a
C (2)
HL
U (4)
HL (0)
4-g
U (4)
C (2)
HL (0)
HL
4-h
U (2)
U (2)
U (2)
4-j
U (2)
U (2)
U (2)
2-h
U (3)
C (1)
C (1)
4-i
U (2)
C (1)
U (2)
4-f
U (4)
HL (0)
HL (0)
HL
2-b
U (3)
L (0)
L (0)
(Continued)
172 Chapter 14
Indicator
Scenario 1
Scenario 2
Scenario 3
Scenario 4
Score
3-d
U (3)
HL (0)
HL (0)
4-e
U (2)
HL (0)
C (1)
1-c
HL
L (1)
L (1)
HL (0)
1-a
HL
HL (0)
L (1)
HL (0)
1-h
L (0)
C (1)
L (0)
4-a
HL (0)
HL (0)
L (1)
HL
4-c
HL (0)
HL (0)
L (1)
HL
1-b
HL
HL (0)
HL (0)
HL (0)
1-d
HL
HL (0)
HL (0)
HL (0)
1-e
L (0)
L (0)
L (0)
2-c
L (0)
L (0)
L (0)
4-b
L (0)
HL (0)
L (0)
Note: HL = highly likely to appear; L = likely to appear; C = could appear; U = unlikely to appear; HU = highly unlikely to appear.
Step 8: Once nondiscriminating indicators have been eliminated, regroup the indicators under their home scenarios.
Overall, twenty indicators were deemed diagnostic, and
ten were discarded as not sufficiently diagnostic to be useful
in the analysis. When these twenty indicators are re-sorted
by scenario, as shown in Table 14.13, it is immediately
apparent that there is an insufficient number of diagnostic
indicators for Scenario 1, Simple Armed Assault.
Step 9: If a large number of indicators for a particular
scenario have been eliminated, develop additionaland
more diagnosticindicators for that scenario.
Step 10: Recheck the diagnostic value of any new indicators by applying the Indicators ValidatorTM to them as well.
In this case, students should generate a new set of
diagnostic indicators for Scenario 1. The problem confronted when trying to come up with Scenario 1 indicators is that the scenario is a fairly basic scenario and most
Indicator
Scenario 1
Scenario 2
Scenario 3
Scenario 4
Score
Scenario 1, Simple Armed Assault: LeT conducts an armed assault with AK-47s and grenades launched from the sea against the Taj Hotel.
1-i
HL
U (4)
U (4)
C (2)
10
Scenario 2, Simultaneous Attacks: LeT launches simultaneous attacks from the sea using small arms
and explosives targeting several hotels, a train station, religious sites, and restaurants.
2-e
HU (6)
HL
L (1)
L (1)
2-f
U (4)
HL
C (2)
C (2)
2-i
U (3)
HL
C (2)
C (2)
2-a
C (2)
HL
U (4)
HL (0)
2-h
U (3)
C (1)
C (1)
2-b
U (3)
L (0)
L (0)
Scenario 3, Suicide Attacks: LeT orchestrates several simultaneous attacks launched from the sea using
suicide bombers to target several public places, including hotels, a train station, and religious sites.
3-b
HU (6)
HU (6)
HL
HU (6)
18
3-c
HU (6)
HU (6)
HL
HU (6)
18
3-h
HU (5)
HU (5)
HU (5)
15
3-a
U (4)
U (4)
HL
HU (6)
14
3-i
U (3)
U (4)
U (4)
11
3-e
U (4)
L (1)
HL
C (2)
3-d
U (3)
HL (0)
HL (0)
Scenario 4, Hostage Taking: LeT attacks the Taj Hotel and possibly other sites from the sea,
including those frequented by foreigners, with small arms and takes hostages.
4-g
U (4)
C (2)
HL (0)
HL
4-h
U (2)
U (2)
U (2)
4-j
U (2)
U (2)
U (2)
4-i
U (2)
C (1)
U (2)
4-f
U (4)
HL (0)
HL (0)
HL
4-e
U (2)
HL (0)
C (1)
Note: HL = highly likely to appear; L = likely to appear; C = could appear; U = unlikely to appear; HU = highly unlikely to appear.
174 Chapter 14
Sources report that only small numbers of weapons
and small amounts of ammunition will be used in
the operation.
Ca
C
Cama
&A
Albless
b ess H
Hospital
p
il Terminus
T
u
Rail
Ob
-T d
H t
Oberoi-Trident
Hotel
Leop
p d Cafe
C
Leopold
T
h lP
t
Taj Mahal
Palace Hotel
Chabad House
0
0
1000 feet
500 meters
Driven from
station, heads
for Cama
& Albless
Hospital.
TEAM 2
enters Victoria
Station and
attacks
travelers.
10PM 10:30PM
TEAM 5
forces its
way inside
the
Chabad
House.
TEAM 4
enters
OberoiTrident Hotel
and attacks
guests.
TEAM 3 heads
to Taj Hotel
and begins its
assault in the
lobby.
TEAM 1
enters
Leopold
Cafe and
opens fire.
9:30PM
Attacks of
26 November
8:30PM 9PM
Takes
hostages
and fortifies
itself inside
building.
Moves to
upper
floors of
hotel and
takes
hostages.
Four militants
head upstairs
in hotel,
taking
hostages and
starting fires
as they go.
Ambushes
police van,
commandeers it, and
heads for
Trident
Hotel.
Shoots at
theatergoers and
other targets
of
opportunity
as they
drive.
Two taxi
cabs
utilized
by
attackers
explode.
Immense
blast at
Taj Hotel.
1AM
Encounters
En
police
po
roadblock;
roa
engages
in
en
firefight
that
fifire
leaves
one
le
ea
attacker
atta
dead,
one
de
wounded
wo
and
an in
custody.
cus
11PM
Abandons van,
steals a
car, and
heads for
coast
after
failing to
reach
Trident.
2AM
Miltary
arrives.
Military
arrives.
Military
arrives
as fires
spread
inside
hotel.
Events on
27 November
Figure 14.1 Timeline of Mumbai Attacks and Aftermath, 2629 November 2008
TIME
6AM
9AM
Indian
assault on
the hotel
begins. A
militant
inside the
hotel tells
the Indian
media seven
attackers
have taken
hostages.
Indian
security
forces enter
the hotel
and begin
searching
inside.
11AM 5:30PM
An IED
destroys
part of
one of
the
walls.
Most of the
building
has been
cleared
by Indians;
all but fifty
hostages
accounted
for.
8PM
Commandos
storm
the
Chabad
House.
8AM
Fire erupts
on fourth
floor of the
hotel;
grenade
blasts and
explosions
are heard.
= Team 5
= Team 4
= Team 3
= Team 2
= Team 1
Operations
end at
Chabad
with
both
gunmen
dead.
More
gunfire,
explosions,
and fires
inside hotel.
Last
gunman
is killed
by
Indian
security
forces.
Events on
29 November
Operations
end at
Trident
Hotel
with
both
militants
dead.
Events on
28 November
178 Chapter 14
i. Government of India, Mumbai Terrorist Attacks: Nov. 2629, 2008, Federation of American Scientists website: http://www.fas
.org/irp/eprint/mumbai.pdf.
ii. Angel Rabasa et al., The Lessons of Mumbai, Santa Monica, CA: RAND Corporation, 2009. Available at http://www.rand.org/pubs/occasional_
papers/2009/RAND_OP249.pdf.
iii. New York Police Department Intelligence Division, Mumbai Attack Analysis, December 4, 2008, http://publicintelligence.net/
nypd-law-enforcement-sensitive-mumbai-attack-analysis/.
The Aftermath
More than 160 people died, and over 300 people sustained injuries during the 60-hour rampage.69 In the wake of
the attacks, Indian investigators quickly identified the attackers as Pakistani. It was not difficult to link the attackers to
LeT once their nationality was established. By the time the
investigation concluded, Indian officials alleged that elements within the Pakistani intelligence services had helped
LeT with the assaultor, at the very least, had known about
the attack and done nothing to prevent it. The government
of Pakistan initially denied there was any connection
between that country and the attack.70 However, faced with
hours of intercepted phone calls and a mountain of forensic
evidence, Pakistani officials were ultimately forced to concede the assault was planned in their country and that the
gunmen had trained in LeT camps located there. In 2009,
Pakistan charged LeTs military chief and six less influential
suspects in the Mumbai attacks and brought them to trial.
US officials say, however, that the trial seems hopelessly
stalled over legal complications and conflict with India.71, 72
Kasab, the only gunman who survived the attack, initially
confessed to taking part in the attack, and he went on to
provide a great deal of information about his recruitment in
Pakistan, his training, and his fellow attackers.73 He later
changed his story in court and argued that he was a tourist
who had been framed by the Mumbai police. Kasab was
convicted of murder, damage to public property, and a host
of other minor charges in May 2010. It was not a simple act
of murder, the presiding judge said of the attacks at the conclusion of Kasabs trial. It was war.74 Kasab was sentenced
to death. More than thirty-eight other people, most of whom
live in Pakistan, have been charged in connection to the
attacks. LeT commander Rehman and at least nineteen others have been found guilty in absentia by Indian courts.
KEY TAKEAWAYS
Predicting how a terrorist group might launch an
attack is a daunting task. The best analyses consider
the broadest range of credible alternatives and then
narrow the list down to those that are most attention
deserving.
Structured Brainstorming provides a good method
for ensuring that all possible options have been
considered; its power is that it stimulates creative
thinking. Classic Quadrant CrunchingTM is a
more rigorous and systematic process that usually
generates a robust set of alternatives because it forces
the analyst to think about the problem from a wide
variety of very different optics.
When generating a list of indicators to guide
collection, analysts should focus their energies on
developing truly diagnostic indicators that can drive
the analysis and focus the attention of investigators
on what really matters, especially when time is of
the essence. Collectors usually prefer working with a
short list of tailored indicators as opposed to a long
list of all possible indicators that might be relevant.
In a crisis environment, imprecise and often incorrect
reporting is the norm, especially when relying
on eyewitness reports. Always include with such
information caveats as, for example, initial reports.
180 Chapter 14
NOTES
1. The description of Red Hat Analysis in this case was taken
from the first edition of Structured Analytic Techniques for
Intelligence Analysis. A more robust approach for conducting Red
Hat Analysis has subsequently been developed that appears in the
second edition of the book but was not used in this case study.
2. Richards J. Heuer Jr. and Randolph H. Pherson, Structured
Analytic Techniques for Intelligence Analysis, 2nd ed. (Washington,
DC: CQ Press, 2015), 63.
3. Ibid., 209.
4. Ibid., 144.
5. Angel Rabasa et al., The Lessons of Mumbai, Santa Monica,
CA: RAND Corporation, 2009. Available at http://www.rand.org/
pubs/occasional_papers/2009/RAND_OP249.pdf.
6. For a full explanation of Analysis of Competing Hypotheses,
see Structured Analytic Techniques for Intelligence Analysis, 2nd ed.
(Washington, DC: CQ Press, 2015), 181.
7. Sebastian Rotella, On the Trail of a Terrorist, ProPublica,
Washington Post, November 14, 2010, http://www.washingtonpost
.com/wp-dyn/content/article/2010/11/13/AR2010111304345
.html.
8. Ibid.
9. Richard Esposito, Brian Ross, and Pierre Thomas, US
Warned India in October of Potential Terror Attack, ABC World
News, December 1, 2008, http://abcnews.go.com/Blotter/
story?id=6368013.
10. Rotella, On the Trail of a Terrorist.
11. Pranab Dhal Samanta, Mumbai Sea Attack Alert Came
Nov. 19, Indian Express, November 30, 2008, http://www.indian
express.com/news/mumbai-sea-attack-alert-came-nov-19/392351.
12. Rotella, On the Trail of a Terrorist.
13. Government of India, Mumbai Terrorist Attacks: Nov.
2629, 2008, Federation of American Scientists website: http://
www.fas.org/irp/eprint/mumbai.pdf.
14. Rotella, On the Trail of a Terrorist.
15. New York Police Department Intelligence Division,
Mumbai Attack Analysis, December 4, 2008, http://publicintel
ligence.net/nypd-law-enforcement-sensitive-mumbai-attackanalysis.
16. Government of India, Mumbai Terrorist Attacks: Nov.
2629, 2008.
17. Ibid.
18. New York Police Department Intelligence Division,
Mumbai Attack Analysis.
19. Government of India, Mumbai Terrorist Attacks: Nov.
2629, 2008.
20. Rabasa et al., The Lessons of Mumbai.
21. Government of India, Mumbai Terrorist Attacks: Nov.
2629, 2008.
22. Ibid.
23. Ibid.
24. New York Police Department Intelligence Division,
Mumbai Attack Analysis.
25. How Mumbai Attacks Unfolded, BBC News, November
30, 2008, http://news.bbc.co.uk/2/hi/south_asia/7757500.stm.
26. Ibid.
27. Rotella, On the Trail of a Terrorist.
28. Government of India, Mumbai Terrorist Attacks: Nov.
2629, 2008.
post.com/wp-dyn/content/article/2008/11/27/AR2008112701128
.html.
67. Sengupta, Dossier Gives Details of Mumbai Attacks.
68. Government of India, Mumbai Terrorist Attacks: Nov.
2629, 2008.
69. After extensive investigative work, journalist Sebastian
Rotella (On the Trail of a Terrorist) concluded that 166 people
were killed and 308 wounded in the attack.
70. Rotella, On the Trail of a Terrorist.
71. Ibid.
72. Ibid.
73. Ajmal Kasab, New York Times, May 4, 2010, http://topics
.nytimes.com/top/reference/timestopics/people/k/ajmal_kasab/
index.html.
74. Rina Chandran, Indian Court Convicts Mumbai Attack
Gunman, Reuters, May 3, 2010, http://www.reuters.com/assets/
print?aid=USTRE6420WU20100503.
Analytic Family
Starbursting
p. 113
Idea Generation
Morphological Analysis
p. 119
Idea Generation
Structured Brainstorming
p. 102
Idea Generation
Indicators
p. 149
183
184 Chapter 15
not try to answer the questions as they are identified; just
focus on generating as many questions as possible. (See
Figure 15.2.)
WHO?
WH
WHEN?
HO
W?
E
ER
?
AT
H
W
Y?
and Impacthave already been identified in the confidential report by the Bahraini government and could be used to
frame the analysis. (See Table 15.5 in the book.) The
counterclaims by the Bahraini opposition and Iran could
also serve as additional alternative expressions of the
dimensions.
Step 2: Create additional dimensions as needed.
Step 3: Consider all the combinations of dimensions
to create a list of possible alternative scenarios. (See
Table 15.6.)
Identifying the main claims, counterclaims, and null
hypothesis are easily accomplished by looking down the
columns:
Bahraini opposition members receiving clandestine
training in Iranian-backed Hezbollah camps with the
purpose of overthrowing the Khalifa monarchy.
Bahraini opposition members receiving clandestine
financial support with the purpose of overthrowing
the Khalifa monarchy.
Bahraini opposition members who are overtly
campaigning for minority Shia rights but are
receiving no support.
No activity.
The table also helps identify several alternatives,
including:
Bahraini opposition members who are unwitting of
financial support that is aimed at overthrowing the
Khalifa monarchy.
Equally interesting is the possibility that unaffiliated
or rogue opposition members are receiving training
in camps but the activity has no impact because the
Bahraini elements lack the organizational structure
Unaffiliated Opposition
No Activity
Activity
Financial Support
No Support
Method
Clandestine
Overt
Unwitting
Impact
No Impact
186 Chapter 15
that would enable them to put the training into
action once they return to Bahrain.
Step 4: Eliminate any combinations that are impossible,
impractical, or undeserving of attention.
Nonsensical combinations should be discardedfor
example, a scenario in which individuals receiving the
training are unwitting of it.
Step 5: Refine the scenarios so that they are clear and
concise.
Bahraini opposition members are receiving
clandestine training in Iranian-backed Hezbollah
camps with the purpose of overthrowing the Khalifa
monarchy.
Bahraini opposition members are receiving
clandestine financial support with the purpose of
overthrowing the Khalifa monarchy.
Bahraini opposition members who are overtly
campaigning for minority Shia rights are receiving
no Iranian support.
Bahraini opposition members are receiving financial
support with the purpose of overthrowing the
Khalifa monarchy but are unwitting of the source of
that funding.
Unaffiliated or rogue opposition members are
receiving clandestine training in camps that has not
yet had an impact in Bahrain.
Analytic Value Added: Which scenarios are most
deserving of attention? Do any assumptions underlie
the scenarios? Certainly, the main claims and counterclaims deserve attention, but equally important in this
case is the possibility that the opposition is unwitting that
it is receiving support from Iran. In this scenario, there is
a possibility that cooptation and influence by Iran are
occurring, but the opposition is not yet aware of that
activity. It also raises the possibility that only select individuals associated with otherwise legitimate Bahraini
opposition groups may be aware of the activity while the
larger organization is not.
Are there any information gaps that affect your ability
to assess the likelihood of a scenario? Information is lacking about the locations of the alleged training camps, the
individuals who have traveled there, or the specifics relating
to alleged financial support such as bank accounts or
amounts of transfers. These gaps limit our ability to assess
the likelihood of several of the scenarios.
Foreign Media
Iranian Regime
Marriage/Relationships
Vulnerability
Iran
Personal Attributes
Money Needs
History of Employment
Language Spoken
Green Revolution
Unemployment
Age
Neda
Criminal Record
Ethnicity
Religion
Beliefs
Narcotic Use/Distribution
Intelligence
Personal Goals
Mentor(s)
Values
Degree of Organization
Associates
Administrative Savvy
Wealth
TV Shows/Foreign Media
Ownership in Bahrain
Anger
Chance
Ownership in Iran
Injustice
Location
Education
Travel
Social Affiliations
Religious Education
Accounting
Social Background
Children
Support in West
Discontent
Family History
TV Shows
Family Ties
Iranian Aggressiveness
188 Chapter 15
TV Shows
Ownership in Iran
Love/Marriage
Foreign Media
Criminal Record
Children
Green Revolution
Family History
Narcotic Use/Distribution
Family Ties
Marriage/Relationships
Beliefs
Personal Attributes
Personal Goals
Opportunity to Be Influenced
Language Spoken
Values
Degree of Organization
Age
Administrative Savvy
Ethnicity
Vulnerabilities
TV Shows/Foreign Media
Religion
Chance
Intelligence
Anger
Injustice
Travel
Discontent
Accounting
Outside Influences
Mentor(s)
Associates
Vulnerability
Education
Money Needs
Foreign Actors
Religious Education
History of Employment
Social Affiliations
Social Background
Iranian Aggressiveness
Wealth
Iranian Regime
Support in West
Ownership in Bahrain
Iran
190 Chapter 15
No demonstrated marital or familial problems
192 Chapter 15
Analytic Value Added: Are the indicators mutually
exclusive and comprehensive? Have a sufficient number
of high-quality indicators been generated for each scenario to enable an effective analysis? Are the indicators
collectible, and if so, what should be the collection priorities? The indicators in this case were generated on the
basis of the dimensions developed in Task 3, and therefore
reflect the range of issues identified in the divergent phase
of Structured Brainstorming. This has resulted in a high
number of indicators per dimension that analysts can reasonably expect to collect. The collection priorities for this
case should focus on using the indicator sets to rule out the
possibility that opposition members are engaged in activities to overthrow the Khalifa regime, rather than ruling in
activity. Once the list has been narrowed, additional analysis and collection can be conducted to review thoroughly
the basis for judgments about activities consistent with one
or more of the scenarios. Some of the most interesting indicators surround the financial dealings of the opposition
groups and members, their social networks, and the content
and quality of their social media activities.
CONCLUSION
The standoff between the government and opposition did
not abate in the months following the arrest of the eight
opposition leaders. In June 2011, King Hamad sought to
deescalate tensions by creating the Bahrain Independent
Commission of Inquiry (BICI). The five-person commissions mandate was to determine whether the events of
February and March 2011 involved violations of international human rights laws and norms and to make recommendations to the government. 1 In a 500-page report
released in November 2011, the commission detailed government abuses and offered recommendations, some of
which the government took steps to implement.2 The commission found that force and firearms were used in an
excessive manner that was, on many occasions, unnecessary, disproportionate, and indiscriminate.3 The report also
documented 35 deaths, 559 allegations of torture, and 1,624
complaints of employment termination as a result of the
uprising in Bahrain.4 By early 2012, several of the boards recommendations had been implemented, including compensating families of deceased protestors and victims of torture,
reviewing convictions, and promising to investigate allegations of torture.5 On 8 January 2012, Bahrains cabinet proposed granting more power to the elected legislature in
order to achieve greater balance between the executive and
the legislative, but no effort was made to increase Shia representation in the political sphere.6
In addition to general recommendations to establish
more independent institutions to investigate and oversee
current and future claims of abuses, the commission offered
specific recommendations to address the following:
The use of force, arrest, treatment of persons in
custody, detention, and prosecution in connection
with the freedom of expression, assembly, and
association.
Demolition of religious structures, termination of
employees of public and private sectors, dismissal of
students, and termination of their scholarships.
Media incitement issues.
Better understanding and appreciation of human
rights, including respect for religious and ethnic
diversities.7
In many respects, however, the commissions recommendations and the governments response were too little
and too late. For example, the government instituted a new
code of conduct calling on police to be respectful of human
rights principles; however, the governments detention of
hundreds of opposition members in the months preceding
and following the commissions report only fueled opposition calls for reforms and sparked additional protests that
were met with government force.8 In addition, the arrest
and sentencing of forty-eight Bahraini doctors and nurses
to five to fifteen years in prison for treating injured protestors fanned the flames of dissent and elicited stern rebukes
from international institutions.9 UN Secretary General Ban
Ki-Moon, through his spokesperson, expressed his deep
concern over the harsh sentences handed down in Bahrain
to civiliansmedical professionals, teachers and othersby
the Bahraini military Court of National Safety, pointing out
that proceedings were conducted under conditions that
raised serious questions of due process irregularities.10 In
the months following the report, clashes between police and
protesters continued, prompting the Office of the U.N. High
Commissioner for Human Rights to issue a statement on
worrying reports about the use of tear gas, rubber bullets,
and birdshot pellets. The OHCHR said reliable sources
indicated that a number of deaths were linked to the use of
tear gas fired by security forces into crowds and called on
the government of Bahrain to investigate the alleged use of
such excessive force.11
NOTES
1. Report of the Bahrain Independent Commission of
Inquiry, November 23, 2011, http://www.bici.org.bh.
2. Background Note: Bahrain, U.S. State Department
Bureau of Near Eastern Affairs, January 13, 2012, http://www.state
.gov/r/pa/ei/bgn/26414.htm.
3. Report of the Bahrain Independent Commission of
Inquiry, 268.
4. Ibid., 219, 282, 331.
5. Information Affairs Authority, Progress Report on the
Implementation of the BICI Recommendations, January 17, 2012.
6. Report of the Bahrain Independent Commission of Inquiry.
7. Ibid.
8. Post BICI Report: A BCHR Report on Human Rights
Violations Since the BICI Report, Bahrain Centre for Human
Analytic Family
Structured Brainstorming
p. 102
Idea Generation
Outside-In Thinking
p. 228
Simple Scenarios
p. 139
195
196 Chapter 16
Task 1.
Conduct a Structured Brainstorming of the factors that will
determine the outcome of the Ukrainian election.
Step 1: Pass out sticky notes and marker-type pens to all
participants. Inform the team that there will be no talking
during the sticky-notes portion of the brainstorming
exercise.
Students will be limited to the case study for this exercise, but it is important to point out that in real-life situations, it is helpful to include in the brainstorming group
both experts on the topic and generalists who can provide
more diverse perspectives. When only those working the
issue are included, often the groups perspective is limited to
Demographic distribution
Yushchenko
Likelihood of fraud
Media
Media coverage
New media
NGOs
Russian meddling
R
ole of external official institutions like OSCE,
Council of Europe
Symonenko
Medvedchuk maneuvering
State-controlled media
Effectiveness of election monitoring
Political demography
Additional compromising information about Kuchma or Yanukovych
New constitutional reform bill
US support for NGOs
Energy interests
198 Chapter 16
See Figure 16.4 for an example of affinity-clustered
results.
Only two clusters are shown in Figure 16.4, but four or
five themes usually emerge from this part of the exercise.
In this case, a notional set of groups might include the
following:
INFLUENCE
MEDIA
media
Ukraine and to what effect?
Russian
State of
energy
Business interests:
Media
Russian
media
interests
covera
business
ge
freedom
Ukrainian
businesspeople are
interests
ian
Russ
s and
s
e
in
in a position to influence the
bus
al
politic ns
ctio
conne
election by providing financial support to the candidates
and enabling access to the
media. Some businesspeople
have withdrawn their support
for Yanukovych and are backing Yushchenko. Which businesspeople are supporting the main candidates, how strong
is their support, and how might their support tip the balance in one direction or the other?
Nongovernmental organizations: NGOs are operating
in Ukraine. To what extent can NGOs organize the kinds of
activities that took place in Georgias Rose Revolution? To
what extent is Kuchma taking preemptive action to prevent
such activities?
Popular sentiment: How does the Ukrainian electorate perceive the candidates and the contest in general?
What are their perceptions of Western or Russian involvement? And what will be their level of voter turnout and
activism?
Analytic Value Added: What key factors will influence the outcome of the election? What gaps deserve
additional attention? The value added by this combination
of Structured Brainstorming and Outside-In Thinking is
not only the list of driving factors but also a clear exposition of why the factors could influence the outcome and
how additional collection can narrow the range of uncertainty by filling important information gaps. This process
can focus information collection tasks on the most meaningful and potentially fruitful avenues of inquiry because
analysts have focused on factors that they have reason to
suspect will influence the outcome and the specific information needs surrounding them. Some gaps are knowable,
and information can be collected. Some of them are not
knowable, but the mere act of considering them helps analysts identify the variables at play and place bounds around
their uncertainty.
Task 2.
Conduct a Simple Scenarios analysis to consider the range
of possible outcomes and driving factors that will shape the
outcome of the Ukrainian election.
Step 1: Clearly define the focal issue and the specific goals
of the Simple Scenarios exercise.
In this case, the task above defines the focal issue, but
students may want to consider whether any other focal
issues warrant further consideration.
Step 2: Make a list of forces, factors, and events that are
likely to influence the future.
Students can draw from the list of factors developed
using Techniques 1 and 2 or brainstorm a list of factors that
would have some effect on the issue being studied.
Step 3: Organize the forces, factors, and events that are
related to each other into five to ten affinity groups that are
expected to be the driving forces in how the focal issue will
evolve.
Again, students can use their previous list and/or tailor
or augment it to include the most relevant grouping of factors. For this case, those notional groups of factors included
the following:
Kuchmas maneuvering.
Expected candidates and their bases of support.
Role of the media.
Russian influence.
US/EU/Western influence.
Business interests.
Nongovernmental organizations.
Popular sentiment.
Step 4: Write a brief description of each or use the
descriptions previously developed.
Kuchmas maneuvering: Kuchma is taking steps to alter
the constitution to deprive the new president of significant
powers. Kuchma has been accused in the past of unscrupulous dealings, raising questions about just how far he will go
to ensure Yanukovychs victory and how effective he might
be in doing so. Would he try to prolong his own rule by
provoking a crisis? Would he take ruthless steps to silence
the opposition? Or would he attempt to divide the opposition by wooing one or more of its significant members away
from Yushchenkos camp?
200 Chapter 16
Expected candidates and their bases of support: How
the candidates conduct their campaigns, including their
ability to garner support from voters and business leaders,
will affect voter turnout and financial support. The degree
of corruption and fraud are key unknowns.
Role of the media: The media are largely controlled by
the government in Ukraine and present few, if any, opposing political viewpoints. The opposition at their February
convention showed a creative use of technology and nontraditional media to broadcast their message. Also, there is
an underlying assumption that control of the media will
only help the incumbent, when it is possible that the lack
of alternative perspectives could encourage an engaged
electorate to seek out nontraditional sources of information. A gap that additional research could fill is the extent
to which the opposition is tapping other forms of communication and, if it is, what these forms of communications
are.
Russian influence: The case narrative highlights strong
motivations to discourage a Yushchenko presidency, but the
case does not identify specifically Russias potential means
for influencing a transition. Russias means of influencing
the outcome and indications that Moscow is exercising
those means are an avenue for further research. If Russia
sees Ukraine as its most important foreign policy issue, how
far will it go to protect its interests in Ukraine?
US/EU/Western influence: The United States and other
Western countries, including international organizations,
Worst Case
Constitutional Coup
Mainline Triumph
of the Oligarchs
Additional Ukraines
Rose Revolution
Viktor Yanukovych
Viktor Yushchenko
Russian Influence
Western Influence
Popular Sentiment
Note: + = strong or positive influence; = weak or negative influence; no entry = blank or no change.
202 Chapter 16
organization in May. In August, key Yushchenko ally Yulia
Tymoshenko dies in a car bombing, and Kuchmas past
involvement in the killing of opposition journalist Gongadze
prompts speculation that his government arranged the
assassination. With US and EU support, the OSCE withdraws its election-monitoring team, declaring that the new
circumstances preclude a free and fair election. Yushchenko
manages to qualify for a runoff election in the first round of
voting on 31 October, but he loses the runoff vote to
Yanukovych. Ukrainian NGOs claim the vote involved massive fraud, but the regime precludes alternative vote count
efforts, and opposition calls for protest spark little action
from the public.
Additional scenario (Ukraines Rose Revolution):
Kuchmas constitutional reform bill falls short of winning a
two-thirds majority in the Rada. Ukraines oligarchs align in
support of the Yanukovych campaign, and Russia intervenes
heavily in support of Yanukovych, fueling a nationalist backlash that benefits the Yushchenko candidacy. It also reinforces the determination of international organizations and
Western-financed NGO groups to organize alternative vote
counts and strict election monitoring. Activists from
Georgias Rose Revolution train their Ukrainian counterparts in civic organization and popular mobilization.
Yushchenko is shut out from the mainstream media, but his
following grows through public appearances and his Internet
presence. Much as in Georgias Rose Revolution, the regime
claims its candidate won the election, but the public protests
against the perception of massive fraud and the government
cannot rely on security forces to stop the demonstrators,
who peacefully take over state television and key ministries
and declare Yushchenko president. Sensing the inevitable,
Yanukovych concedes the election to Yushchenko, and
Kuchma and his key associates flee to Russia.
Step 10: For each scenario, describe the implications for
the decision maker. The implications should be focused on
variables that the United States could influence to shape the
outcome.
Following are some examples:
Best case (democratic transition): US diplomatic
outreach to Russia and a bilateral agreement to
respect the Ukrainian democratic process are key
means of holding Russian influence in abeyance.
Worst case (constitutional coup): The key variable
in this scenario is the vote in the Rada, over which
the United States exerts little influence.
CONCLUSION
Ukraines presidential transition wound up producing what
became known popularly as the Orange Revolution, but
in retrospect it is apparent that this outcome was far from
preordained; several other alternative scenarios came close
to being realized (see Figure 16.5 for a chronology of this
period). Constitutional reform, for example, proved to be a
near miss. On 8 April 2004, Ukraines Rada fell just six
votes short of the two-thirds majority needed to pass
Kuchmas constitutional reform bill.2 Opposition blocs boycotted the vote, and the government failed to garner
enough support from independent deputies to carry the
day. The Rada chair declared the bill dead until sometime
after the presidential elections, and the leaders of pro-government parties in the legislature voted to unite behind
Yanukovychs candidacy.3
The campaign turned out to be a bare-knuckled contest.
The governments intended tactics became clear in the
mayoral election in Mukachevo held in April, when the
regime employed gross falsifications and pure thuggery
at the polling stations to defeat a popular Yushchenko ally,
alarming opposition groups.4 As the presidential campaign
progressed over the summer into the fall, Kuchmas operators pulled out all the stops to bolster Yanukovych, but
many of their tactics proved counterproductive. The government regularly issued so-called temnykyinformal
guidance on coverageto media organizations. Statecontrolled television coverage amounted to little more than
crude propaganda, and the refusal to broadcast
Yushchenko only encouraged larger attendance at his campaign events by voters curious to learn about him. 5
Yushchenkos campaign also faced near-constant harassment. At one point, a truck attempted to force his motorcade from the road, and in September he was taken ill with
a mysterious malady that nearly took his life. Austrian doctors diagnosed the illness as dioxin poisoning; Yushchenko
accused the Kuchma regime of involvement, but the perpetrators were never identified. The poisoning left
Yushchenkos once handsome face badly scarred, but it also
cemented his image as a courageous opponent of the
regimes brutality and redoubled his determination to win
the presidency.6
Like the Kuchma regime, Russia intervened massively in
support of the Yanukovych campaign, but if anything its
efforts backfired. To all appearances, Russian President
Putin made the Ukrainian election a personal mission,
meeting with Kuchma on an almost monthly basis during
204 Chapter 16
the campaign, coming out publicly in favor of Yanukovych
in July, and even campaigning for Yanukovych in Ukraine
on the eve of the election.7 Dozens of Russian political consultants descended upon Ukraine, appearing frequently on
Ukrainian- and Russian-language television shows praising
Yanukovych and criticizing Yushchenko.8 Hundreds of millions of dollars in Russian money poured into Yanukovych
campaign coffers.9 The Kremlins campaign came across as
a transparent attempt to impose its will on Ukraine and may
actually have hurt Yanukovych.10
Arrayed against the Kuchma regime, Russia, and
Yanukovych were Ukraines opposition groups and a range
of NGOs. For several years, the United States, Europe, and
private donors had been funding Ukrainian NGOs
involved in voter education, judicial reform, and election
monitoring, and these groups in turn had developed an
extensive network of local activists and officials trained in
election laws and community organization.11,12 In parallel,
several independent Internet media sites were established,
including the cyber-newspaper Ukrainska Pravda, which
became a key source of news on the Yushchenko campaign,
and the website Maidan, which served as a virtual civic
organization in cyberspace for regime opponents.13 In late
March 2004, a Ukrainian student organization named Pora
(Its Time) emerged, modeled on groups that had helped
to topple presidents in Serbia and Georgia; it provided
both formal and informal support for the Yushchenko
campaign, despite harassment by the regime that Pora
activists sometimes captured on cell-phone cameras.14 The
United States adopted a neutral stance toward the candidates but pressed the Kuchma government to ensure a free
and fair electoral process.15 In May 2004, then Deputy
Assistant Secretary of State Steven Pifer told the House
International Relations Committees Subcommittee on
Europe that
the US Government does not back any particular can
didate in the election; our interest is in a free and fair
electoral process that lets the Ukrainian people demo
cratically choose their next president. We would be
prepared to work closely and eagerly with whomever
emerges as president as the result of such a process.16
Events
2004
18 March
Late March
1 April
Viktor Pinchuk and George Soros announce plans to combine philanthropic efforts by forming legal aid society.i
16 April
Viktor Medvedchuk meets Russian President Vladimir Putin at the Kremlin. Putin supports will of people but says he prefers
continuity in the bilateral relationship.ii
23 April
2324 May
3 July
26 July
5 September
Viktor Yushchenko falls ill after dinner with the head of the Ukrainian Intelligence Service.
24 September
Yanukovych struck in chest with an egg, hospitalized for several hours, and released.
1516 October
Pora youth organization offices raided by government special police.iv Kuchma meets with Putin in Sochi.
20 October
23 October
24 October
A group of 100 journalists marches in support of Channel 5. Separately, a bottle of combustible liquid is hurled into Yushchenkos
chief of staffs car in Kyiv. The Ukrainian CEC votes unanimously to establish forty-one exceptional voting sites in the Russian
Federation.vi
25 October
Pora announces a wave of student protests and actions for 2530 October in response to alleged government intimidation.
26 October
28 October
Supreme Court overturns CEC decision on exceptional voting sites in the Russian Federation.vii
31 October
First round of presidential election held. Voting in the presidential election gives Yushchenko a small lead against Yanukovych
and triggers a second-round vote. OSCE says the vote fails to meet a considerable number of Ukraines OSCE commitments.
21 November
Second round runoff presidential election held. It triggers a flurry of fraud accusations.
22 November
The Central Electoral Commission declares Yanukovych the winner, and Yushchenko supporters take to the streets.
25 November
Supreme Court suspends publication of the voting results by the CEC following a complaint by Yushchenko.
26 November
1 December
Yushchenko lifts a blockade on government buildings and encourages his supporters to remain on the streets.
3 December
Supreme Court annuls results of second round, paving the way for new elections.
11 December
26 December
2005
11 January
CEC announces the election results and names Yushchenko the winner.
20 January
Yanukovych concedes.
23 January
i. George Soros, Viktor Pinchuk to Create Legal Aid Foundation in Ukraine, US-Ukraine Business Council. April 1, 2004, http://www.usubc.org/AUR/
aur4052.php.
ii. Russia Watches Ukraine Election, Ukraine Weekly, May 30, 2004, http://www.ukrweekly.com/.
iii. Putin: Broadcasting Not an Issue, Ukraine Weekly, May 9, 2004, http://www.ukrweekly.com/.
iv. Andrew Wilson, Ukraines Orange Revolution (New Haven, CT: Yale University Press, 2006), 76.
v. Ukraine TV Station on Hunger Strike Ahead of Poll, Reuters, October 27, 2004.
vi. Organization for Security and Cooperation in Europe (OSCE), Ukraine Presidential Election OSCE/ODIHR Election Observation Mission Final Report, May 11, 2005.
vii. Ibid.
206 Chapter 16
self-policing and security, providing no pretexts for a government crackdown.26
The Kuchma regime scrambled to regain control of
events, but it soon became clear that the regimes options for
dealing with the protests were sharply constrained.
Dnipropetrovsk clan leader Viktor Pinchuk defected from
the ranks of Yanukovych supporters, dealing a critical blow
to the regimes hopes.
Within Ukraine, one force after the other abandoned the
authorities. One early group of official defectors was
Ukrainian diplomats. The armed forces split. Two former
SBU (Ukraines intelligence service) generals spoke in favor
of the opposition in Maidan square on 25 November, and
the SBU leadership seemed to follow. The same day, the
commander of Ukraines Western Military Command
declared that his troops would not be used against the
nation, indicating that the military was regionally divided, as
were the civilian police. The regime could deploy only select
special forces of the Ministry of Interior for a crackdown.27
Sensing the inevitable, Kuchma entered negotiations
with key parties to reach a settlement. The presidents of
Poland and Lithuania joined as mediators, and Yanukovych
invited Russias Duma Speaker as well. To facilitate a deal,
Yushchenko agreed to a reduction of presidential power,
transferring some key authorities to the Rada.28 Both sides
NOTES
1. Richards J. Heuer Jr. and Randolph H. Pherson, Structured
Analytic Techniques for Intelligence Analysis, 2nd ed. (Washington,
DC: CQ Press, 2015, 149).
2. Roman Woronowycz, Verkhovna Rada Fails, by 6 Votes,
to Pass Constitutional Amendments, Ukraine Weekly, April 11,
2004, http://www.scribd.com/doc/12815581/The-UkrainianWeekly-200415.
3. Roman Woronowycz, Majority C oalition Taps
Yanukovych as Presidential Candidate, Ukrainian Weekly, April
18, 2004, http://www.scribd.com/doc/12815982/The-UkrainianWeekly-200416.
4. Nadia Diuk, The Triumph of Civil Society, in Revolution
in Orange: The Origins of Ukraines Democratic Breakthrough,
Anders slund and Michael McFaul, eds. (Washington, DC:
Carnegie Endowment for International Peace, 2006), 78.
5. Anders, slund, How Ukraine Became a Market Economy
and Democracy (Washington, DC: Peterson Institute for
International Economics, 2009), 18084.
6. Adrian Karatnycky, Ukraines Orange Revolution, Foreign
Affairs, MarchApril 2005, http://www.foreignaffairs.com/
articles/60620/adrian-karatnycky/ukraines-orange-revolution.
7. Andrew Wilson, Ukraines Orange Revolution (New Haven,
CT: Yale University Press, 2005), 8695.
Analytic Family
p. 304
Decision Support
Decision Matrix
p. 297
Decision Support
Pros-Cons-Faults-and-Fixes
p. 300
Decision Support
Task 1.
Conduct a Force Field Analysis of the factors for and against
additional violence directed at US interests in Belgrade.
Step 1: Define the problem, goal, or change clearly and
concisely.
In this case, the initial problem at hand by Tuesday,
19 February, is to determine whether the violence against US
and other Western interests in Belgrade will increase and, if so,
what the US embassy should do to maintain building security,
protect its personnel, and advance its policy objectives. A Force
Field Analysis should therefore focus on the forces driving and
constraining additional violence against the US embassy.
Step 2: Use a form of brainstorming to identify the main
factors that will influence the issue.
Using Structured Brainstorming,1 students should generate
an exhaustive list of forces, factors, and issues that will affect
the chances of more violence. Encourage students to jumpstart
their brainstorming by using STEEP +2 (Social, Technological,
Economic, Environmental, Political plus Military and
Psychological). The process should prompt a discussion of
information gaps and assumptions that require further
research or require refinement of the forces and/or groupings.
Step 3: Make one list showing the strongest forces for and
against additional violence.
For this case, some of the key forces for additional violence
include the following:
Formal US and European recognition of Kosovos
unilateral declaration of independence.
Serbian officials strong anti-Western rhetoric.
209
210 Chapter 17
Reports of a secret action plan that includes a
provision for Serbs to reject Kosovos declaration of
independence.
The failure of Serbian riot police to avert damage to
Western assets on Sunday and Monday.
The opportunity for splinter groups to use the
government-sponsored peaceful demonstration
planned for Thursday evening to perpetrate violence.
Forces against violence include these:
Antiriot police actively attempted to repel attackers
on Sunday and Monday.
Serbian officials have urged calm and called for a
peaceful demonstration on Thursday.
Serbias EU aspirations should constrain any
government impulse to endorse or facilitate violence
or military action.
The vast majority of the demonstrators on Sunday
were peaceful.
Step 4: Array the lists in a table such as Table 17.5.
Step 5: Assign a value to each factor to indicate its
strength. Assign the weakest intensity scores a value of
1 and the strongest a value of 5. The same intensity score
Score
Total: 20
Total: 15
212 Chapter 17
3. Pursue US policy position vis--vis Kosovo and
Serbia (i.e., stand by recognition of Kosovo).
Step 6: Work across the matrix one row at a time to evaluate the relative ability of each of the options to satisfy each
criterion. To do so, assign 10 points to each row and divide
these points according to an assessment of the ability of
each option to satisfy the selection criteria.
For example, neither withdrawal of the ambassador nor
closing to the public directly protects US personnel if an
attack occurs and the majority of personnel are still in the
embassy. Administrative closure and total evacuation, however, both have a chance of satisfying this criterion by
removing personnel from the premises.
Step 7: Assess the strength of each option against each criterion by multiplying the criterion weight by the assigned
strength of the option from Step 6. For example, criterion 1
weight option 1 points = score. For ease of calculation, simply use the whole number weight rather than a percentage.
Withdraw Ambassador
Close to Public
Weighted
Value
% Weight
(W)
Value
(V)
Weighted
Value
(W x V)
Value
(V)
Protect US embassy
(physical buildings,
information).
20
80
Protect US persons
(staff, dependents,
foreign service
nationals).
35
Pursue US policy
position vis--vis
Kosovo and Serbia.
35
70
3.5
Minimize economic
costs to US embassy.
10
50
Selection Criteria
Totals
(100%)
120
(W x V)
122.5
40
242.5
Administrative Closure
Weighted
Value
(W x V)
Value
(V)
Weighted
Value
(W x V)
80
40
175
175
3.5
122.5
35
10
Value
(V)
387.5
250
Task 4.
214 Chapter 17
If the closure is prolonged, it will reduce productivity,
increase costs, and still put the core team and
Marines at risk. An extended closure could also
project an image of weakness on the part of the
United States.
Step 4: Determine Fixes to neutralize as many Cons as
possible. To do so, propose a modification of the Con that
would significantly lower its risk of being a problem, identify a preventive measure that would significantly reduce
the chances of the Con being a problem, conduct contingency planning that includes a change of course if certain
indic ators are observed, or identify a need for further
research or to collect information to confirm or refute the
assumption that the Con is a problem.
Private diplomacy: Reach out diplomatically in
private to the Serbians, thank them for the assistance
on Sunday, and request a discussion of strategy in
advance of Thursdays rally. Couple this outreach
with public statements of tempered appreciation for
Serb police assistance on Sunday and the ongoing
dialogue with the Serb government.
Public diplomacy: Publicize the Serbian
governments responsiveness to Sundays attacks and
the ongoing dialogue between the US and Serbian
governments as a deterrent to would-be vandals and
a message to Serbia that the United States expects
proactive Serbian policing.
Better safe than sorry: Find a middle approach that
protects US persons, policy, and information in
the embassy structure while minimizing economic
impact. Develop a plan in concert with the US
Marines and other possible stakeholders to protect
any sensitive information as well as an evacuation
plan.
Step 5: Fault the Pros. Identify a reason why the Pro would
not work or the benefit would not be received, pinpoint an
undesirable side effect that might accompany the benefit, or
note a need for further research to confirm or refute the
assumption that the Pro will work or be beneficial.
The Serbians may not have the ability to manage an
even larger rally than Sundays, which could put US
interests at risk. Given reports of a secret plan and
the difficulty that Serb police had dispelling attackers
on Sunday, it may not be safe to assume that the
Serbian government can manage the situation should
another round of riots break out.
Pros
Cons
Fixes
If the closure is
prolonged, it will
reduce productivity,
increase costs, and still
put the core team and
Marines at risk.
CONCLUSION
In the face of growing fears about more looting and violence, on Wednesday, 20 February 2008, the United States
announced an administrative closure of the US Chancery in
Belgrade beginning at noon on Thursday, 21 February
216 Chapter 17
6,000 protesters broke away from the crowd of peaceful protesters and converged on the US and other pro-Kosovo
embassies. At the time of the attack, press reports indicate
that there was either no police presence at the US embassy
building or that police withdrew when the crowd
approached.8,9,10,11 The attackers tore metal grills from windows, ripped the US flag from its pole, and broke a handrail
off the entrance and used it to smash into the Chancery.
Once inside, they threw furniture from the windows and set
fire to the building, while the crowd outside shouted
Serbia, Serbia.12,13,14,15 One protester died in the blaze.16
According to a firsthand account by Master Sergeant John
Finnegan of the Marine Security Guard Detachment,
There were too many [protesters] for the police to handle
and a whole lot more were on the way....The police
couldnt help us out and [rioters] had free access to the
embassy. We made the call to pull everybody back. We got
everybody to a safe area and hunkered down.17
It reportedly took police between thirty and forty-five minutes to appear at the scene, and firefighters arrived at about
the same time to put out the blaze. The protest lasted about
two hours as police fought to disperse the crowd and secure
the building using tear gas and armored cars.18,19,20 The protesters also attacked the embassies of Bosnia-Herzegovina,
Canada, Croatia, Germany, Slovenia, Turkey, and the United
Kingdom.21,22 In all, over 150 people were injured, nearly were
200 arrested, and 90 shops were ransacked.23
After the attack, the United States lodged a formal protest
with the Serbian government, citing Serbias Vienna
Convention obligations. The White House spokesperson said
the Chancery had been attacked by thugs and that Serbian
police had not done enough to stop them.24 State Department
spokesperson Sean McCormack indicated that there was not
adequate security, either in numbers or capability, to prevent
this breach of our embassy compound.25 He noted, however,
that the protesters did not breach the so-called hard line,
which is the secure area of the Chancery. 26
In comments to the US Senate Armed Services
Committee, Director of National Intelligence Mike
McConnell said, We have good information that when the
US Embassy and the British Embassy and others were
attacked, a decision was taken by the government of Serbia
actually to pull the police back and allow them to be
attacked, burn the embassy and conduct the violence they
conducted.27 A spokesperson for McConnell later clarified
that the statement was based in part on eyewitness
accounts and that there was no final conclusion or determination on this point, although he added, Im not going
KEY TAKEAWAY
In time-sensitive situations, there is often a tendency
to allow the pressure of the moment to drive analysis
toward the most obvious or convenient course of
action. In this case, a decision merely to close the
facility to the publicas several other Western
countries chose to docould have put more lives
at risk if more than just the core team were in the
building at the time of the attack. Decision Support
techniques can slow down cognitive momentum
in highly charged situations so that analysts and
decision makers can fully consider the forces, factors,
options, and angles that will shape the best decision.
NOTES
1. Richards J. Heuer Jr. and Randolph H. Pherson, Structured
Analytic Techniques for Intelligence Analysis, 2nd ed. (Washington,
DC: CQ Press, 2015, 102).
2. Charlie Coon and Kent Harris, Marines at Embassy in
Belgrade Hunker Down, Wait Out Crisis, Stars and Stripes,
February 23, 2008, http://www.stripes.com/news/marinesatembassy-in-belgrade-hunker-down-wait-out-chaos-1.75379.
3. US Embassy in Belgrade Attacked, BBC, February 22,
2008, http://news.bbc.co.uk/2/hi/7256158.stm.
4. Walter Pincus, Serbia Withdrew Police, Intelligence Chief
Says, Washington Post, February 28, 2008, http://www
.washingtonpost.com/wp-dyn/content/article/2008/02/27/
AR2008022703383.html.
5. US Embassy in Belgrade Attacked, BBC.
6. Over 150 Injured in Belgrade Riots, RIA Novosti,
February 22, 2008, http://en.rian.ru/world/20080222/99859211
.html.
7. Ibid.
8. Ibid.
9. Pincus, Serbia Withdrew Police, Intelligence Chief Says.
10. Rioter Dies in Burning Embassy as Serbs Take to Streets
over Kosovo, Times (London), February 22, 2008, http://www
.timesonline.co.uk/tol/news/world/europe/article3413753.ece.
11. State Department Briefs Press on Situation at US Embassy
Belgrade, Federal News Service, February 21, 2009.
12. US Embassy in Belgrade Attacked, BBC.
13. Pincus, Serbia Withdrew Police, Intelligence Chief Says.
14. Rioter Dies in Burning Embassy, Times.
15. State Department Briefs Press on Situation at US Embassy
Belgrade, Federal News Service.
16. Rioter Dies in Burning Embassy, Times.
17. Coon and Harris, Marines at Embassy in Belgrade
Hunker Down, Wait Out Crisis.
18. Rioter Dies in Burning Embassy, Times.
19. US Embassy in Belgrade Attacked, BBC.
20. State Department Briefs Press on Situation at US Embassy
Belgrade, Federal News Service.