Juergen Weichselgartner’s 2001 paper entitled “Disaster Mitigation: The Concept of Vulnerability Revisited” (Disaster Prevention and Management, Vol. 10, No. 2, pp. 85-94, doi:10.1108/09653560110388609) provided a nice summary of alternative definitions for the word “vulnerability” gleaned from a variety of academic publications (copied below; see original paper for citations).

• Gabor and Griffith (1980) Vulnerability is the threat (to hazardous materials) to which people are exposed (including chemical agents and the ecological situation of the communities and their level of emergency preparedness). Vulnerability is the risk context.
• Timmerman (1981) Vulnerability is the degree to which a system acts adversely to the occurrence of a hazardous event. The degree and quality of the adverse reaction are conditioned by a system’s resilience (a measure of the system’s capacity to absorb and recover from the event)
• UNDRO (1982) Vulnerability is the degree of the loss to a given element or set of elements at risk resulting from the occurrence of a natural phenomenon of a given magnitude
• Petak and Atkisson (1982) The vulnerability element of the risk analysis involved the development of a computer-based exposure model for each hazard and appropriate damage algorithms related to various types of buildings
• Susman et al. (1983) Vulnerability is the degree to which different classes of society are differentially at risk
• Kates (1985) Vulnerability is the “capacity to suffer harm and react adversely”
• Pijawka and Radwan (1985) Vulnerability is the threat or interaction between risk and preparedness. It is the degree to which hazardous materials threaten a particular population (risk) and the capacity of the community to reduce the risk or adverse consequences of hazardous materials releases
• Bogard (1989) Vulnerability is operationally defined as the inability to take effective measures to insure against losses. When applied to individuals, vulnerability is a consequence of the impossibility or improbability of effective mitigation and is a function of our ability to detect hazards
• Mitchell (1989) Vulnerability is the potential for loss
• Liverman (1990) Distinguishes between vulnerability as a biophysical condition and vulnerability as defined by political, social and economic conditions of society. She argues for vulnerability in geographic space (where vulnerable people and places are located) and vulnerability in social space (who in that place is vulnerable)
• Downing (1991) Vulnerability has three connotations: it refers to a consequence (e.g. famine) rather than a cause (e.g. drought); it implies an adverse consequence (e.g., maize yields are sensitive to drought; households are vulnerable to hunger); and it is a relative term that differentiates among socioeconomic groups or regions, rather than an absolute measure or deprivation
• UNDRO (1991) Vulnerability is the degree of the loss to a given element or set of elements at risk resulting from the occurrence of a natural phenomenon of a given magnitude and expressed on a scale from 0 (no damage) to 1 (total loss). In lay terms, it means the degree to which individual, family, community, class or region is at risk from suffering a sudden and serious misfortune
  following an extreme natural event
• Dow (1992) Vulnerability is the differential capacity of groups and individuals to deal with hazards, based on their positions within physical and social worlds
• Smith (1992) Human sensitivity to environmental hazards represents a combination of physical exposure and human vulnerability ± the breadth of social and economic tolerance available at the same site
• Alexander (1993) Human vulnerability is function of the costs and benefits of inhabiting areas at risk from natural disaster
• Cutter (1993) Vulnerability is the likelihood that an individual or group will be exposed to and adversely affected by a hazard. It is the interaction of the hazard of place (risk and mitigation) with the social profile of communities
• Watts and Bohle (1993) Vulnerability is defined in terms of exposure, capacity and potentiality. Accordingly, the prescriptive and normative response to vulnerability is to reduce exposure, enhance coping capacity, strengthen recovery potential and bolster damage control (i.e., minimize destructive consequences) via private and public means
• Blaikie et al. (1994) By vulnerability we mean the characteristics of a person or a group in terms of their capacity to anticipate, cope with, resist and recover from the impact of a natural hazard. It involves a combination of factors that determine the degree to which someone’s life and livelihood are put at risk by a discrete and identifiable event in nature or in society
• Green et al. (1994) Vulnerability to flood disruption is a product of dependence (the degree to which an activity requires a particular good as an input to function normally), transferability (the ability of an activity to respond to a disruptive threat by overcoming dependence either by deferring the activity in time, or by relocation, or by using substitutes), and susceptibility (the probability and extent
  to which the physical presence of flood water will affect inputs or outputs of an activity)
• Bohle et al. (1994) Vulnerability is best defined as an aggregate measure of human welfare that integrates environmental, social, economic and political exposure to a range of potential harmful perturbations. Vulnerability is a multilayered and multidimensional social space defined by the determinate, political, economic and institutional capabilities of people in specific places at specific times
• Dow and Downing (1995) Vulnerability is the differential susceptibility of circumstances contributing to vulnerability. Biophysical, demographic, economic, social and technological factors such as population ages, economic dependency, racism and age of infrastructure are some factors which have been examined in association with natural hazard
• Gilard and Givone (1997) Vulnerability represents the sensitivity of land use to the hazard phenomenon
• Comfort, L. et al. (1999) Vulnerability are those circumstances that place people at risk while reducing their means of response or denying them available protection
• Weichselgartner and Bertens (2000) By vulnerability we mean the condition of a given area with respect to hazard, exposure, preparedness, prevention, and response characteristics to cope with specific natural hazards. It is a measure of capability of this set of elements to withstand events of a certain physical character

Of course, this list is by no means complete; in fact, the definitions from obvious sources such as Webster’s dictionary, Department of Defense doctrine, and a host of other papers were not included.  I leave it to the readers of this blog to discover alternative definitions that are most suited for his or her particular application.  But if one was looking for a really short definition of vulnerability to sum up everything above, consider the following two (my preferences):

Vulnerability is the manifestation of the inherent states of a system that render is susceptible to harm or loss (a paraphrased definition of the notion of vulnerability offered by Prof. Yacov Haimes at the University of Virginia)

The vulnerability of an entity to realizing a specified adverse outcome following the occurrence of a particular triggering or initiating event is measured as the conditional probability of the outcome given the triggering event has occurred (an expanded version of the definition I offer in my SRA 311 class at Penn State)

Send post as PDF to

I just got word of a report summarizing a recent workshop held in McLean, VA focused on the “simple” question “why have we not been attacked again” since 9/11? The workshop was sponsored by the Defense Threat Reduction Agency (DTRA) in cooperation with Science Applications International Corporation (SAIC).  The cover page of the report is shown in the Scribd window below.  Per the dissemination control markings, this product is available to the public release.

Basically, this report examines a total of 29 well-defined hypotheses that address the question at issue, namely reasons for the non-occurrence of another attack (a challenging question indeed).  These hypotheses were the compiled as part of an extensive literature review centered on publications that offered reasons why he haven’t experienced a successful attack.  The fact that cells were foiled, according to the authors, was taken to mean that attacks were attempted, but were unsuccessful (a summary of failed attacks is provided in Appendix C of the report).  Using the information collected during this literature review, a comprehensive list of critical assumptions, supporting evidence and contradictory evidence was compiled for each hypothesis. The goal for the workshop was to identify the likely reasons for non-occurrence of an attack based on all available evidence and assumptions combined with the opinions of subject matter experts.

The 29 hypotheses were sorted into four bins as follows, two for each of capabilities and motivations (i.e., the two variables that comprise the DoD definition of threat). Note that these hypotheses are NOT mutually exclusive nor are they collectively, and as stated in the report, it is quite possible that one or more (or perhaps none) of these hypothesis contribute to the fact that the US hasn’t experienced a successful attack in the last 7 years.

Bin I: US and Allied Counterterrorism Efforts (Capabilities part of Threat)

• Hypothesis A: US homeland security efforts
• Hypothesis B: US and allied counterterrorism operations
• Hypothesis C: The wars in Iraq and Afghanistan have drawn jihadists away
• Hypothesis D: Reduced state support for terrorism
• Hypothesis E: Crackdowns on private financing of terrorism since 9/11

Bin II: Terrorist Attack Capabilities (Capabilities part of Threat)

• Hypothesis F: Terrorist threat has been massively exaggerated
• Hypothesis G: Time is required to rebuild al-Qaeda’s capabilities
• Hypothesis H: Al-Qaeda is waiting to acquire a CBRN capability
• Hypothesis I: The assimilation of US muslims
• Hypothesis J: A lull is occurring between Iraq and the next generation of al-Qaeda
• Hypothesis K: Non-Salafist groups have lacked the capability

Bin III: Another Attack Ill-Advised (Motivations part of Threat)

• Hypothesis L: Al-Qaeda’s next attack must surpass 9/11
• Hypothesis M: 9/11 was a strategic miscalculation
• Hypothesis N: Al-Qaeda is safeguarding its sanctuary in Pakistan
• Hypothesis O: Striking the US homeland again could rally support for America
• Hypothesis P: Al-Qaeda has become more sensitive to killing American civilians
• Hypothesis Q: Al-Qaeda is warning the US of its intent to attack
• Hypothesis R: Al-Qaeda needs success, resulting in conservative planning
• Hypothesis S: Domestic extremist organizations have lacked the motivation
• Hypothesis T: “Lone Wolf” terrorists have lacked the motivation
• Hypothesis U: Hezbollah has been restrained by Iran and Syria

Bin IV: Other Attack Priorities (Motivations part of Threat)

• Hypothesis V: Opportunities in Iraq have diverted jihadist resources
• Hypothesis W: Al-Qaeda has shifted its focus to Europe
• Hypothesis X: Al-Qaeda’s focus has returned to toppling Middle Eastern regimes
• Hypothesis Y: Regional groups are focusing on regional targets
• Hypothesis Z: Al-Qaeda’s goal is to “bleed” the United States dry economically
• Hypothesis AA: 9/11 was meant to be a one-time attack
• Hypothesis BB: Al-Qaeda is focused on preventing Shia ascendancy
• Hypothesis CC: Non-Salafist groups have lacked the motivation to attack

Methodology

This information on each hypothesis, to include a full description, critical assumptions, supporting and contradictory evidence, was then fed to one of three working groups comprised of national security practitioners (see Appendix A for a full list).  The goal of each working group was to walk through the available information and assumptions, discuss it amongst the group, and then each expert would render an independent judgment on the “likelihood that the hypothesis is valid” and the “confidence in the assessment given the quality of available evidence, knowledge, experience, …”  The aggregate results for each hypothesis produced a matrix as shown below given a five-tier likelihood scale and a three-tier confidence scale (don’t be fooled by the placement of the X’s along the Y-axis - there is no meaning to the relative position of an X in the box.  The editors simply forgot to center them vertically in the box.).

Several other hypotheses were developed during the meeting as follows, though none were assessed for validity (because no evidence was provided, I suppose).  I am suprised, actually, that these seemingly obvious hypotheses didn’t make the original list.

• Al-Qaeda is still coasting on 9/11
• Terrorists are simply waiting for the right conditions to attack
• The US responses to another attack is too uncertain to jeopardize current successes

For those familiar with the analytic techniques taught by Professor Frank Hughes of the National Defense Intelligence College, this exercise is essentially an implementation of Chamberlin’s Method of Multiple Working Hypotheses (MMWH) (different from the Analysis of Competing Hypotheses, or ACH).  Basically, the available evidence is weighed against each hypothesis in isolation to determine a subjective level of internal support (e.g., internal = human comfort level with asserting the hypothesis as true). No consideration is given to the complementary hypotheses, nor are two or more hypotheses considered in tandem to assess synergies, subadditivities, or independence. In practice, though, it is often very difficult to look at anything in isolation, so I suspect the judgments were shaped, perhaps only in a small way, by how each expert viewed the alternatives.  Unfortunately, while this is good for ACH, it isn’t necessarily good for MMWH.

Results of this Analysis

Despite their apparent lack of concern for page length, unfortunately the authors did not provide a single page summary of the results.  Rather, the authors simply summarized the results up front (bottom line up front, or BLUF style) and provided the raw opinion matrices (such as shown above) for each individual hypothesis in Appendix B.  The outcomes from this study were expressed in terms of the “most compelling hypotheses” (A, B, I, L, S, V, W, Y, CC) and “unpersuasive hypotheses” (F, P). (I am not sure I like the word persuasive, as it implies that the analysis must be good enough to persuade someone to see past their preinclinations, or rather, their anchors).

Additionally, the analysis did summarize areas of uncertainty that might shape future analytic efforts aimed at better understanding Al-Qaeda (pp. 27-28).  In addition, the report offers several “independent counterterrorism strategies” as explained on pages 29-30.

Critique on this Study

Based on my understanding and experience as an intelligence community methodologist, I feel compelled to offer the following critiques.

• The opinion matrices are very confusing to me, a person who has thought extensively about confidence and likeliness and many other uncertainty analysis issues.  Typically, one speaks of likelihood (or what I refer to as likeliness) in the context of uncertainty about future events (what will happen?), not for questions of fact (what happened? why?).  In general, it is sufficient to ask the simple question “is hypothesis xxx a reason for why we haven’t had another attack?”  The answer then is YES or NO (as it should be since it is a question of fact).  The confidence level assigned to this judgment then discounts the opinion in accordance with how much information, background knowledge, etc. was available to support this judgment.  For example, if you believe the answer is YES, then the choice of confidence level places the probability that you are right somewhere between 50% and 100% (where obviously words with higher (lower) intent place you closer to 100%(50%)).  Using two scales basically asks the experts to make as assessment of confidence on their assessment of uncertainty, or as the information science people like to call it, “second-order” uncertainty.  For questions of future events, this is ok since your judgment (with confidence) expresses a probability distribution over alternative futures.  But for questions of fact where the answer is YES or NO, it makes no sense to say “even chance YES” with “medium confidence” (basically, this statement says the expert doesn’t know with less than perfect confidence; does this mean the residual confidence is applied to something other than “even chance”?).

• No guidance was described on how the experts arrived at judgments of confidence, but rather the facilitators left the details of confidence assessment open to individual interpretation.  (Actually, the first paragraph on page 148 described an “intellectual dilemma” concerning confidence as it related to validity).  This individual interpretation problem is such a no-no that government explicitly addressed it in the Intelligence Reform and Prevention of Terrorism Act (IRPTA), subsequently followed up by the Director of National Intelligence in Intelligence Community Directive 203 (ICD 203).  That is, analysts must “properly caveat and express uncertainties or confidence in analytic judgments.”  Subjective confidence judgments, as the psychologists call them, are extremely sensitive to bias, mood, stress, and a variety of other factors separate from sound reasoning.  In fact, much of the intelligence community’s training efforts are focused on providing a variety of tools and techniques aimed at mitigating the effects of irrelevant (non-analytic) contributors to confidence.  I, personally, spent my entire time at DIA constructing alternative methods for expressing analytic confidence that strived, to the maximum extent possible, to remove all subjectivity out of confidence assessments.  According to my scheme (which I am writing up now), I won’t believe a single thing you say unless you show in writing your reasoning and source analysis.  Some of Kris Wheaton’s (Intelligence Studies Program at Mercyhurst College) students also studied this problem; one even went so far as to construct a tool to help in this area (read about it here or download the thesis).  But despite all this, the DTRA study resorted to the old confusing way of doing business <heavy sigh>.  But then again, DTRA is not part of the intelligence community, so ICD 203 and IRPTA I suppose does not apply to them.

• While the experts were treated to a lot of information and assumptions all with citations to some published document, were the experts advised to consider the quality of the underlying sources?  Or were the sources vetted using some scheme to determine whether they should be included as part of a hypothesis write-up?  This is unclear to me, but very important as I would not want to trust any analysis that hasn’t at least performed some sort of source analysis (and I don’t want to hear that an article must be good since it was published in The Guardian).

• What I find funny is that despite some of the hypotheses judged by a majority as “almost certainly” valid with “high confidence,” the authors (perhaps guided by the experts) hedge the analysis by claiming that “the most unassailable conclusion of the study is that we simply do not know why the United States has not been successfully attacked again” (p. 26).  Well of course we do not know, but the way this statement was phrased suggests to me that while the experts found the exercise less than perfectly credible, perhaps because things weren’t clear (e.g., confidence), the structure was too limiting (e.g., you can offer hypotheses, but we won’t assess them), or time was too short (e.g., too much to do, so little time).  Or, perhaps this hedge isn’t a hedge at all, but an otherwise obvious analytical caveat explaining to readers that we can’t ever really know ground truth, so regardless of what we say with high or low confidence, we will never know with complete certainty why there hasn’t been another attack.

Final Thoughts

In general, this report was very interesting to read and actually gave me a lot to think about, both conceptually and methodologically.  The most interesting parts of this report are Appendix B (where the methodology is explained), the Introduction chapter that explains the conference structure and key findings, and the immense detail provided for each of the 29 hypotheses.  Actually, the quality of the information supporting each hypothesis is such that I can strip out the materials from 4 or 5 hypotheses and use it as a basis for an in-class ACH (or MMH) exercise.  I plan to do this sometime this week - should be fun.

Send post as PDF to