Each semester when the schedule of classes are posted, I often find myself perusing the listing to pick out those courses offered at the University Park campus of Penn State that seem like they would be helpful for improving my understanding of  risk and risk analysis.  I don’t actually plan on taking any of these.  But so far I picked out the following from the undergraduate (and graduate) course catalog:

• B A 497: Risk and Decisions (3): Conceptualizing decisions involving risk, analyzing choices, estimating the risk, and communicating the analysis
• CMPSC 443 Introduction to Computer and Network Security (3): Introduction to theory and practice of computer security with an emphasis on Internet and operating system applications [Sp09, TR 1615-1730]
• EARTH 101: Natural Disasters: Hollywood vs. Reality (3): Analysis of the causes and consequences of natural disasters; comparison of popular media portrayal of disasters with perspective from scientific research [Sp09, TR 945-1100]
• ENNEC 473: Risk Management in Energy Industries (3): Analysis of strategies for mitigating business risk from market, atmospheric, geophysical uncertainties including the use of energy/mineral commodity futures/options, weather derivatives, and insurance [Sp09, TR 945-1100]
• FIN 413: Risk Management and Financial Institutions (3): Measuring and managing risk faced by financial institutions.
• GEOSC 402Y: Natural Hazards (3): Case studies of the causes and consequences of natural disasters; analysis of disaster impact in different economic, cultural, and social conditions [Sp09, MWF 1010-1100]
• HLS 410: Public Health Preparedness for Disaster and Terrorist Emergencies (3): Analyzes the history of terrorism and explores the preparation and response to specific terrorist threats, natural disasters, and conventional catastrophes.
• I E 454: Applied Decision Analysis (3): Theory and practice of decision analysis applied to engineering problems.
• I H S 470: Analytical Methods for System Safety (3): Quantitative and qualitative methods of system safety of analysis are covered; issues in risk assessment, acceptance, analysis, and communication, as well as accident cost analysis and cost-benefit analysis are included.
• INS 301: Risk and Insurance (3): Introduction to the principles and methods of handling business and personal risks; emphasis on insurance techniques.
• INS 405: Corporate Risk Management (3): Insurance management for corporate organizations; self-insurance, risk transfer, and other alternatives to insurance.
• INS 575: Risk Management (2): Develop an understanding of the risks facing corporations and the methods available to deal with those risks.
• IST 564: Crisis, Disaster and Risk Management (3): Examines the fundamental elements of crisis, disaster, risk and emergency management. Emphasis is placed on the use of analytic methods and information technologies to prepare for, protect against, respond to, and recover from the effects of naturally-occurring (e.g., earthquakes, hurricanes, diseases) and anthropic hazards (e.g., industrial accidents, malicious attacks).
• M E 446: Reliability and Risk Concepts in Design (3): Introduction to reliability mathematics. Failure data collection and analysis. Components and systems reliability prediction. Effects of maintenance on reliability. Risk Analysis. Case studies in engineering applications.
• METEO 460: Weather Risk and Financial Markets (3): This course will introduce the role that weather plays as a source of financial and operational risk for businesses, market and other institutions [Sp09, TH 1115-1230]
• METEO 476: Atmospheric Natural Disasters Seminar (3): Survey of naturally occurring, catastrophic meteorological events, including severe thunderstorms, tornadoes, aviation hazards, floods, and severe winter storms.
• P ADM 401: Introduction to Homeland Security (3): This course provides foundational knowledge about homeland security, including policy, organization, and legal issues in the American context.
• P ADM 404: Homeland Security and Defense in Practice (3): This course analyzes, evaluates, and critiques homeland security plans in practice.
• SCM 456: Supply Chain Risk Analysis (3): Business processes are modeled as a network of queues using discrete-event simulation and analyzed model outcomes using statistical methods.
• SRA 311: Risk Management: Assessment and Mitigation (3): Assessment and mitigation of security vulnerabilities for people, organizations, industry sectors, and the nation.

Note that I included IST and SRA courses for completeness.  Of course there are plenty other courses of interest to me, such as a pair on creative problem solving in the systems engineering program, one on biological networks in the physics department (PHYS 597B), one on game theory in the economics department, and a host of others.

Send post as PDF to

In my search for good books on risk management and intelligence, I came across the edited volume entitled Managing Strategic Surprise: Lessons from Risk Management and Risk Assessment (edited by Professor Paul Bracken of Yale University, Dr. Ian Bremmer of the Eurasia Group, and Dr. David Gordon and the US State Department and former deputy director of the National Intelligence Council, ISBN: 9780521709606).  What a great book (and yes it is my bias toward anything risk analysis oriented)!  I haven’t quite finished it yet, but I must highlight that the first two substantive chapters - chapters two and three - really highlight some important issues.

For example, Paul Bracken’s article “How to Build a Warning System” (pp. 16-42) emphasizes that warning is only one piece of an organization’s overall risk management program.  Professor Bracken highlights six general strategies for risk management (the first time I have seen this): [1] isolating uncertainty (e.g., protection), [2] smoothing of uncertainty (e.g., diversification), [3] warning systems, [4] agility (e.g., rapid response), [5] alliances, and [6] environmental shaping.  Professor Bracken highlights warning systems’ role in providing advanced notice of emerging threats while emphasizing that warning can also inform decision makers of emerging opportunities.  Moreover, Professor Bracken emphasizes that there are two dimensions to warning analysis - the analytic component and the organizational component.  Warning analysis can be either informal (as it is most often the case), or highly structured (as national-level warning systems); but in general every individual and organization has some warning analysis capability.  The organizational component is absolutely essential in that without a structure in place to annunciate warning messages, warning is useless (a point emphasized in many intelligence analysis courses).  Professor Bracken suggests a contingency theory for warning:  “there is no one best way to build a warning system; it depends on the dangers” (p. 26).  The nature of the strategic environment and the capacities of an organization to collect, process, and distribute warning shape how any particular warning system functions.  Even within a single organization, multiple warning systems may be necessary to accomodate multitudes of hazards and threats.

The chapter written by former Director of Intelligence for the Israeli Mossad, Professor Uzi Arad’s article “Intelligence Management as Risk Management: The Case of Surprise Attack” (pp. 43-77) generalizes Prof. Brackens claim by suggesting the intelligence analysis is a risk management function.  He defines intelligence as a “national risk management mechanism built to cope with the risk of violent attack” (p. 45).  It should be noted that DNI’s Vision 2015 says that “intelligence helps reduce the degree of uncertainty and risk when critical choices are made” (Ch. 2).  Granted this view is rather limited by its suggestion that an intelligence organization only looks at downside risks.  But I must admit this definition, as intuitive as it is, adds another dimension to the debate over what “intelligence means” (subscribe to the IAFIE listserv to see what I mean).  More interesting is the idea that the intelligence community, perhaps unlike other types of organizations, must actually consider both environmental risks (dominated by external factors) and operational risks (dominated by internal factors) holisitcally rather than separately: external threats seek to exploit the vulnerabilities of an organization’s internal processes to prevent them from properly assessing environmental risks, thus decreasing the target organization’s decision advantage.  I believe this idea is what justifies the existence of counterintelligence and counterdeception analysis - to help mitigate an organization’s vulnerability to surprise.  This begs the question - what is the probability of a surprise afflicting an organization in its particular strategic environment?  Thinking back to Prof. Bracken’s article, an answer to this question requires us to think carefully about the nature of the strategic environment, capabilities of the adversaries, the organization’s internal processes and culture, and so on.  What I would like to see is a generic approach for assessing the risk of strategic surprise.  The remainder of the paper examines each element of the standard intelligence cycle in terms of the factors that contribute to probability of surprise.  This is good stuff.

While I haven’t read them yet, I look forward to reading the remaining chapters.  These include:

• “Nuclear Proliferation Epidemiology: Uncertainty, Surprise and Risk Management” by Ambassador Lewis A. Dunn (DTRA analysts should find this paper interesting)
• “Precaution Against Terrorism” by Dr. Jessica Stern of Harvard’s Kennedy School of Government and Professor Jonathan B. Wiener of the Duke University Law School and current president of the Society for Risk Analysis
• “Defense Planning and Risk Management in the Presence of Deep Uncertainty” by Professor Paul K. Davis of the Pardee RAND Graduate School
  
• “Manging Energy Security Risks in a Changing World” by Professor Coby van der Linde of the University of Groningen
• “What Markets Miss: Political Stability Frameworks and Country Risk” by Dr. Preston Keat of the Eurasia Group
• “The Risk of Failed-State Contagion” by Provost Jeffrey I. Herbst of Miami University of Ohio
• “Conclusion: Managing Strategic Surprise” by the book’s editors Bracken, Bremmer and Gordon

Just for reference, two interesting papers come to mind that are at least partly relevant to this book.  These include the paper “Using Risk Analysis to Inform Intelligence Analysis” (2008) by Dr. Henry H. Willis of RAND and “The Intelligence Cycle as a Model for Political Risk Assessment” (1985) [published in Political Risks in International Business edited by Thomas L. Brewer, ISBN: 0275900665] by Thomas W. Shreeve of the Intelligence Case Methods Program.  Both of these papers relate aspects of risk analysis to intelligence analysis, but neither really get to the heart of the issues as done in Managing Strategic Surprise.

Send post as PDF to

In a recent issue of my most favorite news magazine The Week, there was a short review on a book with the intruiging title Guesstimation (by Lawrence Weinstein and John Adams at Old Dominion University, ISBN 9780691129495).  In the engineering world, the best engineers are the ones skilled in “guesstimation,” or the art of quantitative approximation.  Unfortunately, few other communities cultivate professionals to become skilled guesstimators (save for actuaries, underwriters, traders, investors, and other people in the finance world).

This book is designed to teach, through example, people of all professions and backgrounds to become apprentice guesstimators (of course, you are not an expert guesstimator until you do it for problems that matter).  Moreover, this book attempts to illustrate the value of approximation.  In fact, this book does not insist at all on being accurate; rather, the book emphasizes that most practical problems require answers accurate only to within a factor of ten (read for yourself in Chapter 1).  More precision is often unnecessary, unhelpful, and frequently impossible to achieve in the amount of time and resources available to collect data and do analysis.

In my view, this book is a very good attempt to debunk misconceptions about quantification (and let me tell you, the Intelligence and Risk Analysis communities really need a few “mythbusters” to come in and rid these analytic worlds of their aversion to numbers).  In addition to teaching analysts how to employ structured analytic techniques, we should also improve their ability to guesstimate and to appreciate the value in doing so.  For one, the order of battle analysis community would benefit from improved guesstimation abilities.  My position has always been that numbers help with reasoning (such as in decision trees, event trees, etc.) even if they are only approximations or guesstimates.

What is missing from this book are many more examples that demonstrate the power of guesstimation in probabilistic analysis or risk analysis (the authors touch on four examples in their eleventh and final chapter).  For example, one could backcalculate the implicit subjective probability of occurrence for a particular threat based on the benefits and costs of a strategy aimed at, say, decreasing the risks due to a biological weapons attack (I have seen such an example in the past when I took a course on science and technology intelligence analysis, or S&TI, but can’t remember where).  We could also use guesstimation to estimate the maximum amount of money that could be spent to achieve a particular risk reduction objective in order to maintain a benefit-cost ratio of, say, one.  Perhaps this will be the goal of the authors’ second book (nudge nudge).  I will send them an email to put in such a request.

Send post as PDF to

Juergen Weichselgartner’s 2001 paper entitled “Disaster Mitigation: The Concept of Vulnerability Revisited” (Disaster Prevention and Management, Vol. 10, No. 2, pp. 85-94, doi:10.1108/09653560110388609) provided a nice summary of alternative definitions for the word “vulnerability” gleaned from a variety of academic publications (copied below; see original paper for citations).

• Gabor and Griffith (1980) Vulnerability is the threat (to hazardous materials) to which people are exposed (including chemical agents and the ecological situation of the communities and their level of emergency preparedness). Vulnerability is the risk context.
• Timmerman (1981) Vulnerability is the degree to which a system acts adversely to the occurrence of a hazardous event. The degree and quality of the adverse reaction are conditioned by a system’s resilience (a measure of the system’s capacity to absorb and recover from the event)
• UNDRO (1982) Vulnerability is the degree of the loss to a given element or set of elements at risk resulting from the occurrence of a natural phenomenon of a given magnitude
• Petak and Atkisson (1982) The vulnerability element of the risk analysis involved the development of a computer-based exposure model for each hazard and appropriate damage algorithms related to various types of buildings
• Susman et al. (1983) Vulnerability is the degree to which different classes of society are differentially at risk
• Kates (1985) Vulnerability is the “capacity to suffer harm and react adversely”
• Pijawka and Radwan (1985) Vulnerability is the threat or interaction between risk and preparedness. It is the degree to which hazardous materials threaten a particular population (risk) and the capacity of the community to reduce the risk or adverse consequences of hazardous materials releases
• Bogard (1989) Vulnerability is operationally defined as the inability to take effective measures to insure against losses. When applied to individuals, vulnerability is a consequence of the impossibility or improbability of effective mitigation and is a function of our ability to detect hazards
• Mitchell (1989) Vulnerability is the potential for loss
• Liverman (1990) Distinguishes between vulnerability as a biophysical condition and vulnerability as defined by political, social and economic conditions of society. She argues for vulnerability in geographic space (where vulnerable people and places are located) and vulnerability in social space (who in that place is vulnerable)
• Downing (1991) Vulnerability has three connotations: it refers to a consequence (e.g. famine) rather than a cause (e.g. drought); it implies an adverse consequence (e.g., maize yields are sensitive to drought; households are vulnerable to hunger); and it is a relative term that differentiates among socioeconomic groups or regions, rather than an absolute measure or deprivation
• UNDRO (1991) Vulnerability is the degree of the loss to a given element or set of elements at risk resulting from the occurrence of a natural phenomenon of a given magnitude and expressed on a scale from 0 (no damage) to 1 (total loss). In lay terms, it means the degree to which individual, family, community, class or region is at risk from suffering a sudden and serious misfortune
  following an extreme natural event
• Dow (1992) Vulnerability is the differential capacity of groups and individuals to deal with hazards, based on their positions within physical and social worlds
• Smith (1992) Human sensitivity to environmental hazards represents a combination of physical exposure and human vulnerability ± the breadth of social and economic tolerance available at the same site
• Alexander (1993) Human vulnerability is function of the costs and benefits of inhabiting areas at risk from natural disaster
• Cutter (1993) Vulnerability is the likelihood that an individual or group will be exposed to and adversely affected by a hazard. It is the interaction of the hazard of place (risk and mitigation) with the social profile of communities
• Watts and Bohle (1993) Vulnerability is defined in terms of exposure, capacity and potentiality. Accordingly, the prescriptive and normative response to vulnerability is to reduce exposure, enhance coping capacity, strengthen recovery potential and bolster damage control (i.e., minimize destructive consequences) via private and public means
• Blaikie et al. (1994) By vulnerability we mean the characteristics of a person or a group in terms of their capacity to anticipate, cope with, resist and recover from the impact of a natural hazard. It involves a combination of factors that determine the degree to which someone’s life and livelihood are put at risk by a discrete and identifiable event in nature or in society
• Green et al. (1994) Vulnerability to flood disruption is a product of dependence (the degree to which an activity requires a particular good as an input to function normally), transferability (the ability of an activity to respond to a disruptive threat by overcoming dependence either by deferring the activity in time, or by relocation, or by using substitutes), and susceptibility (the probability and extent
  to which the physical presence of flood water will affect inputs or outputs of an activity)
• Bohle et al. (1994) Vulnerability is best defined as an aggregate measure of human welfare that integrates environmental, social, economic and political exposure to a range of potential harmful perturbations. Vulnerability is a multilayered and multidimensional social space defined by the determinate, political, economic and institutional capabilities of people in specific places at specific times
• Dow and Downing (1995) Vulnerability is the differential susceptibility of circumstances contributing to vulnerability. Biophysical, demographic, economic, social and technological factors such as population ages, economic dependency, racism and age of infrastructure are some factors which have been examined in association with natural hazard
• Gilard and Givone (1997) Vulnerability represents the sensitivity of land use to the hazard phenomenon
• Comfort, L. et al. (1999) Vulnerability are those circumstances that place people at risk while reducing their means of response or denying them available protection
• Weichselgartner and Bertens (2000) By vulnerability we mean the condition of a given area with respect to hazard, exposure, preparedness, prevention, and response characteristics to cope with specific natural hazards. It is a measure of capability of this set of elements to withstand events of a certain physical character

Of course, this list is by no means complete; in fact, the definitions from obvious sources such as Webster’s dictionary, Department of Defense doctrine, and a host of other papers were not included.  I leave it to the readers of this blog to discover alternative definitions that are most suited for his or her particular application.  But if one was looking for a really short definition of vulnerability to sum up everything above, consider the following two (my preferences):

Vulnerability is the manifestation of the inherent states of a system that render is susceptible to harm or loss (a paraphrased definition of the notion of vulnerability offered by Prof. Yacov Haimes at the University of Virginia)

The vulnerability of an entity to realizing a specified adverse outcome following the occurrence of a particular triggering or initiating event is measured as the conditional probability of the outcome given the triggering event has occurred (an expanded version of the definition I offer in my SRA 311 class at Penn State)

Send post as PDF to

_{[[NOTE: I revised this post on 6 November 2008]]}

A few weeks ago I came across a post on Kris Wheaton’s blog Sources and Methods describing a master of science thesis by Mercyhurst graduate student Bradley E. Perry entitled “Fast and Frugal Conflict Early Warning in Sub-Saharan Africa: The Role of Intelligence Analysis.”  Since then I have been meaning to download the document and give it a careful read.  I am glad that I did - the literature review on early warning systems and risk assessment is very good.  In fact, it provided me with some incentive to read several of the books I purchased recently on the subject (e.g., Preventive Measures by Davies and Gurr (eds.)) and also pointed out some new references I will be sure to check out in the near future (e.g., “Conflict Prognostication” by Verstegen).  I highly recommend this literature review to those individuals working in the warning community - it is a relatively quick read that is well written and packed with good information.  I will be sure to advertise its existence whenever I speak to my colleagues on the subject of warning.

One thing that caught my attention was no reference to any citations that describe warning systems as a risk management tool.  We do warning to manage risk - the sooner we are made aware of an emerging situation, the sooner we can take action to ensure it doesn’t escalate in an unfavorable direction.  A good paper on this subject was written by M. Elisabeth Pate-Cornell in her 1986 article entitled “Warning Systems in Risk Management” published in the journal Risk Analysis, Vol. 6, No. 2, pp. 223-234 (DOI: 10.1111/j.1539-6924.1986.tb00210.x) [note that Professor Pate-Cornell is/was a member of the President’s Foreign Intelligence Advisory Board, the State Department International Security Advisory Board, as well as an active participant in many other very high-profile public service activities).

On the technical side, I am a bit confused with the idea of taking the highest and lowest possible values for the conflict score, 6.03 and 1.77 respectively, and assuming the middle value (3.9) as a cutoff point between conflict being likely and conflict being unlikely (see page 47).  The implication here is that the range of 1.77 to 6.03 is an unnormalized probability scale that, when normalized by subtracting the offset 1.77 and dividing by the resulting maximum 6.03-1.77, produces a scale on the range of 0 to 1.  The middle value in this case corresponds to 0.5, where values 0.5 or greater are taken as likely, and values less than 0.5 is taken as unlikely (check: (3.9-1.77)/(6.03-1.77) = 0.5).  Basically, the assumption here is that the “fast and frugal” model does produce a probability distribution on the finite frame covering the mutually exclusive and collectively exhaustive events “Violent Conflict” and “Not Violent Conflict.”  I am not convinced based on the arguments outlined in the thesis that this assumption is justified.  In fact, there appears to be no clear basis for selecting 3.9 other than it being the median value of possible score combinations (of which there are only 27).

In my original version of this post, I went on to get into the nitty-gritty of the regression, comparing the author’s analysis with that of one of his cited references.  Soon after I published the original post, I found myself delving into logistic regression and attempting to replicate the results of the cited references (which, by the way is less straightforward that one might think).  Then I realized I was getting too obsessed over work that was not my own and did not pay much attention to the bigger picture.  So I stepped back, took a deep breath, and after a careful re-examination of the thesis author’s work, I now think that the model is not bad (perhaps “good enough”, but definitely not without its flaws), is quick to use and does seem to produce reasonably good results.

My final question centers on how the particular model in this thesis informs decision making.  Some of the independent variables described in the thesis are not variables that can be changed easily, that is, they are well beyond any external actors’ ability to control.  For example, it is not that easy to change a country’s political system.  Nor is it straightforward to change the degree of ethnic homogeneity.  But perhaps something can be done to influence income inequity, such as bringing new industry to a country.  In the end I get the point - there very well might exist simple models that enable warning analysts to estimate likeliness of future events in a manner that is good enough.  The question, now, is how to develop such models that not only help predict, but do so in a manner that also offers actionable guidance into what can be feasibly influenced so as to inform strategies to decrease the potential for unfavorable futures.

Send post as PDF to

An 2004 paper by Paul Slovic et al. entitled “Risk As Analysis and Risk As Feelings: Some Thoughts about Affect, Reason, Risk and Rationality” published in the journal Risk Analysis, Vol. 24, No. 2, pp. 311-322 (DOI: 10.1111/j.0272-4332.2004.00433.x) reprinted an excellent Doonesbury strip (by Gary Trudeau) from 1994 entitled “Street Calculus”:

I am not the type (like many other professors and office professionals) to print out comic strips and tack them to my door, leaving them in full view for my visitors to read for years on end as they slowly fade and deteriorate.  But I am the type to post such strips to my blog as it highlights what could very well be going on inside peoples’ heads as they size up different risk situations.

Basically, this comic shows two individuals each using their own mental model for sizing up the risks associated with a completely unknown person passing him or her by in the street.  Each mental model identifies a set of cues that enable the individual to associate the current circumstances with those patterns derived from past experience.  Based on how each individual sizes up the situation, in this case with respect to “risk factors” and “mitigation factors” separately, the individual then runs a mental simulation of a variety of perceived plausible futures to assign a score to RF and MF, where an MF greater than RF means the risk is acceptable.  (Note that pattern recognition and mental simulation are the two sources of power described by Gary Klein’s book of the same name).  Perhaps in reality, though, each individual unconsciously sizes up the situation in a holistic matter, where the resulting level of fear or comfort (consider these two factors opposite feelings along a single continuum) determines perceived acceptability of proceeding along the planned travel path (vice making a course correction to mitigate perceived risk).

Do people actually entertain such checklists in their mind?  I suppose that the speed at which the situation depicted in the comic is unfolding insists that the bearers of risk leverage simple heuristics (again, derived from experience) to make their decision.  I highly doubt that the situation permitted enough time to be systematic in their analysis, but rather Gerd Gigerenzer’s fast and frugal heuristics concept applies.  That is not to say that such heuristics are bad, only that using them produces less transparent decisions that may be prone to the influence of harmful biases or misperceptions.

The topic of risk acceptance will be a large part of the next SRA 311 lecture scheduled for Thursday, 6 Nov 2008.  I think I will flash this comic as part of the discussion.

Send post as PDF to

I recently came across a hard-copy of an email I sent to a colleague sometime back in 2004 while I was an ASME Federal Fellow to the Department of Homeland Security.  If my memory serves me correctly, I sent this email in January 2004, or about midway through my tenure in DHS’s Information Analysis and Infrastructure Protection (IAIP) directorate.  Basically, the question was not how to estimate the psychological impact associated with a terrorist attack, but rather what psychological impact means.  After all, definitions precede measurement.  As a reminder, this note was written over 4 years ago when I was less enlightened, or rather, just 6 months after leaving my job as an aerospace structural engineer focused solely on designing the structural subsystems for scientific, non-defense-related spacecraft.  Nonetheless, I felt that posting it here might inspire similar thoughts entertained by others or perhaps even prompt discussion.

A recent lecture in my Political Analysis course [PUAF 620 at the University of Maryland] inspired me to think about “psychological impact” as a form of consequences.  The lecture was on special interest groups, their causes, and the consequences of their existence.  One theory is that special interest groups are created to protect and preserve the rights of its constituency (either natural rights or rights/benefits bestowed on the group from previous legislation).  A special interest group will form (or mobilize) to protect its interests if it feels its rights are being threatened.

Based on what I learned during this lecture, I propose the following definition for psychological impact: “psychological impact is the degree to which an individual or group of individuals perceive they have been deprived of their rights.”

Let’s think about this - if several department stores scattered across the nation are targeted for a coordinated attack [a very common scenario that has provided the basis for numerous thought experiments], following an attack people will feel that their freedom to shop a department stores has been taken away from them.  Similarly, in the wake of September 11, many Americans felt deprived of their freedom to travel by air.  One can come up with a host of other examples.

It is also interesting to consider the collateral economic impacts.  Perhaps coordinated attacks on several department stores will prevent people from shopping anywhere such stores are located.  One might argue that the downstream impact of this behavior could propagate throughout the entire retail industry.  On the flip side, the inability (and unwillingness) to travel by air following 9/11 attacks did not impact the entire transportation industry.  Rather, people who would otherwise fly opted to travel by trains and automobiles.

So how does one assess psychological impact?  For any attack scenario, one must identify how a successful attack might threaten perceived rights and freedoms.  To do this, we must first understand what rights the public thinks it has.  In the two examples above, the focus was on either freedom to shop and freedom to travel.  To prioritize scenarios based on potential psychological impact, we must order all these freedoms according to their perceived importance to the affected public.  This can be done at the national, regional, state, local, sector, etc. level.  Doing so will facilitate cost-benefit tradeoffs (in the descriptive sense).  Proposed countermeasures must demonstrate a tradeoff between the freedoms such policies protect versus the rights they appear to take away.  In the case of the Patriot Act, freedom to live without terrorism is enhanced in exchange for a weakened right to privacy [was there a positive ROI here?].

It would also be interesting to explore how the economy might respond to any perceived loss of freedom.  For example, a perceived loss of freedom to shop will keep people from spending money.  [how does percieved deprivations of freedom correspond to economic impact?]

Now that I had four years to sit on this, I still think that the proposed definition for psychological impact has merit despite the fact that it essentially equates psychology to perceptions, and does not consider such things as PTSD.  But before I or anyone else accepts this definition, more thought is needed on whether it is complete and all encompassing, whether it precisely articulates what we care about, to what degree such a measure is redundant with respect to other measures of loss (economic impact is a function of societal behaviors), and how such impact can be assessed with confidence.   Even now, 5 years after DHS opened its doors, I am sure any answer to the question of measuring psychological impact would be of interest to DHS risk analysts.

I can finally throw the hard-copy of this email out now that it is posted to my blog.  Just another step toward a purely paperless life…

Send post as PDF to

In a recent set of lectures for my SRA 311 course (risk management), I had my students run through a simple risk analysis case study derived from an unclassified 1993 Defense Intelligence College master’s thesis by then Captain David Lawrence Graves (USAF) entitled “Bayesian Analysis for Threat Prediction.”  The case study consisted of a “notional” scenario supported by five items of intelligence.  The case study is provided in the PDF document below.

According to the scenario statement, the Key Risk Questions center on whether it is the intent of the Revolutionary Party to attempt a hostile takeover of the US embassy in Country A, and if so, when such a takeover would be attempted.  Based on the way this scenario was written, I can’t help but feel that takeover was assumed to be guaranteed given an attempt, which is not necessarily true given the nature of typical embassy defenses, less-than-perfect capabilities of adversaries, and so on.  For the purpose of this case study, we did not make this assumption but did focus on “embassy takeover” as the sole outcome of concern. Presumably, the judgment of whether to evacuate depends on the perceived risks of a takeover attempt occurring in the next 72a-hours - if the perceived likeliness of this event (takeover attempted) combined with the severity of the associated consequences (successful takeover of US embassy) is uncomfortably “high,” then the US Ambassador in Country A might feel justified in ordering an evacuation of non-essential (or all) personnel and destruction/removal of all sensitive information.

The objective of this in-class exercise was to do a little but of source analysis (using Schum and Morris’ 25 questions) given the admittedly limited metadata on sources (which is typical), then to synthesize the nuggets of information provided by each “credible” source to order the four hypotheses based on relative likeliness. To do this, I recommended that the students use a suitable structured analytic technique to help them reason through the problem. In particular, I advocated the use of the Analysis of Competing Hypotheses technique, but without labeling it as such (I didn’t have time to offer a formal lecture on ACH, so I just gave them a list of steps). Once the hypotheses were ordered on the basis of likeliness, then, based on group judgment, the hypotheses were again ordered on the basis of vulnerability, or rather, the likeliness of outcome (e.g., takeover) given a particular hypothesis occurs. A complete risk picture examines all likeliness-severity pairs for each hypothesis (i.e., H1 through H4 as shown in the PDF file above). For example, a quick analysis might produce the following ordering of hypotheses in terms of likeliness of event (i.e., Pr(e)) and in terms of likeliness of a successful takeover given attempt (i.e., Pr(o|e)) such as is shown below.

In the end, the final risk analysis product should communicate what can happen (i.e., hypotheses H1 through H4), the relative likeliness of each, and the probability of the stated undesirable consequences. Combined, this information describes the total risk exposure for this problem. A complete risk summary such as this provides the decision maker with much of what he needs to know to make an evacuation decision. Oftentimes, professional analysts are tempted to reduce this complete narrative to a single statement such as the “risk of embassy takeover is high.” Well, perhaps this statement is true, and in some instances it may be appropriate to make such an aggregate judgment to further summarize the complete risk picture (i.e., … therefore, the risk is high). I tend to avoid making such statements as it starts to impose value judgments that are often best done by the decision makers themselves, and forces an aggregation procedure that often draws counter-constructive criticism. For example, is the analyst really in a position to assume a cut-off value of likeliness above which the decision maker should be concerned and below which the decision maker should not worry? (think of Cheney’s 1%-doctrine - was it the analyst that suggested a 1% cutoff value, or was it the decision maker?). Or, is the analyst really in a position to judge the severity of “takeover” with respect to the interests of the supported decision maker? Keeping with the goal of producing analysis that is as objective as possible, I stand by my suggestion to provide a complete narrative of risk (appropriately formatted, whether in text or as a table) that provides the decision maker with everything he needs to make his own judgment of risk.

(As an aside, note that I do, however, advocate for producing actionable risk assessments, which include both an assessment of risk combined with knowledge on which variables have the potential for risk reduction.)

The final step of this