McGill Research Blog

Siambabala Bernard Manyena’s 2006 paper entitled “The Concept of Resilience Revisited” (Disasters, Vol. 30, No. 4, pp. 433-450, doi:10.1111/j.0361-3666.2006.00331.x) provided a nice summary of alternative definitions for the word “resilience” gleaned from a variety of academic publications (copied below; see original paper for citations).  The number of definitions are fewer than that for the word vulnerability as talked about in my previous post.

• Wildavsky (1991) Resilience is the capacity to cope with unanticipated dangers after they have become manifest, learning to bounce back.
• Holling et al., (1995) It is the buffer capacity or the ability of a system to absorb perturbation, or the magnitude of disturbance that can be absorbed before a system changes its structure by changing the variables.
• Horne and Orr (1998) Resilience is a fundamental quality of individuals, groups and organisations, and systems as a whole to respond productively to significant change that disrupts the expected pattern of events without engaging in an extended period of regressive behaviour.
• Mallak (1998) Resilience is the ability of an individual or organisation to expeditiously design and implement positive adaptive behaviours matched to the immediate situation, while enduring minimal stress.
• Miletti (1999) Local resiliency with regard to disasters means that a locale is able to withstand an extreme natural event without suffering devastating losses, damage, diminished productivity, or quality of life without a large amount of assistance from outside the community.
• Comfort (1999) The capacity to adapt existing resources and skills to new systems and operating conditions.
• Paton, Smith and Violanti (2000) Resilience describes an active process of self-righting, learned resourcefulness and growth—the ability to function psychologically at a level far greater than expected given the individual’s
  capabilities and previous experiences.
• Kendra and Wachtendorf (2003) The ability to respond to singular or unique events.
• Cardona (2003) The capacity of the damaged ecosystem or community to absorb negative impacts and recover from these.
• Pelling (2003) The ability of an actor to cope with or adapt to hazard stress.
• Resilience Alliance (2005) Ecosystem resilience is the capacity of an ecosystem to tolerate disturbance without collapsing into a qualitatively different state that is controlled by a different set of processes. A resilient ecosystem can withstand shocks and rebuild itself when necessary. Resilience in social systems has the added capacity of humans to anticipate and plan for the future.
• UNISDR (2005) The capacity of a system, community or society potentially exposed to hazards to adapt, by resisting or changing in order to reach and maintain an acceptable level of functioning and structure. This is determined by the degree to which the social system is capable of organising itself to increase this capacity for learning from past disasters for better future protection and to improve risk reduction measures.

Send post as PDF to

Juergen Weichselgartner’s 2001 paper entitled “Disaster Mitigation: The Concept of Vulnerability Revisited” (Disaster Prevention and Management, Vol. 10, No. 2, pp. 85-94, doi:10.1108/09653560110388609) provided a nice summary of alternative definitions for the word “vulnerability” gleaned from a variety of academic publications (copied below; see original paper for citations).

• Gabor and Griffith (1980) Vulnerability is the threat (to hazardous materials) to which people are exposed (including chemical agents and the ecological situation of the communities and their level of emergency preparedness). Vulnerability is the risk context.
• Timmerman (1981) Vulnerability is the degree to which a system acts adversely to the occurrence of a hazardous event. The degree and quality of the adverse reaction are conditioned by a system’s resilience (a measure of the system’s capacity to absorb and recover from the event)
• UNDRO (1982) Vulnerability is the degree of the loss to a given element or set of elements at risk resulting from the occurrence of a natural phenomenon of a given magnitude
• Petak and Atkisson (1982) The vulnerability element of the risk analysis involved the development of a computer-based exposure model for each hazard and appropriate damage algorithms related to various types of buildings
• Susman et al. (1983) Vulnerability is the degree to which different classes of society are differentially at risk
• Kates (1985) Vulnerability is the “capacity to suffer harm and react adversely”
• Pijawka and Radwan (1985) Vulnerability is the threat or interaction between risk and preparedness. It is the degree to which hazardous materials threaten a particular population (risk) and the capacity of the community to reduce the risk or adverse consequences of hazardous materials releases
• Bogard (1989) Vulnerability is operationally defined as the inability to take effective measures to insure against losses. When applied to individuals, vulnerability is a consequence of the impossibility or improbability of effective mitigation and is a function of our ability to detect hazards
• Mitchell (1989) Vulnerability is the potential for loss
• Liverman (1990) Distinguishes between vulnerability as a biophysical condition and vulnerability as defined by political, social and economic conditions of society. She argues for vulnerability in geographic space (where vulnerable people and places are located) and vulnerability in social space (who in that place is vulnerable)
• Downing (1991) Vulnerability has three connotations: it refers to a consequence (e.g. famine) rather than a cause (e.g. drought); it implies an adverse consequence (e.g., maize yields are sensitive to drought; households are vulnerable to hunger); and it is a relative term that differentiates among socioeconomic groups or regions, rather than an absolute measure or deprivation
• UNDRO (1991) Vulnerability is the degree of the loss to a given element or set of elements at risk resulting from the occurrence of a natural phenomenon of a given magnitude and expressed on a scale from 0 (no damage) to 1 (total loss). In lay terms, it means the degree to which individual, family, community, class or region is at risk from suffering a sudden and serious misfortune
  following an extreme natural event
• Dow (1992) Vulnerability is the differential capacity of groups and individuals to deal with hazards, based on their positions within physical and social worlds
• Smith (1992) Human sensitivity to environmental hazards represents a combination of physical exposure and human vulnerability ± the breadth of social and economic tolerance available at the same site
• Alexander (1993) Human vulnerability is function of the costs and benefits of inhabiting areas at risk from natural disaster
• Cutter (1993) Vulnerability is the likelihood that an individual or group will be exposed to and adversely affected by a hazard. It is the interaction of the hazard of place (risk and mitigation) with the social profile of communities
• Watts and Bohle (1993) Vulnerability is defined in terms of exposure, capacity and potentiality. Accordingly, the prescriptive and normative response to vulnerability is to reduce exposure, enhance coping capacity, strengthen recovery potential and bolster damage control (i.e., minimize destructive consequences) via private and public means
• Blaikie et al. (1994) By vulnerability we mean the characteristics of a person or a group in terms of their capacity to anticipate, cope with, resist and recover from the impact of a natural hazard. It involves a combination of factors that determine the degree to which someone’s life and livelihood are put at risk by a discrete and identifiable event in nature or in society
• Green et al. (1994) Vulnerability to flood disruption is a product of dependence (the degree to which an activity requires a particular good as an input to function normally), transferability (the ability of an activity to respond to a disruptive threat by overcoming dependence either by deferring the activity in time, or by relocation, or by using substitutes), and susceptibility (the probability and extent
  to which the physical presence of flood water will affect inputs or outputs of an activity)
• Bohle et al. (1994) Vulnerability is best defined as an aggregate measure of human welfare that integrates environmental, social, economic and political exposure to a range of potential harmful perturbations. Vulnerability is a multilayered and multidimensional social space defined by the determinate, political, economic and institutional capabilities of people in specific places at specific times
• Dow and Downing (1995) Vulnerability is the differential susceptibility of circumstances contributing to vulnerability. Biophysical, demographic, economic, social and technological factors such as population ages, economic dependency, racism and age of infrastructure are some factors which have been examined in association with natural hazard
• Gilard and Givone (1997) Vulnerability represents the sensitivity of land use to the hazard phenomenon
• Comfort, L. et al. (1999) Vulnerability are those circumstances that place people at risk while reducing their means of response or denying them available protection
• Weichselgartner and Bertens (2000) By vulnerability we mean the condition of a given area with respect to hazard, exposure, preparedness, prevention, and response characteristics to cope with specific natural hazards. It is a measure of capability of this set of elements to withstand events of a certain physical character

Of course, this list is by no means complete; in fact, the definitions from obvious sources such as Webster’s dictionary, Department of Defense doctrine, and a host of other papers were not included.  I leave it to the readers of this blog to discover alternative definitions that are most suited for his or her particular application.  But if one was looking for a really short definition of vulnerability to sum up everything above, consider the following two (my preferences):

Vulnerability is the manifestation of the inherent states of a system that render is susceptible to harm or loss (a paraphrased definition of the notion of vulnerability offered by Prof. Yacov Haimes at the University of Virginia)

The vulnerability of an entity to realizing a specified adverse outcome following the occurrence of a particular triggering or initiating event is measured as the conditional probability of the outcome given the triggering event has occurred (an expanded version of the definition I offer in my SRA 311 class at Penn State)

Send post as PDF to

You have to hand it to good old Ben Franklin, the nation’s first decision analyst, or at least the first decision analyst advocating the use of structured analytic techniques.  In a letter dated 1772 to his friend Joseph Priestly, Ben Franklin spoke about his “moral algebra” or “prudential algebra,” or what is perhaps one of the simplest, and most effective structured analytic technique for aiding decision making:

Dear Sir:

In the affair of so much importance to you, wherin you ask my advice, I cannot, for want of sufficient premises, advise you what to determine; but, if you please, I will tell you how.  When these difficult cases occur, they are difficult, chiefly because, while we have them under consideration, all the reasons pro and con are not present to the mind at the same time; but sometimes one set present themselves, and at others times another, the first being out of sight.  Hence the various purposes or inclinations that alternately prevail, and the uncertainty that perplexes us.

To get over this, my way is to divide half a sheet of paperby a line into two columns; writing over the one pro, and over the other con; then during three or four days’ consideration, I put down under the heads short hints of the different motives that at different times occur to me, for or against the measure.  When I have thus got them all together in one view, I endeavor to estimate their respective weights; and, where I find two (one on each side) that seem equal, I strike them both out.  If i find a reason pro equal to some two reasons con, I strike out the three.  If I judge some two reasons con, equal to some three reasons pro, I strike out the five; and thus proceeding I find at length where the balance lies; and if, after a day or two of further consideration, nothing new that is of importance occurs on either side, I come to a determination accordingly.  And through the weight of reasons cannot be taken with the precision of algebraic quantities, yet, when each is thus considered separately and comparatively, and the whole lies before me, I think I can judge better, and am less likely to make a rash step; and in fact I have found great advantage from this kind of equation, in what may be called moral or prudential algebra.

Wishing sincerely that you may determine for the best, I am ever, my dear friend, yours most affectionately.

B. Franklin

For a moment let’s assume that the “Sir” in the salutation was not directed toward Joseph Priestly, but to future decision makers in the United States government.  How would they take it?  I, for one, would heed Dr. Franklin’s advice - he is after all a noted philosopher, mathematician, scientist, artist, statesman, etc., etc.  More importantly, he played a major role (if not the largest role) as one of the founding fathers of the United States government.  So with all this, would he steer us wrong?

From the point of view of one interested in the mechanics of Franklin’s “moral algebra,” let’s examine what he advises more closely.  First, he clearly points out the cognitive limitations of most humans to not be able to entertain the whole picture in their minds at all times.  Sometimes a decision maker might be in an optimistic mood and thus see a decision option for its potential gains, and at other times a decision maker might take the pessimistic view and focus only on the potential losses or harm.  Sometimes a decision maker emphasizes reasons for, and at other times stresses reasons against a particular proposal.  The point is that, as Franklin observes, it is very difficult to be mindful of all reasons pro and con at once.  Hence the need for the use of structured analytic techniques to help decision makers reason through their decision challenges.

Second, the methodology put forth is simple and structured in that it forces each reason for or against a decision into one of two columns, either pro (for the decision) or con (against the decision) (see below for an image of such a thing that I picked up at the National Archives in Washington, DC). Dr. Franklin stresses deliberation in his method, adding items to each column “during three or four days’ consideration.”  This element of time allows one’s conscious and subconscious to continuously (and hopefully critically) evaluate his or her thinking on the particular decision at issue.  I’d imagine that Dr. Franklin would not balk at taking more time to ensure that all majors reasons pro and con are accounted for, but I would also suggest that his pragmatic nature would be mindful of the “80-20 rule.”  Once the major reasons are on paper, the decision maker then does a comparative analysis of all pros and cons, appropriately weighing each individually or in groups against items from the opposite column to see on what side of the pro-con spectrum the balance of reasons lie.  This task, of course, insists on a subjective valuation of each reason along some dimension of personal utility (which inherently includes whatever types of uncertainty, ambiguity, probability, etc. are associated with each reason).  The net result offers a tentative answer to the decision problem; as Dr. Franklin advises, the decision maker should take one or two days to sit on the results to see if anything new comes of importance on either side, such as new reasons, revised weightings, and so on.

Pro Con Paper

I beileve that Dr. Franklin doesn’t really insist that this tool (or any tool) make the answer for the decision maker.  After all, the extra time afforded to final deliberation allows the decision maker to consider the tentative result, and if displeased, seek additional arguments in opposition to the stated advice.  At the very least, this tool helps decision makers get to the heart of (or rather make explicit) their reasons for or against a particular decision.  Having the reasons explicitly stated now allows for objetive analysis as to their merit and value, and may very well help the decision maker identify the harmful biases that may be influencing their gut feelings.  In the end, though, decision makers ultimately make the decision regardless of what the tool says.  As Dr. Franklin believes, users of this tool “can judge better, and [are] less likely to make a rash step.”  In this complex world we live in, that’s all we can really hope for.

Now can this tool also help analysts in general (e.g., risk analysts, intelligence analysts), say, argue for or against a particular hypothesis?  Can this tool be applied to a finite set of competing decisions or hypotheses in the same manner, or does it insist that all decision problems be restated as yes/no questions?  At first glance and without giving it much thought, I would say “moral algebra” bears a striking resemblance to Heuer’s Analysis of Competing Hypotheses (ACH) or Chamberlin’s Method of Multiple Working Hypotheses (MMWH) with the exception that any particular column corresponds to one pro-con table (one column asks the question is a particular hypothesis true or false, and each row element states a reason where a “+” sides with true, “-” sides with false, and “0,” “NA,” or [blank] indicates no bearing either way).  In fact, some argue that the true value of ACH or MMWH is in getting all evidence that in support or against each among a large set of hypotheses on the table for the analyst to view holistically.  Of course, as a scientist, I must ask whether there is any evidence that does not refute the assertion that such a technique as moral algebra actually improves decision making.  But if I were to judge based only on Ben Franklin’s advice, I would probably take him at his word.

Send post as PDF to

Hot off the press is the latest issue of the International Association for Intelligence Education (IAFIE) newsletter.  In it I contributed an article describing my strategy for, and experiences thus far, teaching my security risk analysis course at Penn State.  The title of the article is “A New Approach to Teaching Security Risk Analysis,” and can be viewed by going to the IAFIE web page, newsletter section.  At the time of writing of this post, the newsletter is not yet available via the website, but I suspect it will be available really soon. So, see below for the full version of the article in the form I submitted it (which may differ from the final version as I did give the editor free-reign to make changes):

A New Approach to Teaching Security Risk Analysis

Interest in risk analysis has increased in the homeland security and intelligence communities in recent years.  The homeland security community uses elements of risk analysis to help decide how to buy-down the potential for loss due to naturally-occurring and anthropic events.  The intelligence community thinks about different aspects of risk issues in most, if not all, strategic assessments.  Private industry, too, leverages risk analysis in both the traditional economic sense (financial risk, insurance) as well as for security (physical, information) and to inform strategic and operational decisions (project risk, political risk).  Unfortunately, while the need for risk analysts is great and perhaps increasing, few educational programs educate students in what risk is and how to go about assessing risk in a manner that best informs the decision making process.

In Fall 2006, the College of Information Sciences and Technology at The Pennsylvania State University established a first-of-its-kind undergraduate major in Security Risk Analysis (SRA).  The goal of the SRA degree program is to educate future security professionals on the threats that challenge society, how decision makers think, and how to properly assess, communicate, and make suggestions on ways to manage risk.  Accordingly, among the many courses students must take include SRA-specific courses in the threat environment, information security, decision analysis, risk management, visual analytics, human-computer interaction, and so on.

As part of my role as a new assistant professor at Penn State, I was asked to develop and instruct the junior level course in risk management (SRA 311).  If one takes a moment to survey the literature on security risk analysis, there is no established pedagogy for teaching risk management at the undergraduate level save for a discussion on the subject that might occur in an course on probability and statistics or industrial engineering.  Textbooks on security risk analysis tend to focus their attention on the technical details of physical or cyber security, often leaving only a chapter-length (e.g., marginal) treatment of risk analysis.  These same books present risk analysis as a tool to order scenarios (e.g., risk analysis = risk matrices) much like the way ACH is treated as a tool to facilitate reasoning.  The one thing I can say with confidence is that risk analysis is not a tool - it is a way of thinking about problems that applies to security, intelligence, and just about every other discipline where critical decisions must be made.

So here I was - a new professor tasked with teaching a course that has never been offered before and with no textbook to guide its development.  Fortunately, the philosophy of risk and risk analysis is really not that hard to explain.  In its most generic form, risk “measures” the potential for gain or loss associated with future events.  The process of doing risk analysis comes down to providing defensible answers to the following three of questions (i.e., the “risk triplet”):

• What can happen?
• How likely is it to happen?
• What are the consequences if it does?

In my experience doing risk analysis, the challenge isn’t understanding what risk analysis is – after all, it often only takes one chapter in a book or a few lectures to explain the fundamentals of risk.  The real difficulties lie in producing analysis that carefully reasons from available evidence to a statement of risk, is mindful of alternative plausible events and outcomes, is free of undue and harmful bias, is critical of the competence and credibility of information sources, and communicates risk in a manner that is informative yet non-judgmental regarding its acceptability. After much thinking about this, it occurred to me that the same things taught to basic analysts in the IC are equally applicable to emerging risk professionals and for the same reasons.  As it turns out, the pedagogy for teaching risk analysis the “right” way was already there, but not where I expected.

Now that I am most of the way through my first offering of SRA 311, I found that many of the same topics discussed in intelligence training courses have been very helpful in getting my students to think carefully about each question of the risk triplet.  Besides covering the basic philosophy of risk and all the components of traditional security risk analysis (e.g., threat, vulnerability, consequence), we discussed the cognitive aspects of analysis from the point of view of descriptive models and empirical evidence, the mechanics of variety of structured analytic methods aimed at assisting reasoning (e.g., problem restatement, divergent/convergent thinking, event/possibility/decision trees), source analysis and analytic confidence (DNI intellectual standards), and risk communication.  We used a variety of in-class examples to give students practice doing risk analysis, to include information security (e.g., benefits/risks of cell phones in SCIFs), physical security (e.g., terrorist attacks, theft/pilferage), and intelligence case studies (e.g., embassy threat analysis).  Finally, I stress over and over again Elder and Paul’s Eight Elements of Thought and Intellectual Standards as an approach to thinking critically about everything we do, whether it be in the form of critical article reviews, methodology/analysis appraisals, and as guidelines for completing the final course project.

Of course, at present I have no real basis for saying whether my approach to teaching risk analysis is any better than an alternative approach I have not conceived.  After all, this is my first time teaching such a course on risk analysis and have no baseline with which to make a comparison.  But having seen real risk professionals in action and knowing what they do and what they need to do better, combined with experiencing first hand the marked improvement in analytic quality of those intelligence professionals that received formal schooling on structured analysis, I assign a high degree of subjective confidence that this approach will serve the security risk analysis community well.  While my educational strategy is not new in the context of intelligence analysis, it is truly a new approach to teaching security risk analysis.

Now it is time to write some journal articles, so I suspect I will not be authoring any more newsletter articles for a few months…

Send post as PDF to

_{[[NOTE: I revised this post on 6 November 2008]]}

A few weeks ago I came across a post on Kris Wheaton’s blog Sources and Methods describing a master of science thesis by Mercyhurst graduate student Bradley E. Perry entitled “Fast and Frugal Conflict Early Warning in Sub-Saharan Africa: The Role of Intelligence Analysis.”  Since then I have been meaning to download the document and give it a careful read.  I am glad that I did - the literature review on early warning systems and risk assessment is very good.  In fact, it provided me with some incentive to read several of the books I purchased recently on the subject (e.g., Preventive Measures by Davies and Gurr (eds.)) and also pointed out some new references I will be sure to check out in the near future (e.g., “Conflict Prognostication” by Verstegen).  I highly recommend this literature review to those individuals working in the warning community - it is a relatively quick read that is well written and packed with good information.  I will be sure to advertise its existence whenever I speak to my colleagues on the subject of warning.

One thing that caught my attention was no reference to any citations that describe warning systems as a risk management tool.  We do warning to manage risk - the sooner we are made aware of an emerging situation, the sooner we can take action to ensure it doesn’t escalate in an unfavorable direction.  A good paper on this subject was written by M. Elisabeth Pate-Cornell in her 1986 article entitled “Warning Systems in Risk Management” published in the journal Risk Analysis, Vol. 6, No. 2, pp. 223-234 (DOI: 10.1111/j.1539-6924.1986.tb00210.x) [note that Professor Pate-Cornell is/was a member of the President’s Foreign Intelligence Advisory Board, the State Department International Security Advisory Board, as well as an active participant in many other very high-profile public service activities).

On the technical side, I am a bit confused with the idea of taking the highest and lowest possible values for the conflict score, 6.03 and 1.77 respectively, and assuming the middle value (3.9) as a cutoff point between conflict being likely and conflict being unlikely (see page 47).  The implication here is that the range of 1.77 to 6.03 is an unnormalized probability scale that, when normalized by subtracting the offset 1.77 and dividing by the resulting maximum 6.03-1.77, produces a scale on the range of 0 to 1.  The middle value in this case corresponds to 0.5, where values 0.5 or greater are taken as likely, and values less than 0.5 is taken as unlikely (check: (3.9-1.77)/(6.03-1.77) = 0.5).  Basically, the assumption here is that the “fast and frugal” model does produce a probability distribution on the finite frame covering the mutually exclusive and collectively exhaustive events “Violent Conflict” and “Not Violent Conflict.”  I am not convinced based on the arguments outlined in the thesis that this assumption is justified.  In fact, there appears to be no clear basis for selecting 3.9 other than it being the median value of possible score combinations (of which there are only 27).

In my original version of this post, I went on to get into the nitty-gritty of the regression, comparing the author’s analysis with that of one of his cited references.  Soon after I published the original post, I found myself delving into logistic regression and attempting to replicate the results of the cited references (which, by the way is less straightforward that one might think).  Then I realized I was getting too obsessed over work that was not my own and did not pay much attention to the bigger picture.  So I stepped back, took a deep breath, and after a careful re-examination of the thesis author’s work, I now think that the model is not bad (perhaps “good enough”, but definitely not without its flaws), is quick to use and does seem to produce reasonably good results.

My final question centers on how the particular model in this thesis informs decision making.  Some of the independent variables described in the thesis are not variables that can be changed easily, that is, they are well beyond any external actors’ ability to control.  For example, it is not that easy to change a country’s political system.  Nor is it straightforward to change the degree of ethnic homogeneity.  But perhaps something can be done to influence income inequity, such as bringing new industry to a country.  In the end I get the point - there very well might exist simple models that enable warning analysts to estimate likeliness of future events in a manner that is good enough.  The question, now, is how to develop such models that not only help predict, but do so in a manner that also offers actionable guidance into what can be feasibly influenced so as to inform strategies to decrease the potential for unfavorable futures.

Send post as PDF to

An 2004 paper by Paul Slovic et al. entitled “Risk As Analysis and Risk As Feelings: Some Thoughts about Affect, Reason, Risk and Rationality” published in the journal Risk Analysis, Vol. 24, No. 2, pp. 311-322 (DOI: 10.1111/j.0272-4332.2004.00433.x) reprinted an excellent Doonesbury strip (by Gary Trudeau) from 1994 entitled “Street Calculus”:

I am not the type (like many other professors and office professionals) to print out comic strips and tack them to my door, leaving them in full view for my visitors to read for years on end as they slowly fade and deteriorate.  But I am the type to post such strips to my blog as it highlights what could very well be going on inside peoples’ heads as they size up different risk situations.

Basically, this comic shows two individuals each using their own mental model for sizing up the risks associated with a completely unknown person passing him or her by in the street.  Each mental model identifies a set of cues that enable the individual to associate the current circumstances with those patterns derived from past experience.  Based on how each individual sizes up the situation, in this case with respect to “risk factors” and “mitigation factors” separately, the individual then runs a mental simulation of a variety of perceived plausible futures to assign a score to RF and MF, where an MF greater than RF means the risk is acceptable.  (Note that pattern recognition and mental simulation are the two sources of power described by Gary Klein’s book of the same name).  Perhaps in reality, though, each individual unconsciously sizes up the situation in a holistic matter, where the resulting level of fear or comfort (consider these two factors opposite feelings along a single continuum) determines perceived acceptability of proceeding along the planned travel path (vice making a course correction to mitigate perceived risk).

Do people actually entertain such checklists in their mind?  I suppose that the speed at which the situation depicted in the comic is unfolding insists that the bearers of risk leverage simple heuristics (again, derived from experience) to make their decision.  I highly doubt that the situation permitted enough time to be systematic in their analysis, but rather Gerd Gigerenzer’s fast and frugal heuristics concept applies.  That is not to say that such heuristics are bad, only that using them produces less transparent decisions that may be prone to the influence of harmful biases or misperceptions.

The topic of risk acceptance will be a large part of the next SRA 311 lecture scheduled for Thursday, 6 Nov 2008.  I think I will flash this comic as part of the discussion.

Send post as PDF to

Everyone says that structured analytic techniques are good things to have as part of a “Thinkers Toolkit.”  In the security risk analysis degree program at Penn State, several of my colleagues and I make every attempt to instruct our students in the proper application of and value added of using structured analytic techniques to enhance one’s ability to think clearly, carefully and rigorously through complex problems.  Unfortunately, our situations suffer from a significant setback - most of our students lack “real world” experience doing analysis for problems in the security and intelligence communities (or perhaps doing any real analysis at all for any community).  Accordingly, we often find ourselves searching for carefully constructed case studies that provide the right balance of realism and accessibility to students that may not have sufficient domain knowledge to speak credibly on any particular issue.  We desire case studies that contain enough information to allow students to define the problem, articulate alternative hypotheses, leverage evidence to establish probability distributions over a set of future alternatives and degrees of confidence in analytic judgments, do source analysis, and so on.

To date we have come across several case studies used in the intelligence community, such as those developed by Professor Francis Hughes at the National Defense Intelligence College and several of the cases authored by Thomas Shreeve as part of the Intelligence Community Case Method Program.  And fortunately for us, these case studies have proven to be moderately successful when used as part of our classes.  However, we are still in search of more case studies that walk students through a problem, asking them to apply different structured analytic techniques to enable them to draw defensible inferences from data, make judgments of risk and choose from among alternative strategies for mitigating risk, explore how different ways of communicating analytic results might influence the decision maker, and so on.  And of course, we are also interested in case studies that have a variety of alternative endings, mainly to highlight that the results of the analysis and the way its communicated does have an affect on the outcomes of a situation as well as setting the stage for later analysis.

In my pursuit of fun books to read to my kids before bedtime, I recently came across the Choose Your Own Adventure series of books that many of us enjoyed during our more youthful years.  I tried to recall my experiences reading these books, such as navigating through all the alternative storylines one can follow based on the choices made during the book (one CYOA fan actually took the time to actually develop a map of The Mystery of Chimney Rock by Edward Packard; I must admit that I was tempted to do the same).  Then a thought hit me - would it be possible to develop a CYOA book that resembled a storyline that one might encounter in a professional security or intelligence position?  In addition to providing a compelling story, such a book would, of course, provide greater depth to a problem, provide evidence, and try to be as real as possible so that readers can draw on external resources to aid them in their analysis.  Now here is the kicker - each analysis or decision node would insist that the reader apply a specific structured analytic technique to arrive at the best possible answer or decision.  Once the answer is chosen, the story will then continue.  Some decision nodes would be critical to preserving national security, whereas some others might be less so or even irrelevant to the outcome.  When used as part of a course, the analyst would then prepare written reports along the way outlining the steps they took to arrive at a judgment or decision.

As an attempt to appeal to those individuals having read and enjoyed CYOA books in the past, I decided to label this idea as “Choose Your Own Analytic Adventure” or CYOAA.  See the prototype cover I prepared for the first such book in the series shown above.  I imagine that the analytic training community could create an entire series of such analytic books spanning all aspects of interest, to include terrorism, resource allocation, HUMINT targeting and collection, counter-deception, counter-proliferation, risk analysis, post-blast investigation, cyber security, communicating to decision makers, etc.  What we would need to do this are good writers, good ideas, good researchers, and of course, good artists capable of drawing pretty maps, figures, and sketches (and perhaps permission from the CYOA people to model our books after their likeness).  Just imagine it - we could hand these books out as part of class, and not only would they provide a basis for practicing analysis, but they would also make for a good addition to one’s professional library.  And if they are truly written well, then perhaps they might also make for good recreational reading.