Note: Article updated on 17 Jan 2010

In 1981, Kaplan and Garrick published a paper entitled “On the Quantitative Definition of Risk” that defined risk as the set of all ordered triplets comprised of answers to the following triplet of questions (Kaplan and Garrick 1981):

• What can go wrong?
• How likely is it to go wrong?
• What are the consequences?

These three questions set the stage for what most risk professionals consider to be the fundamental questions of risk assessment. In recent years, more questions have been suggested, including:

• How much uncertainty is present in the analysis? (Lowder 2008)
• Over what time frame? (Haimes 2009)
• Are these risks tolerable?

In 1991, Professor Yacov Haimes offered a second set of three questions focused on the practice of risk management (Haimes 1991):

• What can be done?
• What options are available and what are the benefits and costs of each?
• What impact do these options have on future options?

Mr. Bob Ross offered a few more interesting risk questions, including several for establishing the risk context (Ross 2009):

• What are my risk management responsibilities?
• What outcomes and objectives am I expected to achieve?
• How are risks perceived by those to whom I am answerable?

Ross also offered a few more for risk management (labeled risk response or more generally risk treatment):

• What could I do about it? (the “options” part of the second Haimes risk management question)
• What should I do about it?
• What will I do about it?

And a few more on risk management effectiveness:

• How well is my chosen course of action working?
• Has anything changed that requires altering my existing risk management measures?
• Are there current trends and/or potential future developments that could require altering my existing risk management measures?

At a high level, Dr. Tony Cox summarizes all of risk analysis in terms of four high-level questions as follows (Cox 2009):

• How bad is it? (Risk Assessment)
• What to say about it? (Risk Communication)
• What to do about it? (Risk Management)
• Who to blame for it? (Risk Attribution)

Seeing how the ultimate goal of studying risk in general is to communicate risk knowledge to people that can then use it to make better (i.e., risk informed or risk supported) decisions. Risk communication, then, must consider the following lower-level questions that would help analysts decide on what to say about risk (Morgan et al. 2002; Apgar 2006):

• What does the intended recipient think or know?
• What does the recipient need to know?
• How should it be told?

Mr. Bob Ross offered the following additional questions for risk communication:

• Between whom does it need to be communicated?
• How can the necessary risk information be most effectively communicated?

Of course, there is always the risk that a communication goes south, thus we should also entertain the questions:

• How likely is it that the communication will work?
• How bad would it be if it doesn’t?

If you look carefully at these questions, you might find some overlap among them and also find that they may be interpreted in different ways by different people. In fact, we could consolidate all of these questions into a triplet of risk analysis triplets. These are summarized as follows.  Given a clearly and precisely specified situational context (e.g., security context), risk analysis centers on the following nine broad questions:

Risk Assessment Triplet

1. What can happen? Answer: scenarios characterized by the pairing of cause and outcome, where associated with outcome is the time frame
2. How likely is it? Answer: product of probability of cause and probability of outcome given cause; uncertainy in the answers is captured using imprecise probabilities
3. How bad would it be? Answer: severity of the cause/outcome pair

Risk Communication Triplet

1. What does the recipient presently think, know and perceive? Answer: the recipient’s mental model and lens for interpreting and integrating new information
2. What does the recipient need to know? Answer: key messages to improve the recipient’s understanding
3. How should it be told? Answer: in what form must the information be communicated and who should communicate it, this includes all risks associated with communications

Risk Negotiation Triplet*

1. What can be done? Answer: the types of changes that can be made in the time frame of interest
2. What options are available? Answer: Answer: real feasible options that are available with assessed benefits and costs of each, where benefits and costs include impact on future options, and all assessments include uncertainty
3. What should be done? Answer: compares benefits, costs and risks of each option in addition to other factors with a variety of non risk-related alternatives including the “do-nothing” option

*Note: In this context, Risk Negotiation refers to an organization’s discussions and deliberations around a variety of risk treatments relative to the organization’s attitude and tolerance for risk.

Risk management revisits this triplet of triplets over and over again in perpetuity. With time, we learn how well our choices fared through continuous analysis and reanalysis of our systems and their environments. With every action we take, the systems we protect respond with new or modified risks with updated probabilities and severities, and new options and considerations emerge while others become infeasible or irrelevant. And of course, with time and change comes new uncertainties and misunderstandings, both of which require the dedicated attention of risk professionals to study and resolve.

References

Apgar, D. (2006). Risk Intelligence: How to Manage What You Don’t Know. Harvard Business School Press (ISBN 1591399548).

Coles-Kemp, L. (2009). “The Effect of Organisational Structure and Culture on Information Security Risk Processes.” Risk Research Symposium (link here).

Cox, L. A. (2009). “Traditional and Current Risk Analysis.” Presented at the MORS 2009 Workshop, April 2009 (link here).

Haimes, Y. Y. (1991). “Total Risk Management.” Risk Analysis, Vol. 11, No. 2, pp. 169-171 (doi link).

Haimes, Y. Y. (2009). “On the Complex Definition of Risk: A Systems-Based Approach.” Risk Analysis, Vol. 29, No. 12, pp. 1647-1654 (doi link).

Kaplan, S. and Garrick, B. J. (1981). “On the Quantitative Definition of Risk.” Risk Analysis, Vol. 1, No. 1, pp. 11-27 (doi link).

Lowder, J. (2008). “The Difference Between Quantitative and Qualitative Risk Analysis and Why it Matters (Part 1).” BlogInfoSec.org (link here).

Morgan, M. G., Fischhoff, B., Bostrom, A. and Atman, C. (2002). Risk Communication: A Mental Models Approach. Cambridge University Press (ISBN 0521002567).

Ross, R. G. (2009). “Total Risk Management Revisited.” Working Paper.

Send article as PDF to

In his book Political/Military Applications of Bayesian Analysis: Methodological Issues (ISBN: 0-86531-945-5), Dr. Douglas Hunter (a good friend of mine) outlines a set of 57 problems, which I label as “Hunter’s 57 Problems” (much like Heinz’s 57 Varieties), often encountered in the application of Bayesian analysis.  These 57 problems comprise the bulk of Table of Contents of the book from Chapter 4 onward, and is provided in list form below for future reference.  The book gives an example and offers strategies for overcoming each problem. (Note that I have my own strategies for dealing with some of these problems, and I do not necessarily agree with everything here; but I will defer discussion of these for later posts).

1. Hypotheses not mutually exclusive and/or not exhaustive: the problem of hypotheses which are defective because they are not mutually exclusive and/or not exhaustive.
2. A recognizably defective residual hypothesis: the problem of a final hypothesis which is recognizably too inclusive, meaningless, and “catch-all.”
3. “Buried” subhypotheses: the problem of unrecognized subhypotheses in a hypothesis (often, but not always, the residual hypothesis), which render the hypothesis somewhat meaningless.
4. The optimal number of hypotheses: the question of how many hypotheses are optimal to use in a given situation.
5. Multiple factors in hypotheses: the problem of how to develop hypotheses when we are dealing with multiple factors.
6. Time and “sliding windows”: the problem is phrasing hypotheses of whether to use a fixed time limit in the future or a “sliding window,” or even to mention the factor of time.
7. Time periods in “sliding windows”: the problem of the appropriate “sliding window” time period to use in hypotheses about decisions that have been made to do something in the future.
8. What types of questions can Bayes answer? the question as to what sorts of questions Bayes can answer and, in particular, as to whether it is reasonable to try to use Bayes to predict either future decisions by actors (such as states) or future events.
9. How far in the future can Bayes predict? the question of how far in advance can we use Bayes to predict either eventual future decisions or future states of affairs.
10. Incorrect results from using conditional independence: the problem of mathematically invalid results because of the use of the conditionally independent form of Bayes rather than the conditionally dependent form of Bayes.
11. Confusing “conditional independence” and “distinct”: the problem of incorrectly equating the concepts of “conditional independence” and “distinct” items of evidence.
12. Results affected by the order of evidence: the problem that the order in which we consider the items of evidence is itself an important factor in determining the revised probabilities of hypotheses when we use the conditionally dependent form of Bayes.
13. The types of evidence to use: the question of whether to use “all” relevant evidence or “just” indicator relevant evidence.
14. Previous data: the problem of the type of “previous data” with respect to previous indicator settings to use and which prior probabilities of hypotheses to use if we choose to use indicators.
    
15. “Causal” evidence: the problem of how to deal with actions which are are intended to alter behavior.
16. The one-sided view: the problem of the one-sided view of a complex interaction process that results if we do not utilize “causal” evidence external to the environment in our Bayesian analyses.
17. Predicting with “mixed” items of evidence: the problem of trying to predict a future event when evidence is a mixture of (1) items of evidence which indicate the state of policy at present and (2) items of evidence which may be the “causes” of possible policy change in the future.
18. Recognizing invalid “evidence”: the problem of identifying “evidence” not really from the environment.
19. “Statistical” evidence: the problem of evidence not really about an event or person, but actually about the class to which the event or person belongs.
20. “Negative” evidence: the problem of the need to consider items (events) of non-occurrences of events in particular time periods.
21. Developing lists of negative evidence: the problem of how to alert analysts to the need to consider negative evidence and how to develop lists of items of negative evidence for a particular set of hypotheses.
22. Deciding the relevance of data: the problem of deciding which items of data clearly are “relevant,” or “peripheral,” or “irrelevant” with respect to hypotheses.
23. The probability of reports vs. the probability of evidence: the problem of deciding when to ask conditional probability that the report (of certain evidence) will occur rather than asking the usual conditional probability that the evidence (contained in the report) will occur.
24. How many events to abstract? the problem of how many events, i.e., items of evidence, to abstract from a given report if we decide that it is units of evidence for which we wish to assess probabilities rather than reports.
25. “Double counting” evidence: the problem  of overweighting items of evidence by assessing the same item each time it is reported.
26. Different characterizations of the true data state: the problem of assessing the probability of an event when there are multiple reports on the event which do not characterize the event the same, but rather characterize the event in a way that is wither complementary (supplementary) or contradictory in nature; in short, the problem of how to handle different characterizations of the true data state.
27. How often to count “negative” evidence: the problem of how often to input a particular item (event) of “negative” evidence.
28. The absence of data: the problem of how to deal with the absence of data about a particular event for a certain period of time.
29. No observations of an event: the problem of how to deal with the item of evidence that a particular event has not been observed in a particular time period.
30. Source reliability: the problem of how to take into account the reliability of a source of information.
31. Assessing source reliability: the problem of how to assess the Zlotnick exponent.
32. New reports and source reliability: the problem, if Zlotnick’s exponent is used, of what to do when new reports reflecting the true data state are received.
33. Multiple reports and source reliability: the problem, if there are multiple reports on the true data state, of whether to include the source reliability for each report, and, if so, how to do so.
34. Unbelievable evidence: the problem of how to deal with evidence which does not appear believable.
35. Phrasing conditional probability judgments: the problem of how to verbalize the conditional probability of events.
36. “Blow-up”: the problem of an unwarranted rapid increase or decrease in the revised probabilities of some hypotheses.
37. “Error bounding”: the problem of the high sensitivity of the revised probabilities of hypotheses to minor variations in the assigned probabilities.
38. The importance of prior probabilities: the problem of the impact of the prior probabilities of the hypotheses if there are a few items of evidence.
39. The importance of ratios: the problem of the importance of the ratios when assigning the conditional probabilities of events occurring.
40. Probabilities or odds: the problem of whether to assign probabilities or odds.
41. Group assessments of probabilities: the problem of how to deal with a group assessment of probabilities.
42. Difficulties in thinking in terms of probabilities: the problem where some analysts think easily in terms of probabilities, others need to work at it every time, and a few need constant attention and retraining to overcome a distorted or unrealistic feeling for probabilities.
43. Lack of analyst expertise: the problem of the lack of necessary expertise to assess the probability of events for some hypotheses.
44. Logically “distant” events: the problem of an event logically “distant” from one or more hypotheses, which makes it difficult or impossible to assess a meaningful conditionally dependent probability for those hypotheses.
45. Systematic bias: the problem of systematic bias – even among sophisticated and knowledgeable – in the probabilities that are assigned to events.
46. Idiosyncratic and situational biases: the problem of biases that only certain people have because of personal, idiosyncratic factors, or because of the special situation.
47. Conscious manipulation of probabilities: the problem where an analyst may manipulate consciously his(her) assigned probabilities to support a favored hypothesis.
48. Trend line biases: the problem of analyst biases emerging from paying attention to trend lines.
49. Time-pressure difficulties: the problem that the time required to do a Bayesian analysis may make it difficult to use it in a crisis situation.
50. Fluctuating revised probabilities: the problem of revised probabilities of hypotheses fluctuating up and down, as each piece of conflicting data, reflective of inconsistent or “hedging” government policies, is processed.
51. Non-stationarity of hypotheses: the problem of a change in the state of nature.
52. “Life-span” of evidence (or “impact of past information”): the problem of when and why and to what extent the probability judgments for “old” items of evidence should be deleted from the analysis (and whether the old evidence should be considered in making probability judgments for new items of evidence)
53. Revising previously assigned probabilities: the problem of deciding under what circumstances previously assigned probabilities of events should be revised.
54. New prior probabilities: the problem of when and why to abandon the original prior probabilities and start a new analysis with new priors.
55. The “reliability” of Bayes: the problem of the “reliability” of Bayesian analysis as a function of proximity to the actual occurrence of one of the hypothesized events and whether or not we can improve reliability.
56. Differences between intuitive and Bayesian predictions: the problem of how to deal with differences (particularly drastic differences between Bayesian predictions and predictions based on personal experience), when you are “sure” your analysis is better than the one based on Bayes.
57. Consumer bias: the problem of consumer bias, particularly either gullibility or cynicism.

I must add that the first chapter of this book offers a neat and concise introduction to probability (not statistics) for intelligence analysts.  In the third chapter, the Dr. Hunter also offers what he views to be 9 major advantages of employing Bayesian analysis for intelligence:

1. Use of Bayesian analysis overcomes the conservative bias it is said we supposedly have regarding our revision of initial probabilities in light of evidence.
2. More information can be extracted from the body of available data because the technique calls for each piece of evidence to add its weight to the final assessment in a systematic way.
3. The technique compels us to employ an improved system of accounting for the evidence used.
4. If different analysts reach different conclusions, the source of the disagreements can be determined more easily than in a normal, verbal analysis.
5. Using Bayes forces us to consider alternative hypotheses.
6. As a related advantage to No. 5, use of Bayes decouples the analyst’s ego from the probability figures assigned.
7. When using Bayes, we are making deductive judgments when we assess the probability of evidence given the truth of a hypothesis and all previous evidence.  Deductive judgments are easier to make than inductive judgments, i.e., the probability of a hypothesis given evidence.
8. We are required to quantify, i.e., make explicit, and get away from the ambiguity of words, judgments which we do not ordinarily express in numerical terms.
9. The revised probability of the hypotheses can change very quickly.

Now, I am not necessarily an advocate for insisting that all analysts use Bayesian analysis in their thinking (though there are plenty of new tools available for free that greatly facilitate the use of Bayesian analysis, such as Microsoft Bayesian Editor and Toolkit).  But I do encourage analysts, if time and energy permits, to diagram their arguments and assign probabilities to events for the purpose of facilitating thinking and diagnosing errors in reasoning and judgment.  Done well, Bayesian analysis also permits sensitivity analysis on reasoning, which would help identify linchpin items of evidence or evidence that, if strengthened or weakened, would cause significant change in judgment or confidence.  Unfortunately, it takes quite a bit of training to become used to Bayesian analysis, and for this reason it will be part of only the most mathematically inclined intelligence analyst.

If one looks at Dr. Hunter’s book as not a text advocating Bayesian analysis, but as a self-help book for Bayesian thinkers (which we all are to a degree), then the 57 problems identified in this book are relevant regardless of whether one uses mathematical formulas to make explicit their thinking.  I highly recommend finding the book somewhere online or in the library and sitting down with it for a few hours to soak in all it has to say.  After all, at the time of the book’s writing in 1984, Dr. Hunter had already accumulated decades worth of experience successfully applying Bayesian analysis to challenging political and military analysis.  And if you ever had the opportunity to sit in a class with him (which I have when I took a nighttime course in the “Analysis of Competing Hypotheses”), you will come to understand that he has seen it all when it comes to both good and bad Bayesian analysis.

But before I conclude, I must point out the existence of several published reviews of this book, including the less-than-favorable one by Walter W. Hill, Jr. published in The American Political Science Review, Vol. 79, No. 2, pp. 615-616, 1985 (permalink here), and the more favorable review by Dina Zines as published in the Annals of the American Academy of Political and Social Science, Vol. 479, pp. 159-160, 1985 (permalink here).  Note that you need institutional access to JSTOR to read these reviews (or pay $9 per review).

Used copies of Dr. Hunter’s book are often available on Alibris.com for about $50.  If somehow was able to get the publisher to release this title to the public domain, I would readily make available a PDF file of the book for all to view and download (and so would Google books).  I am sure the author wouldn’t mind.

Send article as PDF to

Today I gave a lecture to my risk management class at Penn State (SRA 311, Risk Management: Assessment and Mitigation) focused on the words of risk analysis (lecture 2 of 31).  As anyone who provides services to any type of client knows, one of the first things you have to do on day one is ensure a common understanding of key words and phrases.  This was part one of my lecture, that is, explaining that people don’t necessarily assign the same meanings to certain words as others, even if they are in the same field.  The remaining parts focused on two words in particular – “security” and “risk” – and sought to explain what “risk” is and how it fits into security activities.  This lecture was fun for me to deliver, but in hindsight, it was probably a bit too densely packed with ideas for students with less background knowledge.  All in all, I think it went ok.

Class Summary

As a backdrop for discussion, I had my students read two articles.  The first article was entitled “Same Words, Different Meanings: The Need for Uniformity of Language and Lexicon in Security Analysis and Management” by Andrew Harter (a good friend of mine) published by the Critical Infrastructure Protection Program of the George Mason University School of Law in the monograph entitled Critical Infrastructure Protection: Elements of Risk (prepared by Liz Jackson, another good friend of mine).  Basically, this article is a call to action in the security analysis and risk management community for establishing a common lexicon through voluntary consensus standards.  For those unfamiliar with this issue, Mr. Harter’s article addresses the question “why is a common lexicon needed?” and “what can be done to make progress toward this goal?”   Though one might argue that alternative viewpoints (e.g., a common lexicon is not needed) were not addressed in this article (which is a “hit” on fairness), the point surely rings true to anyone who plays the security risk analysis game.  Imagine how difficult it is to communicate on risk matters when your definition of risk (e.g., potential for harm) doesn’t match well with mine (e.g, loss following an event).  I’ve experienced hours of time wasted due to a simple misinterpretation of language, and nothing is worse than arguing semantics when other more important issues have yet to be resolved.

Some might argue that definitions don’t matter so much.  After all, risk analysis is a decision support activity, and really all that matters is whether we have empowered the decision maker with “decision advantage.” [I borrow this phrase from the Jennifer Sims at Georgetown University as it is applicable to ALL areas where analysis is done, risk and intelligence in particular].  Accordingly, one might accept the definition of risk as “whatever is appropriate for the decision maker at the time.”  But as the author of my second paper, Giovanni Manunta, might argue, while such a vague definition might be useful in the client-analyst context, it is not helpful if one desires to treat risk as a science and methodically study all the different subtopics that fall under the heading of risk analysis (see the very first text block on the Society for Risk Analysis homepage for their definition of what “risk analysis” entails).  A common understanding of the various “words of risk analysis” is needed in order to speak sensibly about the subject within the community of educators, scholars, and practitioners.  (as an aside, see Professor Kristan Wheaton’s blog for an interesting and related discussion entitled “What is Intelligence?“)

The second paper discussed in my class was entitled “What is Security?” by Dr. Giovanni Manunta and published in the Security Journal, Volume 12, Issue 3, pp. 57-66 (http://dx.doi.org/10.1057/palgrave.sj.8340030).  I chose this paper for three reasons.  First, for me it was a great read and why not share with my students papers I find worthwhile.  In fact, many of Dr. Manunta’s monographs are really worth spending some time reading and absorbing if you are in the security profession.  Second, this paper is a nice complement to the first in that it goes into great depth as to why a commonly accepted conceptual definition for security is needed.  Third, this paper actually does a good job of describing the conceptual underpinnings of security by explaining in detail the three required elements of a security context – namely, a Protector (the entity that desires security), a Threat (the entity that challenges the protector’s security), and an Asset (the object of conflict).  The general formula for security, S, is then S=f(P,T,A)Si, where the Si outside of the parenthesis is a variable that accounts for the situational factors underlying the relationship between P, T, and A.  If any one of P, T, or A are absent in a given situation, you do not have a security context, and as such it makes no sense to speak about managing risks.

At this point I finished discussing (as socratically as I could in the time I had available) the two articles.  Throughout I attempted to elicit from students answers to questions centered on Elder and Paul’s Eight Elements of Thought and Intellectual Standards to encourage critical analysis of who the people writing such articles are, their purpose for writing, points of view, concepts, assumptions, etc.  However, I tried not to stretch this discussion out too long given that I already had my students complete a written assignment that systematically addresses the eight elements and intellectual standards.

The next portion of this lecture centered on how risk management fits within the world of security.  Borrowing from Manunta’s Diogenes Paper No. 1 (ISBN: 0-9501575-4-6), I sought to leverage assumed prerequisite knowledge of Venn Diagrams and Set Theory to explain the concepts of Security and Not Security, where Not Security includes Total Insecurity and all degrees between.  The degrees in-between represents a fuzzy-boundary between security and not security, that is, if one accepts that the state of security is actually a fuzzy set.  The Venn diagram I used is shown below, though in class I actually drew it on a Tablet PC.

The point I stressed is as follows: in a security context, a Protector has finite resources to make progress toward an unbounded objective.  This is where risk management comes in – risk management is used to maximize the efficiency of these resources by applying them in such a way that maximizes our progress toward a state of security.  The balance of risk between what we want to achieve and what we can achieve is known as the residual risk.  Ultimately, given the options available to us to reduce risk in light of available resources, we want to minimize the residual risk.  But as Manunta points out in “What is Security?,” security involves risk management, but managing risk doesn’t necessarily guarantee security.  That is, risk management and security are not the same thing.

I ended the lecture with a light hearted game of “Risk Mad Libs.”  First, I offered a generic definition of risk intended to guide us through our thinking in the rest of the course.  The definition is as follows:

Risk: The uncertainty around future events

We discussed what was meant by the word “uncertainty” in this definition, and examined the different types of uncertainty that we often encounter in risk analysis.  This includes the variability associated with one or another event occurring among a set of mutually exclusive (distinct) and collectively exhaustive (complete) alternatives, the incertitude associated with whether elements in our set are relevant or whether our set of alternative events is complete, and the inherent vagueness in what any particular element of the set really means.  Unfortunately, my extemporaneous nature kept me from explaining the remaining two words – “future” and “events,” but if I could go back in time I would stress that risk has to do with the uncertainty in what will happen and not what has already happened, where the future “events” can be described as a situational description (“mom will get sick”) or in terms of some measures (“1 morbidity” and “$10,000 in medical fees”).

Now that we had a definition of risk to work with, I asked students to break into groups and fill in the blank:

____________________ Risk

where the blank can represent practically any word.  My specific instructions were to select one “serious” word and one “silly” word, fill in the blank with each in turn, and in doing so characterize the nature of what is meant by the resulting phrase (i.e., who would care, what are some causes of concern and what are outcomes of concern).  I started with the serious word “information” to form the phrase “information risk.”  Then I moved onto the word “political” followed by the silly word “dog.”  For each we identified someone who might be considered a stakeholder in such a field (e.g., “dog owner” for “dog”), and brainstormed what events could occur (“dog runs away”) and the spectrum of ensuing outcomes (“dog gets hit by car,” “dog bites pedestrian,” “dog comes home”).  In the remaining 2 minutes of class following the exercise, we had some cool responses, including “computer mouse risk,” “environmental risk,” “body odor risk,” etc.   The basic idea here was to enable students to reason out what is meant when you see a phrase such as “financial risk,” and after this lecture I am confident the students can do this.

Next Up

The next lecture stands to be a fun one – the topic is “The Role of the Risk Analyst and Decision Advantage.”  This lecture is the second of 3 “Philosophy of Risk” analysis lectures; after these, we will be way more applied in the classroom setting (something I am sure the students would appreciate).

Send article as PDF to