Hot off the press is the latest issue of the International Association for Intelligence Education (IAFIE) newsletter.  In it I contributed an article describing my strategy for, and experiences thus far, teaching my security risk analysis course at Penn State.  The title of the article is “A New Approach to Teaching Security Risk Analysis,” and can be viewed by going to the IAFIE web page, newsletter section.  At the time of writing of this post, the newsletter is not yet available via the website, but I suspect it will be available really soon. So, see below for the full version of the article in the form I submitted it (which may differ from the final version as I did give the editor free-reign to make changes):

A New Approach to Teaching Security Risk Analysis

Interest in risk analysis has increased in the homeland security and intelligence communities in recent years.  The homeland security community uses elements of risk analysis to help decide how to buy-down the potential for loss due to naturally-occurring and anthropic events.  The intelligence community thinks about different aspects of risk issues in most, if not all, strategic assessments.  Private industry, too, leverages risk analysis in both the traditional economic sense (financial risk, insurance) as well as for security (physical, information) and to inform strategic and operational decisions (project risk, political risk).  Unfortunately, while the need for risk analysts is great and perhaps increasing, few educational programs educate students in what risk is and how to go about assessing risk in a manner that best informs the decision making process.

In Fall 2006, the College of Information Sciences and Technology at The Pennsylvania State University established a first-of-its-kind undergraduate major in Security Risk Analysis (SRA).  The goal of the SRA degree program is to educate future security professionals on the threats that challenge society, how decision makers think, and how to properly assess, communicate, and make suggestions on ways to manage risk.  Accordingly, among the many courses students must take include SRA-specific courses in the threat environment, information security, decision analysis, risk management, visual analytics, human-computer interaction, and so on.

As part of my role as a new assistant professor at Penn State, I was asked to develop and instruct the junior level course in risk management (SRA 311).  If one takes a moment to survey the literature on security risk analysis, there is no established pedagogy for teaching risk management at the undergraduate level save for a discussion on the subject that might occur in an course on probability and statistics or industrial engineering.  Textbooks on security risk analysis tend to focus their attention on the technical details of physical or cyber security, often leaving only a chapter-length (e.g., marginal) treatment of risk analysis.  These same books present risk analysis as a tool to order scenarios (e.g., risk analysis = risk matrices) much like the way ACH is treated as a tool to facilitate reasoning.  The one thing I can say with confidence is that risk analysis is not a tool – it is a way of thinking about problems that applies to security, intelligence, and just about every other discipline where critical decisions must be made.

So here I was – a new professor tasked with teaching a course that has never been offered before and with no textbook to guide its development.  Fortunately, the philosophy of risk and risk analysis is really not that hard to explain.  In its most generic form, risk “measures” the potential for gain or loss associated with future events.  The process of doing risk analysis comes down to providing defensible answers to the following three of questions (i.e., the “risk triplet”):

• What can happen?
• How likely is it to happen?
• What are the consequences if it does?

In my experience doing risk analysis, the challenge isn’t understanding what risk analysis is – after all, it often only takes one chapter in a book or a few lectures to explain the fundamentals of risk.  The real difficulties lie in producing analysis that carefully reasons from available evidence to a statement of risk, is mindful of alternative plausible events and outcomes, is free of undue and harmful bias, is critical of the competence and credibility of information sources, and communicates risk in a manner that is informative yet non-judgmental regarding its acceptability. After much thinking about this, it occurred to me that the same things taught to basic analysts in the IC are equally applicable to emerging risk professionals and for the same reasons.  As it turns out, the pedagogy for teaching risk analysis the “right” way was already there, but not where I expected.

Now that I am most of the way through my first offering of SRA 311, I found that many of the same topics discussed in intelligence training courses have been very helpful in getting my students to think carefully about each question of the risk triplet.  Besides covering the basic philosophy of risk and all the components of traditional security risk analysis (e.g., threat, vulnerability, consequence), we discussed the cognitive aspects of analysis from the point of view of descriptive models and empirical evidence, the mechanics of variety of structured analytic methods aimed at assisting reasoning (e.g., problem restatement, divergent/convergent thinking, event/possibility/decision trees), source analysis and analytic confidence (DNI intellectual standards), and risk communication.  We used a variety of in-class examples to give students practice doing risk analysis, to include information security (e.g., benefits/risks of cell phones in SCIFs), physical security (e.g., terrorist attacks, theft/pilferage), and intelligence case studies (e.g., embassy threat analysis).  Finally, I stress over and over again Elder and Paul’s Eight Elements of Thought and Intellectual Standards as an approach to thinking critically about everything we do, whether it be in the form of critical article reviews, methodology/analysis appraisals, and as guidelines for completing the final course project.

Of course, at present I have no real basis for saying whether my approach to teaching risk analysis is any better than an alternative approach I have not conceived.  After all, this is my first time teaching such a course on risk analysis and have no baseline with which to make a comparison.  But having seen real risk professionals in action and knowing what they do and what they need to do better, combined with experiencing first hand the marked improvement in analytic quality of those intelligence professionals that received formal schooling on structured analysis, I assign a high degree of subjective confidence that this approach will serve the security risk analysis community well.  While my educational strategy is not new in the context of intelligence analysis, it is truly a new approach to teaching security risk analysis.

Now it is time to write some journal articles, so I suspect I will not be authoring any more newsletter articles for a few months…

Send article as PDF to

In their really, REALLY good paper entitled “Assessing the Competence and Credibility of Human Sources of Intelligence Evidence: Contributions from Law and Probability” published in the journal Law Probability and Risk, Vol 6, pp. 247-274 (doi:10.1093/lpr/mgm025), authors David A. Schum (of George Mason University) and Jon R. Morris (of CIA DS&T) identified a set of twenty-five (25) questions whose answers bear on the question of whether a human source of information is competent and credible.  The twenty-five questions are as follows divided into four categories: competence, veracity, objectivity, and observational sensitivity.

Competence (or is the source qualified to provide the information?)

Leveraging all relevant existing evidence, for each of the five (5) questions below, respond with one of the following four answers: (a) the evidence on this question favors this source’s competence; (b) the evidence on this question disfavors this source’s competence; (c) I cannot decide whether the evidence on this question favors or disfavors the source’s competence; or (d) there is no available evidence bearing on this question.

1. Did this source actually make the observation being claimed or have access to the information reported?
2. Does this source have an understanding of what was observed or any knowledge or expertise regarding this observation?
3. Is this source generally a capable observer?
4. Has this source been consistent in his/her motivation to provide us with information?
5. Has this source been responsive to inquiries we have made of him/her?

Veracity (or does the source believe what he/she is saying?)

Leveraging all relevant existing evidence, for each of the ten (10) questions below, respond with one of the following four answers: (a) the evidence on this question favors this source’s veracity; (b) the evidence on this question disfavors this source’s veracity; (c) I cannot decide whether the evidence on this question favors or disfavors the source’s veracity; or (d) there is no available evidence bearing on this question.

1. Has the source told us anything that is inconsistent with what this source has just reported to us?
2. Is this source subject to any outside influences?
3. Could this source have been exploited in any way in this report to us?
4. Is there any other evidence from other sources that contradicts or conflicts with what this source has just reported?
5. Is there any evidence from other sources that corroborates or confirms with what this source has just reported?
6. What evidence do we have about this source’s character and honesty?
7. What does this source’s reporting track record show about the source’s honesty in reporting to us?
8. Is there evidence that this source tailored this report in a way that this source believes will capture our attention?
9. Are there collateral details in this report that reflect the possibility of this source’s dishonesty?
10. Evidence regarding the demeanor and bearing of this source during the interview?

Objectivity (or was the source’s belief based on the evidence obtained by the source?)

Leveraging all relevant existing evidence, for each of the five (5) questions below, respond with one of the following four answers: (a) the evidence on this question favors this source’s objectivity; (b) the evidence on this question disfavors this source’s objectivity; (c) I cannot decide whether the evidence on this question favors or disfavors the source’s objectivity; or (d) there is no available evidence bearing on this question.

1. Is there evidence about what this source expected to observe during the reported observation?
2. Is there evidence about what this source wished to observe during the reported observation?
3. Was this source concerned about the consequences of what this source believed during the observation?
4. Is there any evidence concerning possible defects in the source’s memory? Also, how long ago did this source’s observation take place?
5. Is there any other evidence from other sources that contradicts or conflicts with what this source has just reported?

Observational Sensitivity (or how good was the evidence obtained by the source?)

Leveraging all relevant existing evidence, for each of the five (5) questions below, respond with one of the following four answers: (a) the evidence on this question favors this source’s observational sensitivity; (b) the evidence on this question disfavors this source’s observational sensitivity; (c) I cannot decide whether the evidence on this question favors or disfavors the source’s observational sensitivity; or (d) there is no available evidence bearing on this question.

1. The source’s sensory capacity at the time of observation?
2. The conditions under which the observation took place?
3. The source’s track record of accuracy in previous reports?
4. Is there any other evidence from other sources that contradicts or conflicts with what this source has just reported?
5. Are there collateral details in this report that reflect the possibility of this source’s inaccuracy?

Using the Questions

According to the authors, the twenty-five questions above have been implemented in a system called MACE (or Method for Assessing the Credibility of Evidence) that apparently has been under development for some time (I wonder if MACE was fully funded by CIA; if so, do I hear FOIA request?).  The remainder of the paper describes the MACE system and how it works.  For the purposes of this post, it is sufficient to point out that MACE is an evidence marshalling tool.  That is, MACE provides a structured set of questions that enables the analyst to make sense of the evidence bearing on a particular source’s competence and credibility.

In addition to providing an answer to each of the twenty-five questions, MACE insists that the analyst judge the relative importance of each question involving a particular situation and a particular report.  Morever, MACE asks the following two questions:

1. On balance, does the evidence favor or disfavor the source’s competence, veracity, objectivity, and observational sensitivity, keeping in mind the number of questions that remain unanswered?
2. On balance, how strongly does the accumulated evidence favor or disfavor our believing of the report this source has just given us, keeping in mind the number of questions that remain unanswered?

Why Care?

According to the standards for analytic tradecraft articulated in Intelligence Community Directive 203 (ICD 203), all intelligence products must “properly describe the quality and reliability of underlying sources” (section D.4.e.(1)).  [Note that the standard in section D.4.e.(2) is also very important, that is, "properly caveats and expresses uncertainties or confidence in analytic judgments."  But I will defer this discussion until a bit later.]  What Schum and Morris provide is a means for arriving at meaningful statements of source competence and credibility that simply were not available in a documented form prior to publication of this paper.

And why do I, as a risk (not necessarily intelligence, though I can play the part) professional think this is important?  Well, most (if not all) security risk analyses rely mostly on the opinions of subject matter experts, organizational representatives, etc. (i.e., humans) for the information needed to make a judgment about threat, vulnerability, and risk.  Much like in intelligence analysis, risk analysts must carefully appraise the information used to support analysis in terms of both its content and its source so as to ensure that the product is free of unintended bias and influence.

Send article as PDF to

In his book Political/Military Applications of Bayesian Analysis: Methodological Issues (ISBN: 0-86531-945-5), Dr. Douglas Hunter (a good friend of mine) outlines a set of 57 problems, which I label as “Hunter’s 57 Problems” (much like Heinz’s 57 Varieties), often encountered in the application of Bayesian analysis.  These 57 problems comprise the bulk of Table of Contents of the book from Chapter 4 onward, and is provided in list form below for future reference.  The book gives an example and offers strategies for overcoming each problem. (Note that I have my own strategies for dealing with some of these problems, and I do not necessarily agree with everything here; but I will defer discussion of these for later posts).

1. Hypotheses not mutually exclusive and/or not exhaustive: the problem of hypotheses which are defective because they are not mutually exclusive and/or not exhaustive.
2. A recognizably defective residual hypothesis: the problem of a final hypothesis which is recognizably too inclusive, meaningless, and “catch-all.”
3. “Buried” subhypotheses: the problem of unrecognized subhypotheses in a hypothesis (often, but not always, the residual hypothesis), which render the hypothesis somewhat meaningless.
4. The optimal number of hypotheses: the question of how many hypotheses are optimal to use in a given situation.
5. Multiple factors in hypotheses: the problem of how to develop hypotheses when we are dealing with multiple factors.
6. Time and “sliding windows”: the problem is phrasing hypotheses of whether to use a fixed time limit in the future or a “sliding window,” or even to mention the factor of time.
7. Time periods in “sliding windows”: the problem of the appropriate “sliding window” time period to use in hypotheses about decisions that have been made to do something in the future.
8. What types of questions can Bayes answer? the question as to what sorts of questions Bayes can answer and, in particular, as to whether it is reasonable to try to use Bayes to predict either future decisions by actors (such as states) or future events.
9. How far in the future can Bayes predict? the question of how far in advance can we use Bayes to predict either eventual future decisions or future states of affairs.
10. Incorrect results from using conditional independence: the problem of mathematically invalid results because of the use of the conditionally independent form of Bayes rather than the conditionally dependent form of Bayes.
11. Confusing “conditional independence” and “distinct”: the problem of incorrectly equating the concepts of “conditional independence” and “distinct” items of evidence.
12. Results affected by the order of evidence: the problem that the order in which we consider the items of evidence is itself an important factor in determining the revised probabilities of hypotheses when we use the conditionally dependent form of Bayes.
13. The types of evidence to use: the question of whether to use “all” relevant evidence or “just” indicator relevant evidence.
14. Previous data: the problem of the type of “previous data” with respect to previous indicator settings to use and which prior probabilities of hypotheses to use if we choose to use indicators.
    
15. “Causal” evidence: the problem of how to deal with actions which are are intended to alter behavior.
16. The one-sided view: the problem of the one-sided view of a complex interaction process that results if we do not utilize “causal” evidence external to the environment in our Bayesian analyses.
17. Predicting with “mixed” items of evidence: the problem of trying to predict a future event when evidence is a mixture of (1) items of evidence which indicate the state of policy at present and (2) items of evidence which may be the “causes” of possible policy change in the future.
18. Recognizing invalid “evidence”: the problem of identifying “evidence” not really from the environment.
19. “Statistical” evidence: the problem of evidence not really about an event or person, but actually about the class to which the event or person belongs.
20. “Negative” evidence: the problem of the need to consider items (events) of non-occurrences of events in particular time periods.
21. Developing lists of negative evidence: the problem of how to alert analysts to the need to consider negative evidence and how to develop lists of items of negative evidence for a particular set of hypotheses.
22. Deciding the relevance of data: the problem of deciding which items of data clearly are “relevant,” or “peripheral,” or “irrelevant” with respect to hypotheses.
23. The probability of reports vs. the probability of evidence: the problem of deciding when to ask conditional probability that the report (of certain evidence) will occur rather than asking the usual conditional probability that the evidence (contained in the report) will occur.
24. How many events to abstract? the problem of how many events, i.e., items of evidence, to abstract from a given report if we decide that it is units of evidence for which we wish to assess probabilities rather than reports.
25. “Double counting” evidence: the problem  of overweighting items of evidence by assessing the same item each time it is reported.
26. Different characterizations of the true data state: the problem of assessing the probability of an event when there are multiple reports on the event which do not characterize the event the same, but rather characterize the event in a way that is wither complementary (supplementary) or contradictory in nature; in short, the problem of how to handle different characterizations of the true data state.
27. How often to count “negative” evidence: the problem of how often to input a particular item (event) of “negative” evidence.
28. The absence of data: the problem of how to deal with the absence of data about a particular event for a certain period of time.
29. No observations of an event: the problem of how to deal with the item of evidence that a particular event has not been observed in a particular time period.
30. Source reliability: the problem of how to take into account the reliability of a source of information.
31. Assessing source reliability: the problem of how to assess the Zlotnick exponent.
32. New reports and source reliability: the problem, if Zlotnick’s exponent is used, of what to do when new reports reflecting the true data state are received.
33. Multiple reports and source reliability: the problem, if there are multiple reports on the true data state, of whether to include the source reliability for each report, and, if so, how to do so.
34. Unbelievable evidence: the problem of how to deal with evidence which does not appear believable.
35. Phrasing conditional probability judgments: the problem of how to verbalize the conditional probability of events.
36. “Blow-up”: the problem of an unwarranted rapid increase or decrease in the revised probabilities of some hypotheses.
37. “Error bounding”: the problem of the high sensitivity of the revised probabilities of hypotheses to minor variations in the assigned probabilities.
38. The importance of prior probabilities: the problem of the impact of the prior probabilities of the hypotheses if there are a few items of evidence.
39. The importance of ratios: the problem of the importance of the ratios when assigning the conditional probabilities of events occurring.
40. Probabilities or odds: the problem of whether to assign probabilities or odds.
41. Group assessments of probabilities: the problem of how to deal with a group assessment of probabilities.
42. Difficulties in thinking in terms of probabilities: the problem where some analysts think easily in terms of probabilities, others need to work at it every time, and a few need constant attention and retraining to overcome a distorted or unrealistic feeling for probabilities.
43. Lack of analyst expertise: the problem of the lack of necessary expertise to assess the probability of events for some hypotheses.
44. Logically “distant” events: the problem of an event logically “distant” from one or more hypotheses, which makes it difficult or impossible to assess a meaningful conditionally dependent probability for those hypotheses.
45. Systematic bias: the problem of systematic bias – even among sophisticated and knowledgeable – in the probabilities that are assigned to events.
46. Idiosyncratic and situational biases: the problem of biases that only certain people have because of personal, idiosyncratic factors, or because of the special situation.
47. Conscious manipulation of probabilities: the problem where an analyst may manipulate consciously his(her) assigned probabilities to support a favored hypothesis.
48. Trend line biases: the problem of analyst biases emerging from paying attention to trend lines.
49. Time-pressure difficulties: the problem that the time required to do a Bayesian analysis may make it difficult to use it in a crisis situation.
50. Fluctuating revised probabilities: the problem of revised probabilities of hypotheses fluctuating up and down, as each piece of conflicting data, reflective of inconsistent or “hedging” government policies, is processed.
51. Non-stationarity of hypotheses: the problem of a change in the state of nature.
52. “Life-span” of evidence (or “impact of past information”): the problem of when and why and to what extent the probability judgments for “old” items of evidence should be deleted from the analysis (and whether the old evidence should be considered in making probability judgments for new items of evidence)
53. Revising previously assigned probabilities: the problem of deciding under what circumstances previously assigned probabilities of events should be revised.
54. New prior probabilities: the problem of when and why to abandon the original prior probabilities and start a new analysis with new priors.
55. The “reliability” of Bayes: the problem of the “reliability” of Bayesian analysis as a function of proximity to the actual occurrence of one of the hypothesized events and whether or not we can improve reliability.
56. Differences between intuitive and Bayesian predictions: the problem of how to deal with differences (particularly drastic differences between Bayesian predictions and predictions based on personal experience), when you are “sure” your analysis is better than the one based on Bayes.
57. Consumer bias: the problem of consumer bias, particularly either gullibility or cynicism.

I must add that the first chapter of this book offers a neat and concise introduction to probability (not statistics) for intelligence analysts.  In the third chapter, the Dr. Hunter also offers what he views to be 9 major advantages of employing Bayesian analysis for intelligence:

1. Use of Bayesian analysis overcomes the conservative bias it is said we supposedly have regarding our revision of initial probabilities in light of evidence.
2. More information can be extracted from the body of available data because the technique calls for each piece of evidence to add its weight to the final assessment in a systematic way.
3. The technique compels us to employ an improved system of accounting for the evidence used.
4. If different analysts reach different conclusions, the source of the disagreements can be determined more easily than in a normal, verbal analysis.
5. Using Bayes forces us to consider alternative hypotheses.
6. As a related advantage to No. 5, use of Bayes decouples the analyst’s ego from the probability figures assigned.
7. When using Bayes, we are making deductive judgments when we assess the probability of evidence given the truth of a hypothesis and all previous evidence.  Deductive judgments are easier to make than inductive judgments, i.e., the probability of a hypothesis given evidence.
8. We are required to quantify, i.e., make explicit, and get away from the ambiguity of words, judgments which we do not ordinarily express in numerical terms.
9. The revised probability of the hypotheses can change very quickly.

Now, I am not necessarily an advocate for insisting that all analysts use Bayesian analysis in their thinking (though there are plenty of new tools available for free that greatly facilitate the use of Bayesian analysis, such as Microsoft Bayesian Editor and Toolkit).  But I do encourage analysts, if time and energy permits, to diagram their arguments and assign probabilities to events for the purpose of facilitating thinking and diagnosing errors in reasoning and judgment.  Done well, Bayesian analysis also permits sensitivity analysis on reasoning, which would help identify linchpin items of evidence or evidence that, if strengthened or weakened, would cause significant change in judgment or confidence.  Unfortunately, it takes quite a bit of training to become used to Bayesian analysis, and for this reason it will be part of only the most mathematically inclined intelligence analyst.

If one looks at Dr. Hunter’s book as not a text advocating Bayesian analysis, but as a self-help book for Bayesian thinkers (which we all are to a degree), then the 57 problems identified in this book are relevant regardless of whether one uses mathematical formulas to make explicit their thinking.  I highly recommend finding the book somewhere online or in the library and sitting down with it for a few hours to soak in all it has to say.  After all, at the time of the book’s writing in 1984, Dr. Hunter had already accumulated decades worth of experience successfully applying Bayesian analysis to challenging political and military analysis.  And if you ever had the opportunity to sit in a class with him (which I have when I took a nighttime course in the “Analysis of Competing Hypotheses”), you will come to understand that he has seen it all when it comes to both good and bad Bayesian analysis.

But before I conclude, I must point out the existence of several published reviews of this book, including the less-than-favorable one by Walter W. Hill, Jr. published in The American Political Science Review, Vol. 79, No. 2, pp. 615-616, 1985 (permalink here), and the more favorable review by Dina Zines as published in the Annals of the American Academy of Political and Social Science, Vol. 479, pp. 159-160, 1985 (permalink here).  Note that you need institutional access to JSTOR to read these reviews (or pay $9 per review).

Used copies of Dr. Hunter’s book are often available on Alibris.com for about $50.  If somehow was able to get the publisher to release this title to the public domain, I would readily make available a PDF file of the book for all to view and download (and so would Google books).  I am sure the author wouldn’t mind.

Send article as PDF to