In Spring 2009, I anticipate having 65 students (or more) enrolled in my SRA 311 (Risk Management: Assessment and Mitigation) course. SRA 311 is the last required core course for the Security Risk Analysis undergraduate major at Penn State University. Most students in this course will be second-semester juniors or first-semester seniors interested in a career in security risk analysis or intelligence analysis.
All SRA 311 students are required to contribute to a final course project that seeks to perform a risk study for a real problem of real interest to real decision makers. I anticipate 5-person teams, and with 65 students this means I should have about 13 teams. This also means I need at least 13 final course project ideas to choose from.
To meet my needs, I am currently seeking course project ideas for my SRA 311 students. If you have any risk analysis project ideas that would lend itself to student participation, please send me an email or leave a comment to this post. Ideas from last semester include:
Self-assessment methodology for social network participation risk
Risk analysis self-assessment methodology for campus lab theft
Press release preparedness methodology
many others…
Some of the ideas I have for Spring 2009 include:
User risk assessment for an online social/collaborative environment (PSU Home, Second Life, etc.)
Research lab security assessment methodology
Methodology for hazard preparedness (each group focused on a different hazard)
Technology transfer risk assessment methodology
Structure and content of a regional threat and vulnerability forecast
Risk assessment methodology for organizational surprise
What I would really like are some information security-oriented risk analysis project ideas, a few homeland security ones, maybe one or two methods geared toward the national security or business intelligence communities, etc.
Unlike in Fall 2008, many Spring 2009 projects will be focused on building simple decision support tools that implement the methodology, complemented by a media presentation (You Tube video, website, poster, NO POWERPOINT). Of course, for those niche studies, the project will be dominated by a paper.
Hooray! As of about 6:30PM on Saturday, 20 Dec 08, I am done with SRA 311 (Risk Management) for the Fall 2008 semester! I have been thinking for several weeks now about what worked in SRA 311, what didn’t work, and what could be better from both student and instructor (and teaching assistant) perspectives. And now that class is over, I decided to take a few hours to write-about how I plan to do things differently for iteration 2 of SRA 311 in Spring 2009. For reference, I include my Fall 2008 syllabus below:
In the following I will just highlight a few of the changes I plan to make in the second iteration of SRA 311.
Revised Course Content for Iteration Two of SRA 311
Probability Theory: Looking back, I think I overstressed some less important topics, and forgot to include others I am now finding to be quite important. The most important thing I should have done was go over the basics of probability theory in detail rather than assume that my students were fully equipped to think probabilistically (I should have known better given that, at best, my students have had only a few lectures on the subject prior to taking this course). So, for the first part of the course, I plan to get serious about teaching probability theory from first principles. But of course, I will discuss this subject with respect to its place in security risk management. (btw: the book I plan to draw from is Introduction to Applied Probability by Pfeiffer and Schum, 1973, ISBN: 0125531508).
The Six Questions of Risk, Risk Triplet, and Definitions: As I did last semester, I plan to stress the six questions of risk assessment and risk management over and over again. The same holds for my repeated mention of the risk triplet and the definition of risk. This time, however, I will emphasize the risk triplet as being the set of scenario, s (which is the pair of initiating event and outcome, or s = (e,o)), probability of the scenario, p, and utility associated with the scenario, u. That is, risk is the set of all relevant ordered triples {<s,p,u>}. As for definitions, I am going to largely focus on risk as the potential for harm or loss, and thus save the more generic definition (i.e., risk as uncertainty about future events) for graduate discussions. Also, I will also stress the need for common definitions issue less as I found that such talk either goes over my students heads at best, or confuses them at worst.
Set Theory and Open vs. Closed Worlds: As I did last semester, I will spend about a week talking about sets (mutually exclusive, collectively exhaustive, conditionally exhaustive, etc.), as well as talk quite a bit about the difference between open and closed world thinking (w/ residual hypothesis). I still think that talking about open worlds (i.e., admitting the possibility of a residual hypothesis) should be introduced at the very beginning of a student’s exposure to risk and uncertainty. See my previous post on the topic. I plan to keep this lecture pretty much intact, but I do think I might add a few more security-oriented examples.
Utility Theory: To accommodate a full discussion of risk, I will be sure to spend a full lecture on the basics of utility theory, to cover what utility is, aspects of multicriteria decision analysis, risk attitudes, and so on. Last semester I spoke about utility for only 10 or so minutes, and consequently my students could not speak to the topic on the final. Though I a lot of utility theory was already covered in the prerequisite class SRA 231 (Decision Analysis), a little review couldn’t hurt.
Support Theory, Possibility Theory and Surprise: Integrated into my discussion of probability theory will be a discussion of support theory (the descriptive side of humans and probability), mention of possibility theory and its axioms, and also mention of Shackle’s theory of potential surprise. I am not clear yet how just where these discussions will occur, but they will surely happen somewhere during the first part (fundamentals) of the course.
Talk About Uncertainty: The discussion of uncertainty (aleatory and epistemic) and all its types will be presented up front with or soon after the introduction of the concept of risk. If all goes well, this will happen during lecture 2. We will also include the discussion of different types of ignorance at this time (unlike last time when it felt out of place in lecture 11).
Security Context and the Eight Elements of Thought: As I did last semester, I will introduce Liner and Paul’s Eight Elements of Thought and Intellectual Standards in great detail during the first lecture. Then, as before, the students will apply the Eight Elements and Intellectual Standards to their first Critical Article Review (CAR) assignment of Manunta’s paper “What is Security?” (published in a 1999 issue of Security Journal). The second lecture will be spent going over the Eight Elements and the Intellectual Standards as they apply to Manunta’s article, discussing the concept of a security context, and then proceed to do an in class example identifying and articulating different security contexts. This semester I want the students to be confident about the Eight Elements and Intellectual Standards by the end of their first week.
Accreditation: As part of the discussion on risk acceptance in Part III of the course, I will include a discussion of accreditation, or the practice of acknowledging that the risk associated with a protected asset is acceptable with respect to its value and purpose. I didn’t do this last semester, but now that SRA 311 is an essential part of the NSA Certificate in Risk Analysis, I figured I ought to start talking about accredidation. I will also include a discussion on standards, whether implicit (e.g., what would a normal security manager do) or explicit (e.g., contractually).
Life-Cycle Cost: When I talk about risk management this semester, I will be sure to include a discussion of life cycle cost of a risk mitigation strategy, to include maintenance costs, replacement costs, operational and procurement costs, and more interestingly, the implicit costs of decreased performance as adversary’s adapt and learn to overcome the countermeasure.
The Insurance Game: My good friend Professor Bilal Ayyub at the University of Maryland recently pointed out to me an interesting pedagogical exercise aimed at teaching undergraduates how to appreciate the role insurance plays in risk management. This semester, I plan to try out this game in the classroom to see how it works (and perhaps spend some money making cool game props, such as custom cards and so on).
Expert Elicitation and Probability Calibration: Last semester I spoke about probability, but did not talk at all about how to elicit probabilistic information when needed. This means that I also did not talk about how to calibrate personal probability judgments. Though I had every intention of talking about this during my fact finding discussion (which I also skipped over), this semester I will be sure to spend a whole lecture on the subject. I will call this lecture “Expert Elicitation and Fact Finding.”
Analytic Confidence: The discussion of analytic confidence will take place sometime in the first 5 weeks of class, probably after my discussion of conditional probability and possibility theory. Last semester I spoke about this all-too-important subject during lecture 20 – by then it was already too late for the concepts to sink in. I won’t make this mistake again.
Influence Diagrams: I am kicking myself for not discussing influence diagrams in class this past semester. Next semester I plan to not only talk about influence diagrams, but also have students use one or more software tools to draw and quantitatively analyze influence diagrams. This should be fun.
Decision Advantage: While risk analysis does promote decision advantage, I think that I will abandon this awkward phrase next semester. Instead, I will simply stick with “risk analysis informs decision making.”
Metrics and Formulas for Risk: As I did last semester, this semester I plan to cover all the relevant measurement scales and formula types one might encounter in a risk analysis methodology. But this time I will do it all in one lecture (or maybe a lecture and a half). I will also have some references to draw on this time around.
Pre-Mortem Analysis, Root Cause Analysis, and Convergent/Divergent Thinking: Last semester I ran an interesting case study focused on the 2007 shooting incident at the Virginia Tech campus. A few lectures after running this case study I figured out how to relate pre-mortem analysis and convergent/divergent thinking ot the case-study. But by the time I did this, it was already too late to solidify the connection in the student’s minds. So, this semester I plan to spend a week covering pre-mortem analysis and covergent/divergent thinking (and introduce the similar topic of root cause analysis) concurrently with running the case study. But unlike last semester’s VT case study, this semester’s focus will be on Aum Shimrikyo and the mid-1990’s sarin gas attack on the Tokyo subway. Other options might be a case study on the Khobar Towers bombing or one focused on the bombing of the Marine barracks in Beirut.
More Information Security and Crime, Less Terrorism: While terrorism is a hot topic these days (though becoming less so), I want to be sure that my course on security risk management (i.e., security in general) covers more than notional terrorists with bombs. This semester I plan to spend more time thinking about information security problems, routine criminal problems, and perhaps a little bit of personnel security/executive prevention. I will also talk about loss prevention as an idea, and also spend some time examining how safety balances and sometimes interferes with security.
CORAS, the McCumber Cube Model, and others: This semester I will start talking about a number of established security concepts and processes, to include CORAS, the McCumber Cube model, and OCTAVE, in addition to reviewing the basic concepts of the security bow tie and the swiss cheese model. But since these topics are no fun to hear about on their own, I still need to figure out a strategy for integrating them into the standard flow of course ideas. I think I figured out a way…
Certifications, Professional Societies, and Ethics: This time around, I will emphasize all the different certifications and security professional societies all throughout the semester. I plan to also integrate ethics into the curriculum in two ways – first by highlighting ethical issues as a matter of course during the semester, and to cap the course off with an ethics story-telling exercise on the last day of class (as I did this past semester, but this time it will be more structured).
Real Questions from Real Certification Exams: This semester I plan to integrate real risk management questions from either the CISSP exam, CPP or PSP exam, perhaps even the CAS Exam P (for actuaries). The goal here is to highlight that everything I teach in my class is relevant to things that matter in the professional world. I anticipate that no less than 25% of the final exam will consist of questions taken from professional exam study guides.
Assignments and Policies
Established Groups at the Beginning of the Semester: On day one I will assign all students to work in groups of my crafting. They are free to make individual trades among themselves long as the class is evenly divided into groups. These groups will work on all in-class exercises, homework, and projects together.
More Quizzes: Attendance was a big problem for me last semester. Without having the data in front of me, I estimate that, on average, only 60% of students showed up for any given lecture. So, this semester, in attempt to better prepare my class for the multiple-choice final exam, to review course material in a fast and effective way, and to take attendance, I plan to give frequent in-class multiple choice quizzes on either the assigned readings (CAR-style questions) or previous lecture’s material. Quizzes will be my means of taking attendance as well as gauging student performance.
More Organized CARs: Unlike last semester where I divided up the class so that 10-14 CARs were due at each lecture, this semester I plan to arrange the schedule such that all students work on CARs at the same time. This means five CARs, each due for all students at the same time. No make-ups. The format for these CAR assignments will be exactly the same as it was last semester.
Critical Book Reviews: Like last semester there will be two required book reviews. But unlike last semester, I will prescribe both books. The first book is Against the Gods: The Remarkable Story of Risk by Peter Bernstein, and the second book is Risk Intelligence by David Apgar. The format for these assignments will be exactly the same as before.
Homework: Ah, there will be homework assignments this semester. Homework will largely consist of preparatory exercises for quizzes. But there will be times when I ask for, say, an influence diagram, some worked problems, etc. All homework will be done in groups.
Final Course Project: This semester, the focus of the final project will be focused on building a risk assessment tool for exploring a particular risk problem of interest to real decision makers. That is, the tool is primary, and will be supported with some multi-media presentation (e.g., reports, poster, auto slide show, You Tube, etc.). While the topics have yet to be determined, I will make available 5-10 topics that groups may choose from. I am tentatively thinking about one or two on maritime piracy, one or two on lab site security, one or two on online communities, one on social engineering, one on party security, and so on. I am still eliciting ideas from people, and hope to have a list in hand by the second week of class.
Methodology Appraisal: There will be no formal methodology appraisal this semester. Rather, I will integrate a methodology review into one or two of the five CAR assignments.
Final Exam: There will be a final exam that, for the most part, will assume the same form as the exam from the Fall semester. There is some question in my mind whether to keep the CAR in the exam or if I should make the entire exam one big multiple choice test. I think I will keep my options open for the next few weeks.
Extra Credit: I always give extra credit, but never anything that amounts to more than 5% on top of a student’s final grade. This semester, extra credit was very helpful for those students who, for some reason or another, didn’t do well on the first assignments of the semester. Although I will not guarantee extra credit opportunities, I suspect that something will come up toward the middle-end of the semester. After all, it helps out those who did well on previous assignments, but not well enough to meet my cutoff values for certain letter grades. But in a perfect world there would be no need for extra credit since everyone would have already done superb routine work…
Attendance: Attendance in required. I will take attendance most of the time this semester, but not always. My policy for attendance is that I don’t give points to students for showing up to class. Rather, I take away points for not showing up to class. My plan for Spring is to implement an attendance policy that is tolerant of up to two (2) random absences, and then for apply a reduction factor to the final exam grade that is in proportion to the number of classes missed. If a student misses all classes, that student will get a zero on the final regardless of whether he or she actually takes it [in math speak, the final exam grade = actual score * (attendance days - missed days)/attendance days]. This is a hard core policy, but perfectly reasonable.
Course Materials: This semester three books will be required – two for the book reviews (see above) and one newer version of a book covering the Eight Elements of Thought. However, this semester I will also insist on using a variety of online soft-copy materials that will all be posted to the PSU course management system (i.e., ANGEL).
Office Hours: Despite having official office hours posted, either no one comes or they try to schedule a different time with me. So, my intentions this year are to have office hours by appointment only. But I also plan to do it in different environments, such as Second Life, PS3 Home, Skype, etc. It is high time I become more IST-ized.
No TA, But One Grader and One TI: Unfortunately, next semester I will not have a TA to help me along with my class. Instead I will have one undergraduate grader working for SRA 311 10 hours per week, and one teaching intern who will take part in class activities and maybe an occasional afternoon or evening event. I, personally, don’t know how I will function without a TA, but I suspose things should be ok if my grader and TI are good (which I suspect they will be).
Better Class Time and Better Room: I am by no means a morning person. This is why I am happy about having a class that begins at 11:15am instead of 9:45am. On top of that, I am pleased to find that I am moving my class from the worst room in the IST building (IST 205 with annoying tabletop Macintosh computers) to the best room in the building (IST 206 with PC laptops). I suppose that, in some may, the later class time and better room make up for the college taking away my teaching assistant.
My Blog About the Class: Next semester I intend to make more efficient use of my blog for recapping course content. The way I am going to do this, though, is to point to relevant reference materials to support learning instead of writing lecture notes from scratch after each class session (my tendency to write a lot about each lecture acted more as a deterrent to writing than I intended). I will also start to tweet about the class and integrate some other types of web communication technology (RSS feeds?).
Future Challenges
Two Sections of SRA 311: In Spring 2009, there will be two offerings of SRA 311. One of these (the larger one) will be taught by me on Tuesday/Thursday mornings. A smaller section of SRA 311 will be taught Tuesday/Thursday afternoons by Professor Dave Mudgett of IST 230-fame. What this means is that we have to coordinate our class schedules, or at least align the learning objectives for our courses.
Cybertorium in Fall 2009: Beginning Fall 2009, my understanding is that SRA 311 will be moving to the infamous IST Cybertorium, a 150+ person computer-ridden ampitheatre not at all designed for fTf (face to face). Fortunately (and by my request) the schedule should be such that the class will meet twice per week for 50-minutes in the Cybertorium, and one more time per week in smaller groups at a location somewhere away from the IST building. The astute reader will see here that I am making lecture more of just that – a lecture. My intent is to move all in-class activities to the recitation sections where students can spend an entire hour applying the things they learned in the lessons prior. To accomodate this move to the cybertorium, I will be, in a small way, treating my Spring 2008 course as a cybertorium class, focusing mostly on lectures with fewer in-class exercises. But when in-class exercises do occur, they will be extensive.
Guest Speakers: For some reason, I feel some pressure to recruit a guest speaker or two this semester. A challenge for me is to identify who would be a good speaker that can (a) entertain the students, (b) convey useful real-world insights, and (c) align his message with the learning objectives I would otherwise have to address were I giving the lecture. Any thoughts?
Epilogue
I invite interested readers to make suggestions regarding what to include, what to stress, what to omit, and what to test. I will be posting a revised syllabus to this blog within the next two weeks. Note that I reserve the right to add more to this post (either directly or via comment) as I things come to mind.
Last week’s SRA 311 (Risk Management: Assessment and Mitigation) lectures focused on all things vulnerability. As defined in a much earlier lecture, the general qualitative (albeit probabilistic) expression for risk, R, is as follows:
R = {<e,p,o>} (1)
where e is one of among many types of initiating events, o is one of many outcomes of concern, and the probability p is the joint probability of both e and o occurring, or:
p = Pr(e,o) = Pr(e)Pr(o|e) (2a)
= Pr(o)Pr(e|o) (2b)
where Pr(e) is the probability of event and Pr(o|e) is the vulnerability to realizing outcome o given e has occurred. The use of the curly braces “{” and “}” in Eq. 1 implies that risk is the complete set of triplets for all possible combinations of e and o for a given situation (i.e., cross product of E and O, where E and O are the sets of all events and outcomes, respectively). And it must be kept in mind that the scope of the risk analysis constrains how e, o, and p are assessed.
Common, but Apparently Different, Expressions for Risk. Now, the experienced risk practitioner might question why Eqs. 1 and 2 look so dramatically different than the prototypical formula for risk:
Risk = Threat × Vulnerability × Consequence (3)
As it turns out, the “colloquial” expression for risk in Eq. 3 is identical to the expression I put forward in Eq. 1. To see this, let’s examine what Eq. 3 actually says. The “colloquial” expression for risk states that risk is the combination of threat and vulnerability and consequence, that is, the “×” denotes the cartesian product and not the more restrictive arithmetic product. Equation 1 says the same thing, namely that risk is the combination of all pairings of initiating events (e.g., threats), outcomes (e.g., consequences), and the probability that binds them. This probability, according to Eq. 2a, is largely a function of the system states that enable event e to result in outcome o (e.g., vulnerabilities). Again, Eqs. 1 and 3 are essentially the same, although I must admit that it is much easier to explain Eq. 3 to decision makers than it is to even come close to explaining Eq. 1.
So what about the commonly accepted definition of risk as “probability times consequence?” This simplification of risk is actually equivalent to Eq. 2a under certain assumptions. Equation 2a provides a means for expressing the full probability distribution over the space of potential outcomes. If the outcomes o are expressed on a cardinal or ratio scale, then one can find the expected value of the vulnerability term, where the expected value is actually the expected loss given occurrence of an event (see any basic textbook on probability and statistics to see how this is done). With vulnerability expressed as expected loss, Eq. 2a reduces to a probability times a consequence. Alternatively, one can decompose the vulnerability term into two distinct probabilities as follows:
Pr(o|e) = Pr(o|e,s)Pr(s|e) (4)
where Pr(s|e) is the probability of adversary success given attack (obviously this value is one when natural events are considered) the Pr(o|e,s) is the probability of an outcome given a successful attack. Here, one can find the expected value of the outcome probability Pr(o|e,s) to arrive at a value for expected loss given a successful attack. Again, Eq. 2a reduces to a probability times a consequence, albeit this time the probability is the product Pr(e)Pr(s|e) and the consequence is the expected loss given adversary success. In fact, this is the form of Eq. 2a that is most often used in probabilistic security risk methods. But it is important to note that Eq. 4 is just one version of Eq. 2a, and that there are many others that are simpler or more complex depending on the needs of the decision maker. But in the end, Eq. 2 (both a and b) is the most general conceptual expression for risk.
Vulnerability As Notion and Vulnerability as Measure. As a notion, Professor Yacov Haimes at the University of Virginia defined vulnerability as “the manifestation of the inherent states of a system that can be exploited to adversely affect that system” (see “On the Definition of Vulnerabilities in Measuring Risks to Infrastructures” by Yacov Haimes, Risk Analysis, Vol. 26, No. 2, pp. 293-296 (2006), doi:10.1111/j.1539-6924.2006.00755.x). According to this definition, a system is said to be vulnerable if there exists a combination of system states that renders it susceptible to adverse effects (outcomes) arising from a particular exploit (initiating event). Consistent with this definition is the measure of vulnerability according to the term Pr(o|e). This vulnerability term can be read as follows: “vulnerability is expressed as the probability of a given outcome following the occurrence of a specified event.” This probability is shaped by the performance of the system under the stress imposed on it by the initiating event, where higher values of this probability for a given combination of (e,o) indicate a greater susceptibility to harm of loss.
A more generic definition for vulnerability was offered in the paper “Vulnerability and Risk: Some Thoughts from a Political and Policy Perspective” by Sarewitz et al and published in Risk Analysis, Vol. 23, No. 4, pp. 805-810 (2003) (required reading for a large fraction of the class): “vulnerability is the inherent characteristics of a system that create the potential for loss.” While similar to the definition posited by Haimes in the context of protecting infrastructures against acts of terrorism, the Sarewitz definition is more generic in that it asserts that vulnerability creates risk (where risk is defined as, in the more restrictive sense of security, as the potential for harm). In fact, Sarewitz et al. emphasizes that “understanding and reducing vulnerabilities does not demand accurate predictions of the incidence of’ events. This statement is 100% consistent with Eq. 2 in that vulnerability reduction yields a reduction in risk even in the probability of event remains unchanged. For security managers this point is particularly important given the fact that it is insanely difficult to express likeliness of adversary actions in quantitative form. Perhaps it is no surprise that vulnerability assessment is the prime focus of a security professional’s career, where the meager threat assessment (i.e., event likeliness assessments) are then used to help prioritize vulnerabilities for management attention. Risk management, then, examines the actions taken by security practitioners to reduce the vulnerability for those event/outcome pairs that make management most nervous.
Extreme Events. Sarewitz et al. also made another point I think is very important: “extreme events are created by context.” I wrote at length about this point in a previous post on natural perils, natural hazards, and natural disasters. In themselves, events are not disasters; for example, a hurricane is not labeled a disaster until it has affected some system. Before then, a hurricane is simply an event that one might label as a peril or hazard. The label “disaster” is assigned only to events that have occurred and wrought a significant toll on the interests of an individual or group of individuals. An extreme event is a game changer event, and much like a disaster is one that disturbs the affected system enough to change its configuration with respect to its pre-event state (e.g., population redistribution, new reactive policies, etc.). It makes no sense to assess the vulnerability with respect to disastrous events because the mere label of disaster implies significant vulnerability. Whether or not an event becomes a disaster depends on the magnitude of the vulnerability to outcomes one labels as disastrous given an event has occurred. That is, the context of the matter determines which outcomes are disastrous and which are not, and the vulnerability assessment then can produce insights into the potential for disaster in the face of a triggering event.
How Vulnerability Assessment Is Done. Unlike previous lectures where I was able to provide guidance on constructing complete sets of events and outcomes, I could not offer my students similar tools for doing vulnerability assessment. Why? Because vulnerability assessments fall under the category of messy problems. While it may be straightforward to articulate potential causes of harm and define a set of undesirable consequences, it is not a trivial matter to make defensible statements about the probability that an event will lead to a particular outcome. Such statements insist that the analyst possess intimate knowledge of all aspects of the system under study, to include its security system, structural configuration, and response and recovery capabilities. Even if you reduce the vulnerability problem into separable components (e.g., protection vulnerability and response vulnerability such as is described in a paper I coauthored), the level of knowledge required to do a vulnerability analysis is quite extensive. Yet, people manage to do vulnerability assessment anyway. How do they do it?
Well, if one appeals to the science of Naturalistic Decision Making, meaningful vulnerability assessments insist that that the vulnerability assessor has command over the two major sources of power: pattern recognition and mental simulation (I wrote something about this in a recent post on the (very tentative) McGill descriptive vulnerability assessment model). Pattern recognition, a power that arrives at only through experience, enables an individual to quickly pick out the most significant environmental cues relevant for a given problem and use these cues to assess the degree to which the environment is similar to other situations from his experience. In the event of a match (the likeliness of which increases with more personal experience), an individual uses his or her mental simulation power to quickly conduct thought experiments that “challenge” the environment and predict how it will respond to different initiating events. (notice my use of the word “quickly”: have you ever seen a former special forces solider do a vulnerability assessment? The more experienced the soldier is, the more quickly he or she can do a vulnerability assessment that means something). I suspect that this simulation process for vulnerability assessment is iterative in that one starts with an outcome, backs out plausible events that might yield that outcome, reappraise vulnerability with respect to each identified initiating event, and so on. But in the end the breadth and depth of the assessment is highly sensitive to the experience, objectivity, and biases held by the assessor.
But here is my challenge – I must teach vulnerability assessment to individuals with a minimum amount of background knowledge. How can I do this? The solution lies in the simple fact that when under pressure to produce answers, the lack of knowledge to render a defensible judgment is typically compensated for by bias, gut feel, and guesses. One way to enable defensible analyses is to provide students with a wide array of structured analytic techniques aimed at alleviating all those aspects of reasoning that are detrimental to the end product (much like the way the intelligence community does it). This is my focus of Part II (risk assessment) of my course – to provide a suite of techniques to help less experienced risk assessors properly structure their thinking so as to make sense of a particular situation and explicitly identify all uncertainties.
An Exercise on Vulnerability Assessment. To highlight the difficulties in actually doing a vulnerability assessment, I had my students spend 30 minutes of the second lecture assessing the vulnerability of Penn State campus (University Park) to disaster (note that much like most questions encountered in practice, I deliberately kept the question vague). This exercise insisted on brainstorming what types of events would be considered disastrous, then identifying a spectrum of different causes for each type of disaster. I provided no techniques for doing this in attempt to see how my students would reason through the problem. The responses were mixed – what constituted a disaster varied among student groups, as well what types of events could causes disaster. Perhaps this is because not a single group put themselves in the shoes of a campus decision maker; rather, each group adopted a personal view of disasters and their causes. As I emphasized in class, analysis done in this manner imposes the personal biases of the analyst on a problem whose answers would inform a decision maker that might have a different opinion of what a disaster is. The first step in any risk analysis is to know your customer well enough so as to properly frame the associated questions. Overall the exercise went well, and provided me with good insight into how to proceed with part II of the course.
My Take on the Lecture
As a whole, I think I could have done better with this week of lectures. For one, I assumed that the students had more background knowledge in probability than they really had. In hindsight, I should have incorporated basic concepts from probability theory throughout the discussion of vulnerability. I will definitely try this approach in next semester’s offering of SRA 311. However, since vulnerability is a conditional probability, I am now forced to restructure the syllabus to start with a discussion on basic probability before getting into Bayes’ rule. Essentially, this means I need to start with event likeliness (the topic of lecture 9) before lecturing on vulnerability.
A second thing I noticed was that I really talk too much, particularly on the topic of vulnerability assessment. While this isn’t always a problem, the topic of vulnerability assessment is dry as a bone unless one already has some experience doing it. In attempt to liven the discussion up, I intend next semester to incorporate more in-class exercises to flex students’ neural muscles on the topic. Some ideas I have in mind include online worksheets that ask students to make general statements of vulnerability for a variety of high-level scenarios, another case study pegged to some current event (the recent bombing in Pakistan, as horrible as it was, would have made for a good case study focused on what makes a system vulnerable), and so on. Feel free to share your thoughts or ideas, if you have any.
This set of lectures (Lecture 5 and Lecture 6) for this week centered on the first two, and perhaps most important, phases of the risk assessment process:
Identifying plausible initiating events
Identifying plausible outcomes of concern
The products of these activities are lists of events and outcomes. In the ideal world, these lists should be as complete as possible, and the elements comprising the list should be as distinct as possible. But how do we do this properly? And what if the lists are not complete? Basically, my challenge for the week was to teach the basic concepts of classical set theory without getting too much into the mathematical nomenclature traditionally associated with the subject.
My first lecture for the week began with a review of selected concepts from previous classes. First and foremost, I reiterated the following 5 points:
The working definition for risk in this class is uncertainty about future events of interest. This is not necessarily the only definition for risk out there, but it is a good working definition that will get me and the students through this course
Risk and certainty don’t mix (that is, risk = uncertainty, and if there is certainty there is no risk)
Risk is about the future, not the past (it makes no sense to speak of risks in terms of loss already realized from past events, that is, unless the goal is to perform a “forensic risk analysis.” But even then you would be simulating the future from the point of view of the past)
Risk analysis does NOT prescribe decisions
Next I restated the six questions of risk, discussed the risk triplet R = <e,p,o> and the mathematical expansion of p = Pr(e,o) = Pr(e)Pr(o|e), and discussed how to decompose this mathematical expression into a series of sequential phases that enable its assessment. If one thinks carefully about this expression for risk, one might deduce that the first thing to do is identify the set of plausible initiating events, proceed to articulate outcomes of concern, then go on to assess the likeliness of events and likeliness of outcomes given the occurrence of each event. This brought me to the theme of the week: how to come up with e and o.
Let me stress the following point before I proceed: it is very important to spend a good amount of time coming up with events e and outcomes o. Defender ignorance is the most significant contributor to vulnerability and risk, particularly if one is concerned about risks stemming from the actions of criminals, terrorists, and other malicious humans. As Cynthia Grabo points out in her book Anticipating Surprise: Analysis for Strategic Warning (available for free download via NDIC College Press):
Strategic surprise is “the unilateral advantage gained by the introduction of a new weapon (or by the use of a known weapon in an innovative way) in conflict against an adversary who is … unaware of its existence …”
Yes, I do admit that the definition of strategic surprise is narrowly focused on military (and asymmetric warfare) style conflict. But with a slight adjustment in language to accommodate naturally-occurring and non-malicious anthropic events, we could easily generalize the concept of surprise to include a wide array of environmental and operational hazards. But the point still holds – if the defender does not know the things that can happen that could lead to undesirable outcomes, the only savior is chance that the event does not occur, that the bad guy goofs, or that some measure that addresses a known hazard is also effective against one that is unknown.
At the conclusion of my Tuesday lecture and again at the start of Thursday’s lecture I displayed a slide of key words (similar to a tag cloud) spanning the current topics. My goal was to ensure that all words were at least familiar to the students after hearing me speak for an hour. I asked whether all words had meaning to the students, and if not, to let me know which ones did not so that I could further clarify. These two “keyword clouds” are shown below (yellow is lecture 5 and blue is lecture 6):
Venn Diagram. A venn diagram (named after the founder of the concept, John Venn, whose work is commemorated by the stained-glass window shown below) is a tool for visualizing sets, the elements contained in sets (and those that are not), and how these elements relate to one another. Typically, venn diagrams are drawn to be rectangular, but in practice they can take on any shape in any dimension. The only requirement is that the shape be closed to enable one to clearly state what elements are contained in the set, and which elements are not contained in the set. In terms of future events, a venn diagram is often said to represent the universe of possibilities.
Possibility vs. Probability. The first point of the day centered on the distinction between the notions of possibility and probability. An event (such as “truck bomb attack”) is possible if it can occur. That is, a possible event is one whose probability is greater than zero (obviously constrained by an upper limit of one). However, under the notion of possibility, there is no indication of what the probability of the event is save for it being bounded by a lower limit (some infinitesimally-small value greater than zero in this case) and an upper limit (one in this case). In contrast, an impossible event is one whose probability is exactly zero. Probability is the measure of likeliness attached to an event, where a value of zero corresponds to the impossible event, a value of one means the event is certain, and values in between zero and one express the chances that the event will occur relative to others in the same sample space (e.g., the same universe, Venn diagram, etc.). In terms of a Venn Diagram, an event is possible if it is contained in the Venn Diagram. The probability of this possible event is expressed by just how much space the event takes up. A more detailed discussion of possibility and probability will be provided in later lectures. But for now keep in mind that just because an event is possible does not mean it is probable; all that the word possible suggests is that the probability of the event is not zero.
Mutually Exclusive. Two events are said to be mutually exclusive if the occurrence of one precludes the occurrence of the other in all respects. Accordingly, mutually exclusive events are not independent. For two events to be independent, the occurrence of one event should have no bearing on the possibility or probability of the other event occurring. Below is an image of a universe “S” consisting of three mutually exclusive events “A,” “B,” and “Not A or B” (the latter event, call it “C,” represents the balance of white space after accounting for “A” and “B”).
Collectively Exhaustive. The next point centered on the concept of exhaustiveness. Given a list (or set) of events answering the question “what can go wrong,” the entire collection of these events are collectively exhaustive if it includes all possibilities. That is, the set consists of all events that are possible. It is important to note that it is ALWAYS POSSIBLE to specify a collectively exhaustive set provided that one includes the residual hypothesis. For example, let’s say that one comes up with three possibilities for the question “what can go wrong” labeled as “A,” “B,” and “C.” In most practical situations, it is conceivable that there are also events “D,” “E,” “F,” and so on that could also occur, but for whatever reason we cannot articulate specifically the nature of these events. If we are honest with ourselves, then we would explicitly admit that set consisting of events “A,” “B,” and “C,” is not collectively exhaustive since it does not account for events beyond our ability to describe. Fortunately, we can “close the set” by including a residual event, call it “R”, that accounts for all events other than “A,” “B,” or “C” (that is, R = NOT(A or B or C)). Together, “A,” “B,” “C,” and “R” represent an exhaustive set. Below shows an exhaustive set “S” with events “A” and “B” that together span the entire set of possibilities (note that event “C” may exists that partially overlaps with “A” and “B,” but this event would not penetrate the boundaries of this venn diagram).
The Residual Event. But there is a problem with the residual event. Given that the residual event lacks specific definition, its presence makes it impossible to assign meaningful probabilities to the stated events “A,” “B,” and “C.” This can be illustrated as follows. According to the second axiom of probability theory, the probability of “A” occurring (or Pr(A) using shorthand notation) is equal to one minus the probability of “Not A” (or 1 – Pr(~A), where the “~” means “Not”). In compact form, Pr(A) = 1 – Pr(~A). According to this expression, any statement about the probability of “A” also makes a statement about “Not A.” That is, the second axiom highlights the implications of our opinions of the complementary event “Not A” given a statement we make about “A” itself. But keep in mind that the event “Not A” really means that “B or C or something else contained in R” will occur. So, if we make a statement about “A,” we are also making a statement about an event we know nothing about. This obviously doesn’t make any sense, which leads me to suggest that inclusion of the residual event does not permit assessment of meaningful probabilities. Now let’s look at this situation a different way. We could, in theory, make a statement about “R” which would yield a complementary statement about the collection of events “A,” “B,” or “C.” But we would not do this because we know nothing about “R,” and accordingly we know nothing about the likeliness of “R” (Pr(R)). If Pr(R) is unspecified, then it can conceivably take on any value between zero and one, and according to the second axiom, Pr(A or B or C) can too take on any value between zero and one. Letting the probability of an event take on any value between zero and one is a vacuous or empty statement of likeliness.
Conditionally Exhaustive. Fortunately, there is a way to alleviate the woes of attributed to the residual event. If an analysts makes the explicit assumption to NOT include the residual event, the focus of the analysis would then be on the set consisting solely of events “A,” “B,” and “C.” While this set is not inclusive of all possibilities, it is inclusive of all known possibilities. This means that while the set is not collectively exhaustive, it is conditionally exhaustive. That is, the set of events is exhaustive only under the condition that we deliberate omit the possibility of the unknown event. In probability notation, when we say Pr(A) under this assumption of conditionally exhaustiveness, we are really saying Pr(A) = Pr(A|~R), where “~R” is the event the residual event is not a possibility. Now we can make statements about probability for the events “A,” “B,” and “C,” but the analyst must bear in mind that any such statement is still under the conditionally exhaustive assumption, and thus is only valid given that no other event is possible.
Open versus Closed World Assumptions. By making the conditionally exhaustive assumption, one is deliberately closing the world to all other events that cannot be articulated or judged as impossible. Thus, the conditionally exhaustive assumption is also known as the closed world assumption. In contrast, the open world assumption admits the possibility of unknown events. The difference between an open and closed world rests on the meaning of exhaustiveness under both assumptions. Exhaustiveness in an open world must consider both the articulated events and the residual event. Exhaustiveness in a closed world considers only those events that can be articulated. Thus, the difference between the closed world and open world is simply whether or not one includes the residual event. And of course, from the previous discussion, probabilistic statements are meaningful only under the closed-world assumption. Outside the closed-world assumption, other theories are necessary (such as possibility theory) if one desires to quantify the uncertainty attributed to the occurrence of future events.
The readings for these lectures consisted of a variety of articles on different types of novel attack types, including the use of lasers, high-powered microwaves, forest fires, and food. Citations for these papers are available via my blog post from mid may on Emerging Terrorist Threats. Basically, these papers were used to provide insight into the types of scenarios that might otherwise fall under the category of unknown events (or residual scenario) due to the fact that many have not previously occurred (at least knowingly). The brief discussion of these threats veered off into a discussion of other novel attacks, to include the man using a homemade armored bulldozer to terrorize a small town.
Striving for MECE. MECE, or “Mutually Exclusive, Collectively Exhaustive” is an acronym used in some analytic circles to express the analyst’s goal of establishing as complete as set of hypotheses as possible for a particular question at issue. While this may be the holy grail of analysis, a practical re-write of this acronym in light of the discussion thus far is “Mutually Exclusive, Conditionally Exhaustive” with the caveat that the analyst must make explicit the assumption of deliberately excluding the residual hypothesis from the analysis of event likeliness. Of course we should not ignore the possibility of unknown, “black swan”-type events, but for the purposes of establishing relative likeliness, it is essential to put them aside until such hypotheses can be made more explicit. After doing an analysis under the “conditionally exhaustive” assumption, one way to hedge the analysis is to make a statement such as “We cannot rule out the possibility of alternative futures not addressed in this study.” Yes, such a statement may make the analysis appear weaker, but not including such a statement might convey a unwarranted sense of certainty in the impossibility of events contained in the residual hypothesis.
Combinations and Possibility Trees. In the event that one must consider possible combinations of events or outcomes, possibility trees facilitate the process of generating an exhaustive set of possibilities. A possibility tree is a technique for visualizing combinations of events without specifying the probabilities, temporal order, or logical relationships of events. If done properly, a possibility tree will construct an exhaustive set of event and outcome combinations. For example, consider three outcome types of concern to the information security professional: confidentiality, integrity, and availability.
Let C be the event “loss of confidentiality” and ~C represent “Not C.”
Let I be the event “compromise of integrity” and ~I represent “Not I.”
Let A be the event “loss of availability” and ~A represent “Not A.”
A possibility tree can be used to construct an exhaustive set of event combinations as shown below, where each combination of events is shown in the the right-most column. Trying to explain how to do construct a possibility tree in words eludes me at the moment, so I will instead suggest checking out the book entitled Discrete Mathematics with Applications by Susanna Epp (ISBN: 0534359450).
Scenarios. A scenario is the combination of an initiating event and outcome. The initiating event can, in principle, be a single event (e.g., explosive attack) or combination of events (e.g., two simultaneous explosive attacks, one attack at the same time as a hurricane, etc.). Likewise, the outcome can be specified along a single dimension (e.g., confidentiality) or along multiple dimensions (e.g., confidentiality, integrity, and availability).
Cross Product. Given a set of events and outcomes, a complete set of event-outcome pairs (i.e., scenarios) can be obtained by systematically pairing an event with each outcome, then repeating the process for all other events. This operation is known as the cross product of events e and outcomes o. For example, consider the case of three initiating events “A,” “B,” and “C” and three outcomes “X,” “Y,” and “Z.” The cross product of events e and outcomes o, denoted as e × o, yields the following scenarios: (A,X), (A,Y), (A,Z), (B,X), (B,Y), (B,Z), (C,X), (C,Y), and (C,Z). Note that the cross-product generates a complete set of scenarios without considering whether the combination is, in fact, possible. For example, let’s say “A” = “cyber attack” and “X” = “structural damage.” The combination (A,X) expresses the scenario “cyber attack results in structural damage.” Now perhaps this is a feasible pairing in some circumstances, but in most situations I can think of it is a practically impossible combination. Nonetheless, the cross-product operation would generate this scenario, thus leaving it up to the analyst to leverage all available evidence to judge such a combination as impossible (or in weaker form, “practically impossible”).
Theory of Scenario Structuring. The Theory of Scenario Structuring, or TSS, is a “theory” proposed by Dr. Stan Kaplan and colleagues, and described in his 1999 monograph entitled New Tools for Failure and Risk Analysis: An Introduction to Anticipatory Failure Determination (AFD) and the Theory of Scenario Structuring (ISBN: 1-928747-05-1). TSS was also later described in relation to Hierarchical Holographic Modeling in the paper entitled “Fitting Hierarchical Holographic Modeling into the Theory of Scenario Structuring and a Resulting Refinement to the Quantitative Definition of Risk” by Kaplan et al. and published in the journal Risk Analysis, Vol. 21, No. 5, pp. 807- (doi:10.1111/0272-4332.215153). (side note: AFD and HHM will be discussed in greater detail in future lectures). TSS assumes the existence of a “success” or “as planned” scenario that describes the future as it should or is supposed to unfold. The success scenario is labeled S0 and is shown as a black curve in the figure below. Deviations in the “success” scenario may be beneficial (in the green zone above the S0 curve) or detrimental (in the red zone below the S0 curve). Since the focus of this course is on security, we restrict our attention to those scenarios in the red zone. At some point in time, an initiating event occurs (the little explosion graphic) that disturbs the system in a manner that yields undesirable consequences. As events unfold, the situation may improve or get worse; this can be observed at branch points, where an upward branch means improvement and downward branch means detriment. Each branch point creates an additional partition (i.e., division) of the outcome space, until ultimately there are a complete set of end states (consequence states).
To emphasize the concept of set partitioning, I ran through a number of examples a shown in the figures below. Note that I leveraged MS PowerPoint’s custom animation feature to systematically partition the event space in a manner that preserves its exhaustiveness
I was also careful to emphasize that one could quickly get carried away with partitioning sets as shown in the sequence below. The top level event is “All Possible Scenarios” which can be divided as follows:
Further partitioning the event “A Malicious Attack Occurs” gives:
Further partitioning “A Malicious Explosive Attack Occurs Against Something that Matters to Me” gives:
where “AP#” reads “Attack Profile #,” where an attack profile describes a specific combination of delivery system and intrusion path leading to the asset (chemical plant, in this case). After all this partitioning, the event “AP8″ shown above in relation to all other possible events in the universe is shown below (circled in yellow). If we proceeding to partition all events in this same manner, we would be left with hundreds of events, each of which needing full consideration in order to assess meaningful estimates of probability. Quite a challenge indeed.
The goal with partitioning sets is to do so in a manner that yields sufficient resolution to answer the question at issue yet is careful to not overwhelm the resources available to do analysis. Fine resolution costs more to analyze, but may not yield any additional insight than a slightly coarser model (e.g., “an attack will occur at 12:01:32 on Tuesday afternoon 26-feet off the northwest corner of Main Street and 45th Avenue with a 324.3 kg ANFO explosive (rho=0.631) packed in a blue 1996 Chevrolet minivan driven by two adults, one 24 yrs, 5 mos, the other bearded …”). In contrast, coarse resolution is cheap to analyze, but the result may be too general or abstract (e.g., “an attack will happen”) to be useful.
In-Class Exercise. To get the class thinking about all of the above, I had students in groups participate the following exercise:
You are the lead security risk manager for the National Spy Office (NSO), an organization that deals with analytical and operational matters at all classification levels. You have been asked by DDNSO to evaluate the proposed policy of allowing personal cell-phones within secured areas on the basis of risk. Establish the security context(s) and articulate what can go wrong and what outcomes are of concern.
The students, as a class, impressed me quite a bit with this exercise. While each group (among about 8-10 groups) came up with their own small set of plausible events and outcomes of concern, as a class the students brainstormed everything including disguising bad things as a cell phone (bomb, jammer, etc.), leveraging the cell-phones capabilities to collect information (camera, audio), using advanced phones as a web-server to break into classified networks, using the GPS capabilities to track spies, using the phones to enable command and control within the organization, lost productivity due to cell phone usage at work, hazardous cell phone accidents (e.g., battery explosions), using an ad hoc network of bluetooth phones to create a surveillance net, and so on. After this exercise, I highlighted to the students that, in some cases such as this one, the information they generated was sufficient to inform decisions on what can be done to improve security. That is, this case is one where assessing the likeliness of alternative events and the vulnerability to outcomes isn’t necessary to generate a prioritized list of scenarios in order to come up with low-cost, yet effective, countermeasure ideas. Such proposals included:
Having employees turn off their cell phones before they enter the secured area
Having employees turn the phones on to show the guards that they are, in fact, cell phones and not something else
Allowing only low-tech cell phones, such as those without cameras, GPS, Internet
Allowing only very senior people to carry cell phones into the building
Having people remove the batteries from the phone before entering the secured area
Creating layers of security, where the outer layer allows cell phones and the inner layer does not
Purchasing approved cell phones for employees
Installing shielding that prevent cellular signals from entering or leaving the secured area
Again, this was a fun and impressive exercise. However, looking back on this exercise, I should have reminded the students to strive for MECE – many forgot to include the residual hypothesis.
Things I Forgot To Include
Since the two lectures for the week was supposed to cover set theory (albeit in disguise), I should have included the following topics, at least briefly:
The intersection of two events “A” and “B” produces an event, call it “C,” defined as “C = A and B.” For example, let “A” be “explosive attack occurs” and let “B” be “hurricane occurs.” The event “C” is then “an explosive attack and a hurricane occurs.” The probability of the joint event “C”, Pr(C), is then equal to Pr(A,B) = Pr(A)Pr(B|A) = Pr(B)Pr(A|B). If the two events “A” and “B” are independent, then Pr(A|B)=Pr(A) and Pr(B|A)=Pr(B). This gives Pr(C)=Pr(A)Pr(B).
The union of two events “A” and “B” produces an event, call it “D,” defined as “D = A or B.” For example, let “A” be “explosive attack occurs” and let “B” be “hurricane occurs.” The event “D” is then “an explosive attack or a hurricane occurs.” According to the third axiom of probability, the probability of “D,” Pr(D), is equal to the sum of the probabilities of “A” and “B” minus the intersection of “A” and “B.” Thus, Pr(D) = Pr(A) + Pr(B) – Pr(A,B). (note that Pr(A,B) = Pr(C) in the intersection discussion). If the two events “A” and “B” are mutually exclusive, then Pr(A|B)=Pr(B|A)=0, and Pr(D) = Pr(A) + Pr(B).
A set “D” is said to contain “A” if “A” is a subset of “D.” Conversely, “D” is a superset containing “A.” In the discussion of union above, “D = A or B,” thus single events “A” and “B” are subset elements of “D” since “D” contains both. The same cannot be said about “C,” since the intersection of two events “A” and “B” does not contain either unless “A” and “B” are equivalent. All of “A,” “B,” “C,” and “D,” are contained in the universe “S.”
The complement of the event “A” is simply the event “Not A.”
Final Impressions
I covered quite alot of ground this week, and I personally worry that the students might not have completely absorbed all of this content. My concerns would have been tempered had I felt assured that my students had adequate exposure to set theory and probability theory prior to enrolling in my course. But alas the system does not appear to be working in my favor, thus leaving it up to me to provide sufficient background instruction to enable good risk analysis. Fortunately, my impression is that the students at least kind-of get it, and the good news is that we have 25 lectures to go before the semester is done. All I really want is for my students to develop a “risk intuition” and knowledge of what is required of a risk analysis (plus critical thinking skills, of course). From this point of view I believe we are on the right track.
Next week’s topic: vulnerability (this should be a good one).
The fourth lecture of my SRA 311 (Risk Management: Assessment and Mitigation) class was by far my favorite this semester. The lecture topic was “The Six Questions of Risk,” and centered on (1) the three risk assessment questions posed by Dr. Stan Kaplan and Dr. B. John Garrick in their 1981 research paper “On the Quantitative Definition of Risk” (Risk Analysis, Vol. 1, No. 1, pp. 11-27, doi: 10.1111/j.1539-6924.1981.tb01350.x), and (2) the three risk management questions offered by Professor Yacov Haimes (UVA) in his 1991 editorial “Total Risk Management” (Risk Analysis, Vol. 11, No. 2, pp. 169-171, doi: 10.1111/j.1539-6924.1991.tb00589.x). These “six questions” are as follows (in slightly revised form relative to what I presented in class):
What can happen?
How likely is it to happen?
What are the consequences if it does?
What can be done?
What options are available and what are their benefits, costs, and risks?
What are the impacts of current management decisions on future options?
The first three questions define the the scope of risk assessment and the second three questions bound the scope of risk management. In general, risk analysis should focus only on answering these questions, which as a whole is often a very difficult and costly task in terms of time and analytic resources. There is a seventh question (or perhaps a few more than that) that is risk-relevant but not included in the list of questions above. That question is “are the risks acceptable?,” a legitimate inquiry that would come up somewhere between questions 3 and 5 above. In order for a risk analyst to answer this question, he or she must impose subjective value judgments and preferences on behalf of their client. As discussed in one of the articles from my previous lecture 3 (the one by Pate-Cornell), risk analysts are, not paid to pass judgment on the risks and options for risk reduction; rather, risk analysts are paid to be as objective about the nature of a particular problem and all its uncertainties as possible without expressing preference. Nor do risk analysts seek to prescribe decisions (just like the way intelligence analysis should not prescribe decisions). Decision analysts, on the other hand, take risk analysis the next step by characterizing decision maker preferences in attempt to identify the optimal option to a given problem. Perhaps an individual wearing the hats of both risk and decision analyst might add this seventh question to the mix; but since the focus of my course is squarely on risk analysis, it does not factor into the scope of our analyses.
To begin answering the six questions of risk, one has to establish a security context (lecture 2) and clearly define the scope of the analysis (lecture 3). Prior to introducing the six questions of risk I thoroughly reviewed the techniques for these items discussed in the previous two lectures and spent 30-minutes conducting group exercises where students had to articulate the scope of a pandemic-flu risk assessment from multiple stakeholder perspectives (e.g., CDC, pizza shop, parent, Penn State president, bus driver). (this was a good exercise, but I found that I could have perhaps been a bit more organized in its administration).
If one examines the first three questions, one could express the concept of risk (R) as the set of ordered triplets of the form:
R = <s,p,c>
where R is risk, s is the scenario (i.e., one answer to the first question), p is the probability of the scenario (i.e., answer to the second question), and c is the consequences given the scenario were to occur (i.e., asnwer to the third question). According to this “classical risk triplet,” the scenario articulates a sequence of events from cause to some end state, the probability expresses the quantitative likeliness of this scenario occurring all things considered, and the consequence expresses the valuation of direct and indirect consequences associated with this end state (which may also be uncertain). For example, a scenario might be “explosive attack against Building 1 that causes major structural damage.” The probability, then, expresses the likeliness of this scenario considering the likeliness of the initiating event, quality of the explosive material and packaging, position of detonation, and the fragility of the target. The consequence places value on the result of damage, to include lost property, number of individuals injured or killed, business disruption, and so on.
NOTE: regardless of whether numbers are used, the concept of the risk triplet still holds. The key difference is that the probability will be couched in terms of the more generic notion of likeliness using phrases such as “Words of Estimative Probability” (WEPs) arrived at through reason and judgment, and consequence will be expressed in terms of outcome narratives (also see here and here for more information on WEPs. Of course, these descriptions can later be converted to numbers as needed, but in the end the numbers are less useful than the knowledge used to generate them).
A more modern take on the “classical” risk triplet above (taken from my dissertation) is what I call the “modified” risk triplet (using slightly revised notation from my lecture):
R = <e,p,o>
which follows from the revised set of questions:
What initiating events are plausible? (e)
What outcomes are of concern? (o)
How likely are the different combinations of event and outcome? (p)
Regardless of whether one looks at the “classical” or “modified” triplet, risk, as the set of all ordered triples, the above quantitative definition suggests that risk is “simply” the likeliness of alternative outcomes in light of a full complement of initiating events. If numbers are used, likeliness then takes the form of a probability distribution over the space of mutually exclusive, collectively exhaustive outcomes. This is consistent with the working definition of risk offered in lecture 2: risk is the uncertainty about future events of interest (though admittedly the working definition includes more than simply likeliness expressed over a space of articulated outcomes).
Let’s return to the “modified” triplet. Leveraging concepts of joint probability of two events expressed as the probability of one event times the condition probability of the other, the probabilty term in this expression in this expression can be expressed as follows:
Pr(e,o) = Pr(e)Pr(o|e)
where the notation Pr(e) and Pr(o|e) reads “probability of (initiating event) occurring” and “probability of (outcome) given the occurrence of (initiating event),” respectively. The first term Pr(e) is the probability that a specified initiating event will occur (over some time span), and the second term Pr(o|e) is the vulnerability to a specified outcome given the occurrence of a specified initiating event. Separate from its quantitative implications, this expression pretty much articulate the requisite phases of a risk assessment:
Identify a full suite of plausible initiating events (ei), striving for exhaustiveness to the maximum extent possible (which may require explicit assumptions to bound the scope). This phase might be called “initiating event identification.”
Identify a full suite of plausible outcomes of concern (oj), again striving for exhaustiveness to the maximum extent possible (which may require explicitly assumptions to bound the scope). This phase might be called “consequence assessment,” but would be of a different character.
Assess the likeliness (or probability if you so desire) of each initiating event Pr(ei) using all available knowledge and information. This phase might be called “event likeliness assessment.”
Assess the likeliness of each outcome presuming the occurrence of each initiating event Pr(oj|ei) using all available knowledge and information. This phase might be called “vulnerability assessment” or “outcome likeliness assessment.”
Now what is nice about this development? For starters, the expression for risk and the process for its assessment has been derived from first principles (bottom-up) rather than from someone’s ideas on how to map their perceptions on what security risk analysis is to a mathematical formula (often arithmetic or logical) that seems right (top-down). (I suspect that many of the “formulas” for risk proposed or in current use actually were arrived at via top-down thinking, which is perhaps the reason why so many people find it easy to challenge their mathematical integrity). Second, regardless of whether one uses numbers to express likeliness, the expression offer guidance on how to clearly think through a risk analysis problem. It is true the devil is in the details, but in the end the four steps above will lead to knowledge that enables higher confidence statements about risk relative to those generated through other ad hoc analytic approaches.
The four items above – initiating event, outcomes of concern, event likeliness, and vulnerability – will each be addressed in turn over the course of the next four lectures (two weeks). Fortunately for my students, I intend to address these topics in a manner that assumes the minimal amount of background knowledge in set theory and probability theory. But that doesn’t mean my students are exempt from knowing this material; my job is to ensure that they do by the time they finish this course. (Note: Issues centered on the remaining three questions for risk management will begin in part III of the course, or after lecture 20).
For those unfamiliar with this description of risk, check out the website maintained by Peter Sandman. Dr. Sandman is a scholar on risk communication and risk perception, and has made a name for himself via the concept “Risk = Hazard + Outrage.” He has published some very interesting things, one of which can be found on my list of 100 books to review. A selection of his works is available electronically on his curriculum vitae.
Back to the formula “Risk = Hazard + Outrage”… This is not a mathematical formula in any strict sense of the word. Rather it is conceptual in nature, where the “risk” is defined by the objective nature of the “hazard” and augmented by the “outrage” felt by the individuals exposed to it. Through his many inquiries into how people perceive risk, Dr. Sandman put forward what I will call “Sandman’s First Law of Risk Communication” (though he states it may be the only law): Outrage, not hazard, drives reputation (I might prefer to replace the word “reputation” with “acceptability”). Basically, regardless of whether the hazard is objectively high or low, the outrage felt by the public or decision maker is what drives the degree of risk attached to a hazardous phenomenon. People tolerate objectively high hazard (e.g., driving) is the outrage is low, whereas people do not accept objectively low hazards (e.g., terrorism) if the outrage is high. The reputation of a risk manager or decision maker charged with making decisions that affect risk is more by how well they manage outrage than how they manage hazard. Based on this view, Dr. Sandman suggests ways for managing outrage.
Much of Dr. Sandman’s work emphasizes the point that the Society for Risk Analysis makes in their stated definition of risk:
Risk analysis is broadly defined to include risk assessment, risk characterization, risk communication, risk management, and policy relating to risk. Our interests include risks to human health and the environment, both built and natural. We consider threats from physical, chemical, and biological agents and from a variety of human activities as well as natural events. We analyze risks of concern to individuals, to public and private sector organizations, and to society at various geographic scales. Our membership is multidisciplinary and international.
That is, risk analysis includes risk assessment, risk management AND risk communication (among other topics). Based on Sandman’s work, it seems that though a high risk hazard can be managed so as to bring the risk down to a level acceptable to the risk manager, the strategies used to mitigate risk may be inadequate or insufficient unless accompanied by strategies to manage the outrage felt by those affected by the hazard. Sound risk policy must effectively manage risks assessed to be high, and must also manage the outrage felt by the targets of risk. For a risk analysis to be complete, it must look at an issue from all angles.
Now I leave it to you (and myself) to check out the rest of Dr. Sandman’s work to better understand his philosophy on risk communication and risk perception. This is interesting stuff, but keep in mind there is a lot more to read on this issue of risk communication and risk perception, in particular the following:
