I typically come across a few excellent quotes that really resonate with what I am presently thinking about whenever I go on a paper reading binge. Here are some interesting ones that I found recently:
Every year (or, perhaps, every day), some new industry or institution discovers that it, too, has a risk problem. It can, if it wishes, repeat the learning process that its predecessors have undergone. Or, it can attempt to short-circuit that process, and start with its product, namely the best available approaches to risk communication. – Baruch Fischhoff (1995)
Contemporary approaches to disaster reduction need to become more concerned with human-to-human relations, such as conflict resolution and consensus building among people, rather than human-to-nature relations. – Katsuya Yamori (2008)
References
Fischhoff, G. (1995). “Risk Perception and Communication Unplugged: Twenty Years of Process.” Risk Analysis, Vol. 15, No. 2, pp. 137-145 (doi link).
Yamori, K. (2008). “Narrative Mode of Thought in Disaster Reduction: A Crossroad for Narrative and Gaming Approaches.” in Sugiman, T., Gergen, K. J., Wagner, W. and Yamada, Y. eds. Meaning in Action: Constructions, Narratives and Representations. Springer, pp. 241-252 (doi link).
Lecture 11 was really not a lecture at all. Instead we had the luxury of hosting recruiters from the National Security Agency. Unfortunately, I was out of town on travel to a conference in New York City (the International Studies Association Annual Convention). But my understanding is that the NSA reps provided a good overview of the agency and career opportunities. This is good since many of our students in IST desire to start their careers in the Intelligence Community, with NSA often being a top choice among the variety of alternative agencies.
Lecture 5 was one of my favorites. The topic was structured brainstorming, in particular the divergent/convergent thinking technique described in both the CIA and DIA analytic tradecracft primers (both of which are unclassified, and can be obtained by joining IAFIE, contacting the public relations offices of the respective agencies, etc.). I teach structured brainstorming in my risk analysis course because, as often cited by risk scholars, the first step in any risk analysis is to imagine (read “brainstorm”) answers to the question “what can go wrong?”
But before we got into the meat of lecture 5, we began class with a short quiz and a discussion of the day’s reading. The paper for today was entitled “The Case for ‘Risk Awareness’” by Stevyn Gibson (Security Journal, Volume 16, pp. 55-64, doi: 10.1057/palgrave.sj.8340140). As one might tell from the title and my preface to this post, the theme for the week is combating ignorance-induced vulnerability (which I argue is one of the biggest contributors to a person’s risk exposure). The quiz asked for the purpose of Gibson’s article (“purpose” being one of the eight elements of though) and sought answers to five multiple choice questions focused on relevant aspects from set theory (e.g., what the word “possible means,” Venn diagrams, conditional exhaustiveness, and the distinction between open and closed-world assumptions).
Moreover, consistent with this week’s theme of creating risk awareness, I showed off an “interesting” book that took the idea of creating risk awareness to the extreme. The title of the book was An Introduction to Planetary Defense: A Study of Modern Warfare Applied to Extra-Terrestrial Invasion by Travis S. Taylor (a.k.a. “Doc” Travis) and collaborators (2006, ISBN: 978-1581124477). An interesting book, indeed, though it is not without its flaws (some small, one or two VERY big – check out the one-star reviews on Amazon.com to see what I mean).
Now onto the meat of the lecture. The focus of lecture 5 was on a generic building security risk analysis question adapted from problem 8E of Philip P. Purpura’s text Security and Loss Prevention, 5th edition (2007, ISBN: 978-0123725257 ). The problem is shown in the SCRIBD window below.
Building on the materials from lecture 4, the aim of this class was to apply structured brainstorming to identify a complete set of security events that might take advantage of one or more observed facility weaknesses. The only technology we used for this in-class exercise was sticky notes (Office Depot brand) and empty wall space, window space, or an unused chalkboard. My strategy for this exercise was to allow 20 minutes or so for unassisted team divergence, followed by me and my teaching intern walking around the room with our own pads of sticky notes interjecting random ideas to help spark creative thinking. The activity finished with 10-15 minutes of convergence where each group was advised to settle on 5-6 broad classes of initiating security events.Of course, the event sets that the students came up with was by no means complete. However, as I advised, this is ok so long as the students articulate what events they are leaving out and for what reasons. This is the essence of a conditionally exhaustive set.
The only bad thing about this lecture was that it was the first lecture I gave at Penn State where I did not have my tablet PC available. Unfortunately, I spilled hot coffee on my tablet, and now it doesn’t work at all. The warranty doesn’t cover such damages either. This “black swan” event totally forced me to reshape how I can go about delivering future lectures. I suppose I have to use the white board and black boards more often now!
Today was a really information-packed lecture. Perhaps the most dense lecture I ever gave. I did this because I really want to get the basic concepts out on the table now so I can spend most of the next few weeks making sense of these concepts. Here is my account of how the lecture went and what was covered:
I began the lecture with a little literature show and tell – I brought with me a book that I find to be a very good snapshot of the current state of practice of security risk management. The book is by Michael Blythe and it is entitled Risk and Security Management: Protecting People and Sites Worldwide (2008, ISBN: 978-0470373057). Personally, I decided not to use this book as a course text because it does little to address the mathematical basis of risk, does not provide much guidance on how to creatively reason about answer to risk questions, and doesn’t talk much about the more fundamental issues shaping risk communication (e.g., perception issues). My stated goal, after all, is to build risk literacy and risk intuition. However, this book really nails down how risk management is done in practice, to include what types of threats to look at, how to structure a site visit, how to construct a risk assessment report, etc. So, at best this book is complementary to the materials of my course. I highly recommend it to anyone wishing to learn more about how risk assessment is really done (but I must caveat that I do not necessarily endorse the way things are being done now as the way things should be done).
Consistent with one of my proposed changes for SRA 311 this semester, I introduced into today’s lecture several low stakes mini quizzes. The first was a question about the definition of risk taken directly from the study guide for the Physical Security Professional (PSP) credential from the American Society for Industrial Security. The second question related to the six questions of risk, and was taken from the Certified Information Systems Security Professional (CISSP) study guide by Shon Harris (6th edition, ISBN: 978-0-07-149786-2). I think I will continue this trend into the semester.
The remainder of the course focused on some pretty basic concepts of risk, in particular:
What is security?
What is risk management for?
What is a security context and why is it important?
The Six Questions of Risk
The risk triplet and the quantitative meaning of risk (or rather, mathematical phrasing of risk)
Scoping a risk study (STEM+VR)
With some review and reflection, I think the students should now be able to at least articulate what a risk study is supposed to do. In the latter part of next week we will start talking about what answers to the six questions of risk should look like. But first we need to better understand the nature of uncertainty and ignorance, which is the theme of the next lecture.
Oh, and next time starts my weekly quizzes (of which there are 14 this semester). The topic: words, definitions, questions, and scoping. I am also having the students read a pretty good article “An Introduction to the Concept and Management of Risk” by James Matschulat (an adjunct faculty member in the criminal justice department at the University of New Hampshire). This [really good] article is part of an edited volume entitled National Security Issues in Science, Law and Technology (2007, CRC Press, ISBN: 1-57444-908-7).
Hooray! As of about 6:30PM on Saturday, 20 Dec 08, I am done with SRA 311 (Risk Management) for the Fall 2008 semester! I have been thinking for several weeks now about what worked in SRA 311, what didn’t work, and what could be better from both student and instructor (and teaching assistant) perspectives. And now that class is over, I decided to take a few hours to write-about how I plan to do things differently for iteration 2 of SRA 311 in Spring 2009. For reference, I include my Fall 2008 syllabus below:
In the following I will just highlight a few of the changes I plan to make in the second iteration of SRA 311.
Revised Course Content for Iteration Two of SRA 311
Probability Theory: Looking back, I think I overstressed some less important topics, and forgot to include others I am now finding to be quite important. The most important thing I should have done was go over the basics of probability theory in detail rather than assume that my students were fully equipped to think probabilistically (I should have known better given that, at best, my students have had only a few lectures on the subject prior to taking this course). So, for the first part of the course, I plan to get serious about teaching probability theory from first principles. But of course, I will discuss this subject with respect to its place in security risk management. (btw: the book I plan to draw from is Introduction to Applied Probability by Pfeiffer and Schum, 1973, ISBN: 0125531508).
The Six Questions of Risk, Risk Triplet, and Definitions: As I did last semester, I plan to stress the six questions of risk assessment and risk management over and over again. The same holds for my repeated mention of the risk triplet and the definition of risk. This time, however, I will emphasize the risk triplet as being the set of scenario, s (which is the pair of initiating event and outcome, or s = (e,o)), probability of the scenario, p, and utility associated with the scenario, u. That is, risk is the set of all relevant ordered triples {<s,p,u>}. As for definitions, I am going to largely focus on risk as the potential for harm or loss, and thus save the more generic definition (i.e., risk as uncertainty about future events) for graduate discussions. Also, I will also stress the need for common definitions issue less as I found that such talk either goes over my students heads at best, or confuses them at worst.
Set Theory and Open vs. Closed Worlds: As I did last semester, I will spend about a week talking about sets (mutually exclusive, collectively exhaustive, conditionally exhaustive, etc.), as well as talk quite a bit about the difference between open and closed world thinking (w/ residual hypothesis). I still think that talking about open worlds (i.e., admitting the possibility of a residual hypothesis) should be introduced at the very beginning of a student’s exposure to risk and uncertainty. See my previous post on the topic. I plan to keep this lecture pretty much intact, but I do think I might add a few more security-oriented examples.
Utility Theory: To accommodate a full discussion of risk, I will be sure to spend a full lecture on the basics of utility theory, to cover what utility is, aspects of multicriteria decision analysis, risk attitudes, and so on. Last semester I spoke about utility for only 10 or so minutes, and consequently my students could not speak to the topic on the final. Though I a lot of utility theory was already covered in the prerequisite class SRA 231 (Decision Analysis), a little review couldn’t hurt.
Support Theory, Possibility Theory and Surprise: Integrated into my discussion of probability theory will be a discussion of support theory (the descriptive side of humans and probability), mention of possibility theory and its axioms, and also mention of Shackle’s theory of potential surprise. I am not clear yet how just where these discussions will occur, but they will surely happen somewhere during the first part (fundamentals) of the course.
Talk About Uncertainty: The discussion of uncertainty (aleatory and epistemic) and all its types will be presented up front with or soon after the introduction of the concept of risk. If all goes well, this will happen during lecture 2. We will also include the discussion of different types of ignorance at this time (unlike last time when it felt out of place in lecture 11).
Security Context and the Eight Elements of Thought: As I did last semester, I will introduce Liner and Paul’s Eight Elements of Thought and Intellectual Standards in great detail during the first lecture. Then, as before, the students will apply the Eight Elements and Intellectual Standards to their first Critical Article Review (CAR) assignment of Manunta’s paper “What is Security?” (published in a 1999 issue of Security Journal). The second lecture will be spent going over the Eight Elements and the Intellectual Standards as they apply to Manunta’s article, discussing the concept of a security context, and then proceed to do an in class example identifying and articulating different security contexts. This semester I want the students to be confident about the Eight Elements and Intellectual Standards by the end of their first week.
Accreditation: As part of the discussion on risk acceptance in Part III of the course, I will include a discussion of accreditation, or the practice of acknowledging that the risk associated with a protected asset is acceptable with respect to its value and purpose. I didn’t do this last semester, but now that SRA 311 is an essential part of the NSA Certificate in Risk Analysis, I figured I ought to start talking about accredidation. I will also include a discussion on standards, whether implicit (e.g., what would a normal security manager do) or explicit (e.g., contractually).
Life-Cycle Cost: When I talk about risk management this semester, I will be sure to include a discussion of life cycle cost of a risk mitigation strategy, to include maintenance costs, replacement costs, operational and procurement costs, and more interestingly, the implicit costs of decreased performance as adversary’s adapt and learn to overcome the countermeasure.
The Insurance Game: My good friend Professor Bilal Ayyub at the University of Maryland recently pointed out to me an interesting pedagogical exercise aimed at teaching undergraduates how to appreciate the role insurance plays in risk management. This semester, I plan to try out this game in the classroom to see how it works (and perhaps spend some money making cool game props, such as custom cards and so on).
Expert Elicitation and Probability Calibration: Last semester I spoke about probability, but did not talk at all about how to elicit probabilistic information when needed. This means that I also did not talk about how to calibrate personal probability judgments. Though I had every intention of talking about this during my fact finding discussion (which I also skipped over), this semester I will be sure to spend a whole lecture on the subject. I will call this lecture “Expert Elicitation and Fact Finding.”
Analytic Confidence: The discussion of analytic confidence will take place sometime in the first 5 weeks of class, probably after my discussion of conditional probability and possibility theory. Last semester I spoke about this all-too-important subject during lecture 20 – by then it was already too late for the concepts to sink in. I won’t make this mistake again.
Influence Diagrams: I am kicking myself for not discussing influence diagrams in class this past semester. Next semester I plan to not only talk about influence diagrams, but also have students use one or more software tools to draw and quantitatively analyze influence diagrams. This should be fun.
Decision Advantage: While risk analysis does promote decision advantage, I think that I will abandon this awkward phrase next semester. Instead, I will simply stick with “risk analysis informs decision making.”
Metrics and Formulas for Risk: As I did last semester, this semester I plan to cover all the relevant measurement scales and formula types one might encounter in a risk analysis methodology. But this time I will do it all in one lecture (or maybe a lecture and a half). I will also have some references to draw on this time around.
Pre-Mortem Analysis, Root Cause Analysis, and Convergent/Divergent Thinking: Last semester I ran an interesting case study focused on the 2007 shooting incident at the Virginia Tech campus. A few lectures after running this case study I figured out how to relate pre-mortem analysis and convergent/divergent thinking ot the case-study. But by the time I did this, it was already too late to solidify the connection in the student’s minds. So, this semester I plan to spend a week covering pre-mortem analysis and covergent/divergent thinking (and introduce the similar topic of root cause analysis) concurrently with running the case study. But unlike last semester’s VT case study, this semester’s focus will be on Aum Shimrikyo and the mid-1990’s sarin gas attack on the Tokyo subway. Other options might be a case study on the Khobar Towers bombing or one focused on the bombing of the Marine barracks in Beirut.
More Information Security and Crime, Less Terrorism: While terrorism is a hot topic these days (though becoming less so), I want to be sure that my course on security risk management (i.e., security in general) covers more than notional terrorists with bombs. This semester I plan to spend more time thinking about information security problems, routine criminal problems, and perhaps a little bit of personnel security/executive prevention. I will also talk about loss prevention as an idea, and also spend some time examining how safety balances and sometimes interferes with security.
CORAS, the McCumber Cube Model, and others: This semester I will start talking about a number of established security concepts and processes, to include CORAS, the McCumber Cube model, and OCTAVE, in addition to reviewing the basic concepts of the security bow tie and the swiss cheese model. But since these topics are no fun to hear about on their own, I still need to figure out a strategy for integrating them into the standard flow of course ideas. I think I figured out a way…
Certifications, Professional Societies, and Ethics: This time around, I will emphasize all the different certifications and security professional societies all throughout the semester. I plan to also integrate ethics into the curriculum in two ways – first by highlighting ethical issues as a matter of course during the semester, and to cap the course off with an ethics story-telling exercise on the last day of class (as I did this past semester, but this time it will be more structured).
Real Questions from Real Certification Exams: This semester I plan to integrate real risk management questions from either the CISSP exam, CPP or PSP exam, perhaps even the CAS Exam P (for actuaries). The goal here is to highlight that everything I teach in my class is relevant to things that matter in the professional world. I anticipate that no less than 25% of the final exam will consist of questions taken from professional exam study guides.
Assignments and Policies
Established Groups at the Beginning of the Semester: On day one I will assign all students to work in groups of my crafting. They are free to make individual trades among themselves long as the class is evenly divided into groups. These groups will work on all in-class exercises, homework, and projects together.
More Quizzes: Attendance was a big problem for me last semester. Without having the data in front of me, I estimate that, on average, only 60% of students showed up for any given lecture. So, this semester, in attempt to better prepare my class for the multiple-choice final exam, to review course material in a fast and effective way, and to take attendance, I plan to give frequent in-class multiple choice quizzes on either the assigned readings (CAR-style questions) or previous lecture’s material. Quizzes will be my means of taking attendance as well as gauging student performance.
More Organized CARs: Unlike last semester where I divided up the class so that 10-14 CARs were due at each lecture, this semester I plan to arrange the schedule such that all students work on CARs at the same time. This means five CARs, each due for all students at the same time. No make-ups. The format for these CAR assignments will be exactly the same as it was last semester.
Critical Book Reviews: Like last semester there will be two required book reviews. But unlike last semester, I will prescribe both books. The first book is Against the Gods: The Remarkable Story of Risk by Peter Bernstein, and the second book is Risk Intelligence by David Apgar. The format for these assignments will be exactly the same as before.
Homework: Ah, there will be homework assignments this semester. Homework will largely consist of preparatory exercises for quizzes. But there will be times when I ask for, say, an influence diagram, some worked problems, etc. All homework will be done in groups.
Final Course Project: This semester, the focus of the final project will be focused on building a risk assessment tool for exploring a particular risk problem of interest to real decision makers. That is, the tool is primary, and will be supported with some multi-media presentation (e.g., reports, poster, auto slide show, You Tube, etc.). While the topics have yet to be determined, I will make available 5-10 topics that groups may choose from. I am tentatively thinking about one or two on maritime piracy, one or two on lab site security, one or two on online communities, one on social engineering, one on party security, and so on. I am still eliciting ideas from people, and hope to have a list in hand by the second week of class.
Methodology Appraisal: There will be no formal methodology appraisal this semester. Rather, I will integrate a methodology review into one or two of the five CAR assignments.
Final Exam: There will be a final exam that, for the most part, will assume the same form as the exam from the Fall semester. There is some question in my mind whether to keep the CAR in the exam or if I should make the entire exam one big multiple choice test. I think I will keep my options open for the next few weeks.
Extra Credit: I always give extra credit, but never anything that amounts to more than 5% on top of a student’s final grade. This semester, extra credit was very helpful for those students who, for some reason or another, didn’t do well on the first assignments of the semester. Although I will not guarantee extra credit opportunities, I suspect that something will come up toward the middle-end of the semester. After all, it helps out those who did well on previous assignments, but not well enough to meet my cutoff values for certain letter grades. But in a perfect world there would be no need for extra credit since everyone would have already done superb routine work…
Attendance: Attendance in required. I will take attendance most of the time this semester, but not always. My policy for attendance is that I don’t give points to students for showing up to class. Rather, I take away points for not showing up to class. My plan for Spring is to implement an attendance policy that is tolerant of up to two (2) random absences, and then for apply a reduction factor to the final exam grade that is in proportion to the number of classes missed. If a student misses all classes, that student will get a zero on the final regardless of whether he or she actually takes it [in math speak, the final exam grade = actual score * (attendance days - missed days)/attendance days]. This is a hard core policy, but perfectly reasonable.
Course Materials: This semester three books will be required – two for the book reviews (see above) and one newer version of a book covering the Eight Elements of Thought. However, this semester I will also insist on using a variety of online soft-copy materials that will all be posted to the PSU course management system (i.e., ANGEL).
Office Hours: Despite having official office hours posted, either no one comes or they try to schedule a different time with me. So, my intentions this year are to have office hours by appointment only. But I also plan to do it in different environments, such as Second Life, PS3 Home, Skype, etc. It is high time I become more IST-ized.
No TA, But One Grader and One TI: Unfortunately, next semester I will not have a TA to help me along with my class. Instead I will have one undergraduate grader working for SRA 311 10 hours per week, and one teaching intern who will take part in class activities and maybe an occasional afternoon or evening event. I, personally, don’t know how I will function without a TA, but I suspose things should be ok if my grader and TI are good (which I suspect they will be).
Better Class Time and Better Room: I am by no means a morning person. This is why I am happy about having a class that begins at 11:15am instead of 9:45am. On top of that, I am pleased to find that I am moving my class from the worst room in the IST building (IST 205 with annoying tabletop Macintosh computers) to the best room in the building (IST 206 with PC laptops). I suppose that, in some may, the later class time and better room make up for the college taking away my teaching assistant.
My Blog About the Class: Next semester I intend to make more efficient use of my blog for recapping course content. The way I am going to do this, though, is to point to relevant reference materials to support learning instead of writing lecture notes from scratch after each class session (my tendency to write a lot about each lecture acted more as a deterrent to writing than I intended). I will also start to tweet about the class and integrate some other types of web communication technology (RSS feeds?).
Future Challenges
Two Sections of SRA 311: In Spring 2009, there will be two offerings of SRA 311. One of these (the larger one) will be taught by me on Tuesday/Thursday mornings. A smaller section of SRA 311 will be taught Tuesday/Thursday afternoons by Professor Dave Mudgett of IST 230-fame. What this means is that we have to coordinate our class schedules, or at least align the learning objectives for our courses.
Cybertorium in Fall 2009: Beginning Fall 2009, my understanding is that SRA 311 will be moving to the infamous IST Cybertorium, a 150+ person computer-ridden ampitheatre not at all designed for fTf (face to face). Fortunately (and by my request) the schedule should be such that the class will meet twice per week for 50-minutes in the Cybertorium, and one more time per week in smaller groups at a location somewhere away from the IST building. The astute reader will see here that I am making lecture more of just that – a lecture. My intent is to move all in-class activities to the recitation sections where students can spend an entire hour applying the things they learned in the lessons prior. To accomodate this move to the cybertorium, I will be, in a small way, treating my Spring 2008 course as a cybertorium class, focusing mostly on lectures with fewer in-class exercises. But when in-class exercises do occur, they will be extensive.
Guest Speakers: For some reason, I feel some pressure to recruit a guest speaker or two this semester. A challenge for me is to identify who would be a good speaker that can (a) entertain the students, (b) convey useful real-world insights, and (c) align his message with the learning objectives I would otherwise have to address were I giving the lecture. Any thoughts?
Epilogue
I invite interested readers to make suggestions regarding what to include, what to stress, what to omit, and what to test. I will be posting a revised syllabus to this blog within the next two weeks. Note that I reserve the right to add more to this post (either directly or via comment) as I things come to mind.
