Since the events of 9/11, particularly following the creation of the Department of Homeland Security, much attention has been paid to the use of probabilistic or quantitative risk analysis methods for the purposes of informing security investment decisions.  The debate on the appropriateness of these techniques was quite intense for awhile (say 2003-2006), and to some extent I think there was no clear winner (though I think we are finally coming to grips with what “risk-informed decisions” really means, which in a sense weakens the need for this debate).  Among the fighting and debating, I often found myself wondering what the late Prof. Norm Rasmussen would say about the value of QRA for security.  Now, a number of well-respected scholars have spent quite a deal of time and effort writing on the issue (e.g., Apostolakis, Cox, Bier, Ayyub, Haimes, Kunreuther, Slovic, Pate-Cornell, Diesler, Lave, etc.).  But nowhere could I find even a comment from Rasmussen on the issue.

[NOTE: Norman C. Rasmussen was the director of the famous 1975 Reactor Safety Study, or WASH-1400.  Because of its extreme significance in those days, the report was nicknamed "The Rasmussen Report." I hope to have a copy of the WASH-1400 report posted to this site sometime really soon - oddly enough I can't find it anywhere online]

Then there was my visit to Sandia.  I was priviledged to sit in on a presentation delivered by a scientist at Sandia National Labs that walked through the history of Security Risk Analysis from the Sandia perspective.  On one of the slides there was a quote about the appropriateness of QRA for security attributed to Professor Rasmussen himself!  I was truly taken aback!  I asked whether there was citation I could use for this quote, and low-and-behold there was.  Thanks to the Sandia people, I was able to obtain a copy of this paper and post it here via Scribd:

The citation information is as follows:

• Rasmussen, N. C. (1976). “Probabilistic Risk Assessment: Its Possible Use in Safeguards Problems.” Presented at the Institute for Nuclear Materials Management meeting, Fall 1976, pp. 66-88.

Note the timing… this commentary was made just after the 1975 release of the WASH-1400 report.  My understanding was that many believed PRA/QRA could be applied to problems outside the domain of nuclear safety, perhaps to include nuclear safeguards.  Prof Rasmussen believed then that QRA methods, as outlined in WASH-1400, are NOT appropriate for quantifying safeguards risks (though he says nothing about their usefulness in empowering analysts with knowledge to better inform decision makers).

Just to quickly layout the outline for this paper, Prof Rasmussen begins by offering an overview of all three levels of QRA then comments on the differences between security and safety problems, the most clear being that terrorists are not random and that there is some deliberate attempt to maximize consequences.  Rasmussen also points out that the only practical conservative value to assume in security is one, which given the tendency for terrorists to maximize consequences, almost always results in an unacceptable quantitative risk.  His solution – “make the unauthorized access to special nuclear material very difficult,” that is, make the probability of access so small that even if all the other probabilities are unity, the benefit of having nuclear power still outweighs the risk of malicious terrorist use of nuclear material.  Basically, this amounts to a focus on vulnerability reduction, but only those aspects of vulnerability pertaining to the unauthorized access to special nuclear material (not egress, use, response, recovery, etc. dimensions).  The paper concludes with a short question and answer exchange between Prof. Rasmussen and several audience members, some of which is quite interesting (and clearly dated before the existence of the Design Basis Threat).

In the end, I believe this talk is where the idea of “assuming probability of attack is one” came from, though I could be wrong.

Send article as PDF to

A recent commentary piece I authored for the Security Analysis and Risk Management Association’s (SARMA) August-September 2008 Risk Communicator just appeared in my email inbox and on the SARMA website.  For convenience, I repeat this editorial below (it can be linked to here).  Note that I noticed one typo and one inconsistency after the fact, which I corrected in the version below (mods shown with underline or strike-through; hopefully the SARMA folks will follow suit).

Risk analysis, much like any other professional analytic activity, informs decision-making. Most security professionals have no objections to this seemingly obvious statement. But how does risk analysis actually “inform” decision-making? Do the end results of a risk analysis matter, or is the process of doing risk analysis more important?

Much debate in recent years centered on the appropriate arithmetic or logical expression for security risk. It is hard nowadays to call yourself a security risk professional unless you have been party to a debate over the appropriateness of qualitative versus quantitative risk methods — or perhaps even so-called “quantified” approaches. This debate continues today in government and industry, and is unlikely to subside until the debaters discover the “holy grail” of risk formulas that applies equally well to anything and everything; that is, unless we finally learn to accept that such a formula does not exist, nor would we be much better off even if it did.

A useful risk analysis methodology is one that generates meaningful risk knowledge throughout its implementation. Regardless of the strategy used to score and aggregate threat, vulnerability and consequence, good risk analysis seeks to generate useful knowledge of a system and its weaknesses, and estimates how the system might respond to challenges brought on by a variety of plausible threats. Numbers or labels used to describe risk rarely yield any new insights in themselves. At best, risk results offer a sanity check on methodology and intuition — and any disagreement between the intuition and the final result provides a means for revealing flawed reasoning or a flawed analytic approach, and nothing more.

I believe that the debate over formula has less to do with the pursuit of mathematical correctness and more to do with it being much easier to argue over equations than it is to debate the “value-added” of a process. Formulas produce visible numbers (whether correct or not); processes generate invisible insights. Consequently, it is harder to measure the benefit of a methodology in terms of its ability to create understanding than it is to criticize the mathematical correctness of an arithmetic expression. And But much like the opening statement of this essay, most security professionals would agree that the process of doing analysis is more meaningful than the final answer.

The real question, then, is how do we craft a risk analytic process that maximizes knowledge creation? Shifting the debate toward process instead of product offers the potential for a greater return on intellectual investment than quibbling over details of calculation. After all, it is the reasoning that establishes decision-maker trust in the results of a risk analysis, not the form of the risk output. So let’s focus less on how to calculate risk, and more on understanding how to build a methodology that actually improves our ability to make reasoned risk management decisions.

Send article as PDF to