How do vulnerability assessors actually assess vulnerability?  This is an interesting question that I have been thinking about recently, and below are some of my initial thoughts on the issue.  Let’s begin by recalling the following expression of risk:

p = Pr(e,o)   (1)

where the joint probability of initiating event e and outcome o can be expressed in one of two ways:

Pr(e,o) = Pr(e)Pr(o|e)   (2a)

Pr(e,o) = Pr(o)Pr(e|o)   (2b)

From my experience, the more common of these two expressions is Eq. 2a as it really conforms to the more intuitive event tree view of risk (consequence following cause).  The latter expression Eq. 2b is much less commonly used, if it is even used at all.  Yet, Eq. 2b is as much an expression of risk as Eq. 2a.  I actually use part of Eq. 2b later on in this post, which is the reason why I mentioned both equations.

Now why the math?  My first hypothesis is that regardless of whether one can speak the language of probabilistic mathematics, all people think about vulnerability analysis in the same basic way, whether it be as part of one’s profession or routine risk-taking decision making.

Colloquially, when one thinks of vulnerability, one might say something to the effect of “I am vulnerable to outcome o due to event e” (where e and o is defined as before).  More common are statements such as “I am vulnerable [with respect] to e,” where the outcome is implied by the context of discussion.  For example, in the course of discussing an organization’s information systems, the statement “I am vulnerable to attack” made by the organization’s IT security manager most likely refers to a attack directed against IT infrastructure, with the outcomes being loss of confidentiality, integrity, availability, or non-repudiation. This same statement said by a pedestrian walking in downtown Los Angeles might be in reference to a physical assault against his or her person where the outcomes are injury and loss of property.  In both cases, however, these terms are in reference to the conditional assertion that the individual will suffer some type of loss should an “attack” occur.  That is, there is no assessment of the likeliness of event; only the assessment of likeliness of an adverse outcome given event.

Now, refer to Eq. 2a.  In this expression, Pr(e) is the probability of an initiating event and the conditional probability Pr(o|e) is the probability of a particular outcome given an initiating event were to occur.  In the security context, Pr(e) is viewed as a measure for likeliness of attack (i.e., the initiating event), and Pr(o|e) is the measure for conditional likeliness to a particular outcome given an attack were to occur.  I choose to label this latter parameter the “vulnerability to o from e” as it is conceptually equivalent to the manner in which statements of vulnerability are made in everyday language.  Accordingly, in terms of subjective probability, statements of vulnerability express the degree of belief held by an individual in the outcomes that will occur when confronted by a particular challenge.

Back to my original question.  How does a vulnerability assessor do a vulnerability assessment?  Ultimately, the answer to this question should take the form of a descriptive model of human reasoning.  So, as a first step in my quest toward a descriptive model of vulnerability assessment, I decided to contemplate how I, personally, would perform a vulnerability assessment.  The resulting model from this inquiry is what I will tentatively call the “McGill Descriptive Vulnerability Assessment Model“:

Step 1: Soak in the subject environment.  Without looking for anything in particular and without reference to any particular type of attack, explore the subject environment in a thorough, careful and curious manner.  Over time your brain will pick up on both glaring and subtle environmental cues suggestive of strength and weakness.

Step 2: Hypothesize outcomes of particular concern that are relevant to the problem at hand.  These outcomes “o_j” (j = 1, 2, …) can be vaguely defined as “a lot of people hurt” or “significant property damage” or “damage to reputation.”  There is no need to be crisp about the outcomes of concern at this stage.

Step 3: In a very non-quantitative way, attempt to make a judgment about Pr(o_j|E), where the set “E” (big-E) represents the union of all plausible events e_i (i = 1, 2, …) of a particular type.  That is, attempt to make a judgment about the likeliness of one or more of the “bad” outcomes identified in Step 2 assuming that some sort of vaguely-defined event (e.g., “terrorist attack,” “assault,” and so on) occurs (i.e., “E”).

Step 4: For those outcomes where the likeliness is viewed to be sufficiently “strong” (high or intense), assume that these outcome have been realized but the cause is unknown.  This step attempts to hypothesize what the most likely cause of these outcomes were.  This is a sort of pre-mortem analysis.  A list of causes (or initiating events) e_i can be developed in this manner, where the list is ranked in order of decreasing (or increasing if you prefer) likeliness.  If there is no strong feeling of vulnerability, then use this step to try to explain why and attempt to challenge yourself using alternative analysis techniques (e.g., Devil’s advocacy).

Step 5: For each e_i identified in Step 4, assess your subjective degree of belief that undesirable outcome o_j (for each j) will follow from event e_i.  This is a more refined vulnerability assessment of Pr(o_j|e_i) than Step 3 in that we are looking at specific “e_i“’s instead of the whole collection “E.”

[NOTE: you can cycle through steps 2 through 5 over and over again, each time refining the definition of e, adding o's, and so on.]

Step 6: Express your opinion of vulnerability to OUTCOME given EVENT.  A four-tier symmetric linguistic vulnerability scale of the following type can be used (as an example) to aid in expressing vulnerability where the bracketed values express lower and upper probability limits for the phrase:

• Highly Vulnerable … Pr(o|e) = [0.75, 1.00] … (odds are heavily in favor of the adversary)
• Vulnerable … Pr(o|e) = [0.50, 0.75] … (odds are in favor of the adversary)
• Invulnerable … Pr(o|e) = [0.25, 0.50] … (odds are in favor of the defender)
• Highly Invulnerable … Pr(o|e) = [0.00, 0.25] … (odd are heavily in favor of the defender)

[Note that while it may appear that step 6 departs from what one might otherwise think was part of a normative model and not a descriptive one, this is actually how I think.  So it is, in fact, descriptive, but with respect to how I think about vulnerability.]

Let’s see how this descriptive model works.  Suppose I am tasked to assess the vulnerability of my house in Maryland to damage resulting from naturally-occurring events (ignoring that I have insurance).  I admit here that nature is my assumed adversary, and perhaps is my only adversary aside from the occasional disgruntled student.  As I walk around my house, I notice a slightly lopsided roof, sturdy brick exterior, clogged gutters, new windows, canopies of trees (that seem to be on their last leg) blocking the sun, empty garbage cans in the yard, lawn junk (e.g., garden gnome) on the neighbor’s property, curbside lunch trash leftover by contractors than tend to take breaks in front of my house, loose television antennas on neighborhood rooves, etc.  I begin to think that a bad day for me would be when my roof caves in or many of my windows break, since both would cause a significant amount of property damage.  All things considered, I think my vulnerability to many broken windows is quite low, but the roof collapse worries me.  I proceed to consider a variety of causes of roof collapse, to include (in order of decreasing likeliness) tree limbs crashing down from above and excessive rain and autumn leaves weakening the integrity of my roof structure.  Returning to the outcome of concern and leveraging my structural engineering background, I now can make the following judgments:

• My roof is right now vulnerable to collapse due to falling tree limbs (any cause).
• My roof is right now highly invulnerable to roof collapse due to buildup of leaves and rain.
• My roof is right now highly invulnerable to roof collapse due to most other natural causes.
• My windows are right now highly invulnerable to significant damage due to most natural causes.

Notice the underlined words that caveat my vulnerability judgments.

• right now means the vulnerability assessment is valid only for the system in its present state and normal deviations.  If things change (e.g., adding solar panels to the roof, aged roof and windows), then the vulnerability assessment may change as well.
• most other and most are used to allow flexibility for the residual hypothesis I am not considering in my mind.  While I would be hard-pressed to articulate all of the events floating around in my head that might prompt damage, using the word most allows for me forgetting to include a few. (yes, I know this is a cop out, but supposedly more experience = more hypothesized events).

Finally, I must point out that nowhere here do I make any judgment about event likeliness.  That is, what I have here is a method for vulnerability assessment, not threat assessment.  Had I gone on to asset threat as well, the combination of threat and vulnerability (for a given pair of event and outcome) would produce a statement of risk a la Eq. 2a.

I wonder how well does this model matches that of practicing vulnerability assessors in DoD and industry, or with those focused on computers, physical sites, or the fabric of society?  Regardless of how long my model remains unrefuted (which may be a day or much longer), I will continue to seek out ways to discredit it in hopes of converging on a robust descriptive model for vulnerability assessment.

Send article as PDF to