Making sense of the CVSS equations for risk analysis

I am considering to use the CVSS standard to get realistic input values to a real-time risk management model I am developing in a research project. This vulnerability scoring system is meant to be easy to use and understandable. However, I have been trying to make sense of the equations of CVSSv2 without much success. This is how the equation for the base score looks like:

BaseScore =
round_to_1_decimal(((0.6*Impact)+(0.4*Exploitability)–1.5)
*f(Impact))

Impact = 10.41*
(1-(1-ConfImpact)*(1-IntegImpact)*(1-AvailImpact))

Exploitability = 20 *
      AccessVector*AccessComplexity*Authentication

f(impact)= 0 if Impact=0, 1.176 otherwise

It is not clear at all why the equation looks like this. Especially the parts marked with bold. If we look at the old version (CVSSv1) it is much easier to understand:

BaseScore = round_to_1_decimal(10 * AccessVector
                  * AccessComplexity
                  * Authentication
                  * ((ConfImpact * ConfImpactBias)
                  + (IntegImpact * IntegImpactBias)
                  + (AvailImpact * AvailImpactBias)
        ))

As can be seen CVSSv1 is much more straight forward than CVSSv2. In CVSSv1 the different parts might be interpreted as probabilities or costs.

I have a lot of questions about CVSSv2 that I would like to ask:

• Most important: Can anybody tell me what the numerical values of ConfImpact, IntegImpact and AvailImpact mean? Are they probabilities, risk metrics of their own or anything else?
  
• Why are the impact values combined with a noisy-OR instead of just being added?
• Why are the values of the access vector, authentication and access complexity weighted and then added together with the impact instead of just being multiplied?
• Did the people behind CVSSv2 try to adjust the weights of the CVSSv1 to fit the requirements of CVSSv2 before changing to the new equation? If not, why?
• There is a lot of research in knowledge representation and elicitation, how has it influenced this work (if at all)?

Back to the FAIR discussion

Previously I have tried to argue why FAIR is a valid approach. Now I think there is some research result that might be helpful to understand why FAIR works

Overcoming Bias: Average Your Guesses

In short, the article states that making two (many?) guesses is better than making one guess...

So if you have an approach (FAIR) that lets an expert make several informed guesses and combine them rigorously, then it is quite likely you will get a better estimation than only by making one single guess.

Computer Security Strength & Risk

Previously on this blog I have related to an ongoing discussion on risk analysis with FAIR. Also related to this problem is this doctoral dissertation at Harvard university from 2004:

http://citeseer.ist.psu.edu/631841.html

In this dissertation the author suggests an economical model to measure security of a software product. By deriving an upper and lower the bound for the price for finding a new vulnerability he is able to set a value of a vulnerability and a higher value means a more secure product.

My questions are: Has anybody implemented ideas similar to this? What do you think of such an approach?

Richard on risk analysis and FAIR again

Richard at TaoSecurity is addressing FAIR again. This time I have come up with what I think is a pretty good argument in defense of FAIR. I wrote a comment at Richard's post but I cite it below as well:

Richard,

I think you are right in some aspects, that is: since with FAIR you do not usually have real data to make probability estimates and then you will not get as good risk estimate as you might wish.

However, in FAIR and similar frameworks you get help to elicit expert knowledge and transform it into a risk estimation. And the validity of this risk estimation is of course related to the validity of the expert knowledge: If you put garbage in, you get garbage out.

But, I think you are wrong when you are saying that the input to FAIR is arbitrary. Of course, if used incorrectly, the input can be arbitrary.

My question is: why would anybody that seriously wants to use FAIR make "arbitrary" input? Why not make "guesses" that are the best according to your knowledge? Then, based on the input and its modeling assumptions, FAIR will output the best possible risk estimation (at least if you believe in Bayesian statistics and decision theory..).

This means that you cannot make any better risk estimation based on the knowledge you have given as input without changing the FAIR model or adding more input.

So if you have to make decision that is the best according to you knowledge, then FAIR might work well.

What do you think?

FAIR is defended

Alex at http://www.riskanalys.is/ defends FAIR here and here.

In defence of FAIR, I think it should be possible to show that by making more fine grained decisions and then combine them, you get less errors than making a single monolithic decision. However I cannot come up with a good model that shows this. Maybe it is already done? Does anybody know?

Powered by ScribeFire.

Riska analysis

There is a an interesting debate going on about the usefulness of risk analysis. Richard Bejtlich is a strong opponent to using risk analysis for making decisions in computer security. You can read a blog entry about FAIR which is a risk analysis framework that I happen to have know a little bit about: TaoSecurity: Thoughts on FAIR.

I think Richard's arguments against risk analysis are quite convincing but I also think that a detailed analysis as prescribed by FAIR is better than a shallow one. I will come back to the reason later.

Paper 2: Behavioral Distance Measurement Using Hidden Markov Models

In this paper, the authors describes how they to use a hidden markov model (HHM) to model the execution similarities between two process performing the same work. For instance, two Apache web servers running on two different platforms, Linux and Windows. The assumption is that the two process will not have the same vulnerabilities, and thus by measuring the behavioral distance between the two process, we can detect anomalies.

Much of the paper describes the HHM and whether the overhead is small enough to make the algorithm usefull.

Something missing is the significance of the results. For instance, when comparing another distance metric algorithm called an ED-based approach, the result is that the HHM-based approach is 6.32% faster, but nothing about the variance or significance. I would recommend any researcher to choose a good statistical test so results cannot be so easily questioned. A good online handbook for such tests can be found at the NIST/SEMATECH e-Handbook of Statistical Methods.