A paper is out: Impact Estimation using Data Flows over Attack Graphs

In October I presented a part of my work to measure the security of a network at the NordSec 2009 conference. You can find the paper here. Any feedback is welcome.

Abstract

We propose a novel approach to estimating the impact of an attack using a data model and an impact model on top of an attack graph. The data model describes how data flows between nodes in the network -- how it is copied and processed by softwares and hosts -- while the impact model models how exploitation of vulnerabilities affects the data flows with respect to the confidentiality, integrity and availability of the data. In addition, by assigning a loss value to a compromised data set, we can estimate the cost of a successful attack. We show that our algorithm not only subsumes the simple impact estimation used in the literature but also improves it by explicitly modeling loss value dependencies between network nodes. With our model, the operator will be able to use less time when comparing different security patches to a network.

A Fuzzy risk calculation approach as alternative to the CVSS computation

In my previous post I asked some questions about CVSSv2. The looking around for information about CVSS I stumbled over this paper: A Fuzzy Risk Calculations Approach for a Network Vulnerability Ranking System (TM 2007-090). The author describes a fully fuzzy systems approach for ranking vulnerabilities that also can rank networks. The approach is partly based on CVSSv1 and its performance is compared to CVSSv1. It would be interesting to adjust the approach to CVSSv2. One thing that is solved in the paper is that different combinations of input values should yield different output values. This seems to a problem in the CVSSv2, see here and here.

Making sense of the CVSS equations for risk analysis

I am considering to use the CVSS standard to get realistic input values to a real-time risk management model I am developing in a research project. This vulnerability scoring system is meant to be easy to use and understandable. However, I have been trying to make sense of the equations of CVSSv2 without much success. This is how the equation for the base score looks like:

BaseScore =
round_to_1_decimal(((0.6*Impact)+(0.4*Exploitability)–1.5)
*f(Impact))

Impact = 10.41*
(1-(1-ConfImpact)*(1-IntegImpact)*(1-AvailImpact))

Exploitability = 20 *
      AccessVector*AccessComplexity*Authentication

f(impact)= 0 if Impact=0, 1.176 otherwise

It is not clear at all why the equation looks like this. Especially the parts marked with bold. If we look at the old version (CVSSv1) it is much easier to understand:

BaseScore = round_to_1_decimal(10 * AccessVector
                  * AccessComplexity
                  * Authentication
                  * ((ConfImpact * ConfImpactBias)
                  + (IntegImpact * IntegImpactBias)
                  + (AvailImpact * AvailImpactBias)
        ))

As can be seen CVSSv1 is much more straight forward than CVSSv2. In CVSSv1 the different parts might be interpreted as probabilities or costs.

I have a lot of questions about CVSSv2 that I would like to ask:

• Most important: Can anybody tell me what the numerical values of ConfImpact, IntegImpact and AvailImpact mean? Are they probabilities, risk metrics of their own or anything else?
  
• Why are the impact values combined with a noisy-OR instead of just being added?
• Why are the values of the access vector, authentication and access complexity weighted and then added together with the impact instead of just being multiplied?
• Did the people behind CVSSv2 try to adjust the weights of the CVSSv1 to fit the requirements of CVSSv2 before changing to the new equation? If not, why?
• There is a lot of research in knowledge representation and elicitation, how has it influenced this work (if at all)?

PhD Thesis: A logic-programming approach to network security analysis

I have read this interesting PhD thesis called A logic-programming approach to network security analysis (2005) from Princeton University Computer Science Department. It is about modeling a network with respect to its security flaws. The model consists of logical statements written in Datalog that looks very similar to Prolog rules. The author captures the network topology, network and computer configurations and vulnerabilities in the model by analyzing a high-level security policy, the output from a scanner and firewall configurations.

From this model the author claims that he is able to do the following analysis (page 23):

• checking network configurations against a high-level policy specification that captures data confidentiality and integrity,
• hypothetical analysis that assumes various vulnerability situations, and
  
• the generation of attack trees.

Computer Security Strength & Risk

Previously on this blog I have related to an ongoing discussion on risk analysis with FAIR. Also related to this problem is this doctoral dissertation at Harvard university from 2004:

http://citeseer.ist.psu.edu/631841.html

In this dissertation the author suggests an economical model to measure security of a software product. By deriving an upper and lower the bound for the price for finding a new vulnerability he is able to set a value of a vulnerability and a higher value means a more secure product.

My questions are: Has anybody implemented ideas similar to this? What do you think of such an approach?

Psychological warfare

From Anton Chuvakin Blog I read the following blog entry Why Security Is Useless. This is probably true, but that also makes me think: "well, then the only reasonable thing is to give up security". This resembles psychological warfare. As the Borg in Star Trek says: "resistance is futile"

Powered by ScribeFire.

Richard on risk analysis and FAIR again

Richard at TaoSecurity is addressing FAIR again. This time I have come up with what I think is a pretty good argument in defense of FAIR. I wrote a comment at Richard's post but I cite it below as well:

Richard,

I think you are right in some aspects, that is: since with FAIR you do not usually have real data to make probability estimates and then you will not get as good risk estimate as you might wish.

However, in FAIR and similar frameworks you get help to elicit expert knowledge and transform it into a risk estimation. And the validity of this risk estimation is of course related to the validity of the expert knowledge: If you put garbage in, you get garbage out.

But, I think you are wrong when you are saying that the input to FAIR is arbitrary. Of course, if used incorrectly, the input can be arbitrary.

My question is: why would anybody that seriously wants to use FAIR make "arbitrary" input? Why not make "guesses" that are the best according to your knowledge? Then, based on the input and its modeling assumptions, FAIR will output the best possible risk estimation (at least if you believe in Bayesian statistics and decision theory..).

This means that you cannot make any better risk estimation based on the knowledge you have given as input without changing the FAIR model or adding more input.

So if you have to make decision that is the best according to you knowledge, then FAIR might work well.

What do you think?

FAIR is defended

Alex at http://www.riskanalys.is/ defends FAIR here and here.

In defence of FAIR, I think it should be possible to show that by making more fine grained decisions and then combine them, you get less errors than making a single monolithic decision. However I cannot come up with a good model that shows this. Maybe it is already done? Does anybody know?

Powered by ScribeFire.

Riska analysis

There is a an interesting debate going on about the usefulness of risk analysis. Richard Bejtlich is a strong opponent to using risk analysis for making decisions in computer security. You can read a blog entry about FAIR which is a risk analysis framework that I happen to have know a little bit about: TaoSecurity: Thoughts on FAIR.

I think Richard's arguments against risk analysis are quite convincing but I also think that a detailed analysis as prescribed by FAIR is better than a shallow one. I will come back to the reason later.

Paper 4: Allergy Attack Against Automatic Signature Generation

This paper practically shows how to do what Can machine learning be secure? describes. In the paper, they show how to attack systems that uses Automatic Signature Generation (ASG). A typical ASG first detects an intrusion or attack, thereafter automatically generates a signature from the attack data and then filter out all future traffic matching the signature.

By using the fact that many ASG system does not use the same method to detect the attack and then create the signature they are able to fool the system into creating signatures for non-malicious traffic. Also, by not using the full context of an attack, such as the steps leading to the attack, ATG systems are easier fooled.

An ATG system seems to be a kind of unsupervised learning system, using anomaly detection to detect suspicious traffic. Then a signature is created from the traffic based on comparison between many suspicious traffic instances. The signature is often computed from the longest common byte sequence.

Follow up on: Can Machine Learning Be Secure?

In the previous post I mentioned I did not understand the relative distance metric they used for analyzing the security simple learning problem. However, in the paper they refer to a Master thesis that explains the metric in more detail.

The keys to understanding are the following:

1. To move the decision boundary as much as possible, each data point equals the previous mean added with the radius: Xt-1 + R
2. This is done alpha times at each iteration: (Xt-1 + R) * at
3. The next mean will be Xt = [Xt-1 * (n + a1+ ... +at-1) + (Xt-1 + R) * at ]/(n + a1+ ... +at-1 +at) where n is the number of previous data points
4. If we simplify the expression we will get: Xt = Xt-1 + R * at /(n + a1+ ... +at)
5. Assuming that the attacker has complete control of the learning process then n=0 thus: Xt = Xt-1 + R * at /(a1+ ... +at)
6. Using Mt = (a1+ ... +at) as the effort of the attacker and by noticing that the recursion leads to: Xt = X0 + R * [1 + a2/M2+ ... +at /Mt]
7. Thus: (Xt - X0)/R = [1 + a2/M2+ ... +at /Mt]
8. Noticing that ai/Mi = (Mi - Mi-1)/Mi = 1 - Mi-1/Mi
9. Thus we have the relative displacement (Xt - X0)/R = 1 + a2/M2+ ... +at /Mt = 1 + 1 - M1/M2 + ... +1 - Mt-1/Mt = t - [M1/M2 + ... + Mt-1/Mt]
10. The relative distance (or displacement) is then for t = T as in the paper:

• D({Mi}) = T − [M1/M2 + ... + MT-1/MT]
  

Background reading: Can Machine Learning Be Secure?

The next paper from the RAID 2006 proceedings cites a paper called Can Machine Learning Be Secure? as it's source of inspiration. This is a theoretical paper while the RAID paper complements it by being experimental. Thus it seemed reasonable to read it before reading the next RAID paper.

Can Machine Learning Be Secure? That seems to be a good question. This paper analyzes how secure a learning system can be.

A learning system adjusts it's model given new data, these are some of the questions asked:

• Can it be trained by an attacker to allow malicious calls?
• Can it be degenerated such that it becomes useless and must be shut down?
• Are there any defenses against these attacks?

The paper tries to create a taxonomy of attacks on a learning system but I don't think it is that successful. The taxonomy has three axes:

1. Influence: the part of the learning system that is manipulated, causative (alter the training data) or exploratory (trying to discover information about the system)
2. Specificity: a continuous spectrum, from achieving a specific goal, for instance to manipulate the learning system to accept a specfic malicious call, to acheiving a broader goal, for instance to manipulate the learner to reveal the existence of any possible malicious call.
3. Security violation: what security goal is violated, integrity (false negative) or availability (many classification errors making the system useless).

Comment: I don't think the paper gives enough reasons for this taxonomy. It is not that clear to me that these axes and scales are completely orthagonal or at least describes the space of attacks in a good way. Although I cannot, at the movement, come up with something better, I think it should be possible to think this through again and come up with something better. Maybe it is the vocabulary that is problematic; maybe by using different words, the taxonomy will be more readable.

Then the paper lists defenses against the different attacks, such as adding prior distributions (robustness) that makes the system less sensitive to altered data, detecting attacks with intrusion detection mechanism that analyzes the training data, confusing the attacker using disinformation that hinders the attacker from learning decision boundaries and, what seems to be a special case of the former, randomization of the decision boundaries.

Comment: Bayesian learning methods seems to a be natural choice since prior distributions are in the essence of the Bayesian concept.

Last in the paper, they analyze a simple learning example for outlier detection on the bounds of the effort an attacker has to use to manipulate the learning system into wrongly classify a malicious call.

Comment: I cannot write much about this analysis since I could not understand the definition of the relative distance they use. I don't understand why they use it and what it means. Thus I do not understand the result. Is there anybody out there that can help me with this?

See follow up post on this issue.

powered by performancing firefox