This week at RSA Japan, Microsoft published its second Security Intelligence Report, covering the 2nd half of 2006. Similar to the first document from last year, this one provides a lot of information about malware trends as observed by the internal Microsoft team from results of Windows Defender, the Malicious Software Removal Tool, OneCare and the Microsoft Exchange Hosted Filtering service. All good stuff.
Additionally, Microsoft has added a vulnerability trends section to the document this time, and as one of the primary contributors, I want to promote it a bit.
You can download the full report here.
To entice you, here are a few of the highlight charts.
This first figure charts the growth of total vulnerabilities disclosed over the past 7 years, showing that over 40% more vulnerabilities were disclosed in 2006 than 2005 and that there were more vulnerabilities in each half-year of 2006 than in any year up through 2004.
This second figure charts the National Vulnerability Database's (http://nvd.nist.gov) assessment of whether each vulnerability is either "easy" or "complex" to exploit. Note that there was a much higher percentage of complex to exploit issues disclosed in 2006, a sign of the growing maturity of the security research industry, IMO.
That's just a peek, so if you find these interesting, you may want to download the full report.
As a direct result of dealing with Microsoft over the years, I have (had to) become a connoisseur of things which are true but irrelevant. So it is here.
The question is not whether the attacks are complex or even need to be complex, but whether the workfactor of the defender or the attacker is climbing at the faster rate. By analogy, if a street cop is able to say that drug smuggling is much harder than it used to be, that is good, but if the price of the drug in question is falling nevertheless, the cop is losing the war. Same here: the price and availability of stolen data is falling, ipso facto, it is fine to say that the attacks have been made to be more complex, but it is irrelevant.
Let's start by normalizing the data. How fast is the complexity of the Microsoft operating system rising? If it is proportional to the square of code volume, then from XP (40MM lines) to Vista (120MM lines) is 3X hence a complexity figure of merit of 9X. If the complexity rise of the attacks is less than 9X, they (you) are winning the battle and losing the war. (If XP->Vista complexity is 9X over 4 years, then that's a 31.6% CAGR -- recast the argument that way, if you prefer.)
Here's a second way to look at it: The graph shows the cost of a new attack. The workfactor of the defender is proportional to the sum of all attacks seen to date. The workfactor of the attacker is proportional to the cost of crafting a new attack. Since crafting a new attack is increasinginly automated, the workfactor for the attacker per unit of attack firepower is likely falling, such as for polymorphic change which as we know dominates. At the same time, the defender tends to be unable to harness automation to the critical task of initial recognition of attacks and must rely on detecting the effects of attack after the attack has taken place. If attacks propagate ever faster while the defender has an irreducible minimum time pre-detection, then the only question is how much money the attacker can push through the hole pre-discovery. We already see this with new phishing URLs which are up, on average, four days before being abandoned -- about the minimum operational discovery latency when you factor in the time attorneys and law enforcement spend on backtracking.
Third and last (in this brief series), it is likely that with Vista Microsoft has succeeded in making the finding of vulnerabilities too hard to be a hobby, but almost surely not too hard to be a job. We would all agree, I should think, that professionalization in the attacking class is proceeding apace. The side effect, then, of Vista is that the fraction of all vulnerabilities that are held privately is a rising fraction. Just as it is sometimes true that an epidemic of this or that turns out to be a stable rate overlain with improved reporting of disease, the reverse is also true as a falling rate of reporting, such as when it is impossible to report, may imply progress when progress is not, in fact, what is on the table.
Let me clearly applaud the publication of numbers and some statistics about them. As Fred Mosteller said, "It is easy to lie with statistics, but it is easier without them." That we are having this debate is itself progress.
Dan,
Glad you took time to read and comment.
I admit to a little puzzlement in that your response seems at times directed at Microsoft and Windows Vista, but the charts and the Vulnerability Section in the report are not specific to either in any case, but instead are the industry wide numbers for all products. So, for example, with the industry finding and disclosing 42% more vulns in 2006 than in 2005, that's a tremendous rate, but I can't compare that to a Vista rate, as it wasn't available in those time periods.
I like the discussion though, so let's start with:
The question is not whether the attacks are complex or even need to be
complex, but whether the workfactor of the defender or the attacker is
climbing at the faster rate.
I think that is one question of many with respect to what is happening in the industry, though an important one. Let's make a simplifying assumption though, for the sake of consideration - assume that the workfactor for one type of common malicious attacker remains exactly the same, in that he or she leverage a published sample exploit which he didn't work to develop himself at all.
That of course does not mean that the workfactor for security researchers finding and disclosing vulnerabilities necessarily stays the same. In fact, if the community in whole is finding and disclosing 40% more that before, then it seems that even if I don't know the details of the factors driving that, I can still observe that probably the increase in numbers shows that either the number of researchers is increasing, the workload in finding vulnerabilities has decreased, or some combination of both. Or, maybe there are more products out there with shoddy implementations.
However, what would it say about the security of a product if the industry trend is 40% more vulnerabilities, while the trend for that product is flat or down? Probably too many variable to say really. Are researchers shifting to a different/newer product? Does the product have a higher quality? Is the product just out of favor with researchers? Who knows? A definitive answer may not be possible. However, that does not rule out the practical value of relative answers.
Take your example about the pool of privately-held versus publicly-disclosed issues. Even if privately-held issues rise as a percentage of total vulnerabilites for a product, doesn't a reduction in the publicly dislcosed count imply a benefit in terms of reduced customer risk?
Ultimately, there is lots of fertile ground for deeper research, with harder questions to ask, as you ask some of them here.
For the complex-to-exploit chart above, I didn't go that deep. Roughly 15% of vulns in 2006 were harder to exploit than previously and it was a big departure from years past. Even if every one of those vulns had remained undiscovered, the remaining set of new vulns would have still have represented an increase of 25% over 2005. My takeaway was more an affirmation about the maturing researcher industry and tools. My belief is that the higher number of complex-to-exploit issues found affirms that lower workfactor for finding vulnerabilities that would have been very difficult to find a few years ago without fuzzing and other automated testing tools.
Regards, Jeff