I was somewhat surprised (but pleased) at the level of interest back when I published my Windows Vista - 90 Day Vulnerability Report. It was about the earliest span of time I thought might give us some indicators, and the indicators did look good. (Though, I did not give us an "A+", in spite of some of the attributions ;-)
Six months is a much more interesting time frame, and gives us the opportunity to see if the early trend indicators are holding up, or if the early signs of progress were a short-term gain. Also, I thought it was worth going a little deeper in the analysis to look at the total fixed and unfixed vulns as I did last time, plus these additional views:
- Include a comparison view of Linux distribution workstation builds that exclude vulnerabilities non-default optional components as well as OpenOffice and other applications that do not have equivalents on Windows XP.
- Include a comparison view that excludes Low and Medium severities to just focus on High severity vulnerabilities fixed and unfixed in the first 6 months, and
- A comparison view that combines both of these
For the full details, or to print the report, you can download the report in pdf.
For those that only want the executive summary, here is a key chart that shows the publicly disclosed High severity vulnerabilities during the first 90 days of availability, broken down by vulns fixed and vulns unfixed. Note that this chart is showing the reduced Linux builds that exclude non-default and optional components without equivalents on WIndows. (clicking the chart also gets you to the full report.)
The results of the analysis show that Windows Vista continues to show a trend of fewer total and fewer High severity vulnerabilities at the 6 month mark compared to its predecessor product Windows XP (which did not benefit from the SDL) and compared to other modern competitive workstation OSes (which also did not benefit from an SDL-like process).
If you share the opinion that Windows and applications ported to Windows get a higher level of researcher scrutiny than other OSes, then the 6-month results are even more positive. If you don't share that opinion, then they still stand on their own ...
Read, Enjoy, Forward.
Best regards ~ Jeff
Full Disclosure: I work for Microsoft - read my previous blog post, Exactly how biased am I?.
Also, I'd like to make a shameless plug for my other blog, http://blogs.technet.com/security, where I sometimes post more personal entries such as The Saga of My Luggage & British Air and Building My Windows Vista Media Center - Part 1 - The System.
While this is an interesting metric, it is near useless in the real world. Discovered flaws and patches published are only a small fraction of the whole security picture and present a misleading image of overall security of the competing OS's.
In my view, the simplest way to gauge security is by taking a realistic view. How likely are you to contract a virus for a given OS (this can be broken down in two ways: 1) assuming that you use the default setup and 2) assuming that you use reasonable precautions). At the end of the day, all that matters is whether the respective companies are meeting the challenges faced by their particular OS's (it can be argued that Windows, as the biggest target, faces the biggest challenge--though this is actually irrelevant to the measure of real world security). By this simple and easily verifiable gauge, it seems clear that OSX and the various Unix branch OS's are far more secure.
You can argue for decades on why that is. Is OSX more secure because of its modular construction or because fewer hackers attempt to hack it in the first place? But at the end of the day, it doesn't really matter. The simple fact is that there are no viruses in the wild for OSX. That, in a very real (not simply theoretical) way, makes it more secure. If you don't want to worry about viruses and pop-ups then there is a clear choice at present.
Will it stay that way indefinitely? Nothing lasts forever. But OSX has a 5+ year track record that seems to suggest it will continue it's leadership in security.
Isaac
I attempted to post a somewhat shorter comment previously, perhaps your filter thought is was spam.
Four things I would be interested to know.
1. As you stated in your report, you removed software which had no equivalent in vista. When you where counting security fixes, did you count fixes to the software that performed the same "out of box" functions? IE Web browser, email client etc.. Before you say it, I did read your report. This particular point was unclear.
2. What version of vista was this count made against?
3. I was unable to find information regarding the US DOD ratings for vista. I was wondering if you either had that information or could point me in the right direction.
4. How does vista stack up in a 90 day review against RHEL5? Obviously to early for a 6 month review, but a preliminary analysis would be helpful.
Good work on the report. Personally I'm tired of reading all the "blah blah conspiracy, blah blah Linux is invulnerable" posts. The numbers show the facts. (BTW I run Linux on my MacBook Pro, take that OSX fanboys! and I hate Windows but I must use it for my gaming habit ;) So I commend you for doing this and I HOPE Linux will show better on your next review!
No matter how good the security of any microsoft product is, its implementation is always a last measure.
Why did it take so long for proper measures to be implemented ? Simply because they could get away with it.
The economics of the microsoft business model demand that as soon as the public has been appeased money "wasted" on security will be re-diverted toward other purposes.
This is in huge contrast to linux where no matter if any commercial Linux distributor would become negligent on security, there will be a party that will implement if for you, simply because a potential failing of the distributor would create economic incentive for a third party to step up and compete with them, a situation impossible with any version of windows.
It's one of the benefits of a free market on an equal playing field.
Coincidentally, this mostly invalidates the "time until vendor distributes patch" metric. Administrators caring for an existing critical installation confronted with a critical vulnerability will be able to replace erroneous code regardless of vendor activity. It might be a temporary cludge that will not see be suitable as an official patch to cover the vulnerability but it will take care of the problem in the meantime, something not possible within a closed-source operating system.
For any large or critical installation this is the difference between administrators sitting around shrugging shoulders and saying "Well, we're waiting for Microsoft. What can you do?" and administrators assessing the real threat to the installation and being able to actively engage the problem.
If your business is important to you, *you* should have control over it regardless of how much security any outside metric promises.
Well, let's see, what all was in the linux distros? was it a full DVD install of ubuntu with everything in it? was it a stripped down install meet the features that windows provides out of the box? I mean, if you're talking about the full OS that includes apache, mysql, nfs, etc, yes, it will have alot of vulnerabilities and not much can be done about that fact.
Let's talk servers, how does a full install of windows server 2k3 match up to a LAMP setup?
Well, the report actually answers your question, so I won't repeat it here... but I'll give you a hint, the report covers more than one installation scenario...
As for servers, I can point you at some role-based analysis that has been done to specifically compare 2k3(+.NET + SQL) with LAMP: http://www.microsoft.com/windowsserver/facts/analyses/secinnovation.mspx. It is a Microsoft-sponsored study, but the methodology is laid out pretty clear, as well as the sources, so you should be able to validate it or duplicate it if you are skeptical.
Where is FreeBSD, Solaris?
Stupid test - doesn't show anything :-(
I personally think that you really can't compare companies in this way. Firstly you got to take into account how companies work. If you look at the graphs, all the open source OS's had more bugs than the closed source operating systems. The fact is that you can't really trust any of the reports that come from any of the closed source propriety operating systems. They don't have to tell you about a bug unless they want to or until they have a patch ready for it. On the other hand the open source based companies have a far more open development process. Red Hat can't hide a bug in its software because most of the software is Open Source. So in the end can you really trust the numbers?
Note: I use Linux and Windows almost every day.
.
don't forget that Jeff Jones works at micro$oft ........
it's unuseful to say more
Jeff, you are forgetting to mention all the security features Red Hat and Fedora have:
http://fedoraproject.org/wiki/Security/Features
http://www.awe.com/mark/blog/200701041544.html
If you were truly unbiased, you would have mentioned the various security features in Linux!
This means the number of *exploitable* vulnerabilities on a properly secured install of certain security-enhanced distributions are much smaller and less severe than the number of vulnerabilities detected.
Another interesting tidbit... why is it that the various distributions have greatly unequal number of security flaws when the software components are essentially the same? We must question the accuracy of your findings.