IBM admits erring in statistics on vendor patching

The IBM X-Force research team has revised a part of its recently released trends and risk report that analysed how well popular software vendors did in patching vulnerabilities disclosed in the first half of the year.

Big Blue's Mid-Year Trend and Risk Report included statistics that examined the percentage of major vendors' bug disclosures in the first half of 2010 that went unpatched. It also included the percentage of flaws rated "critical" and "high" that saw no fix.

To quantify this, the X-Force team drew from its database, which tallies and documents vulnerability reports from mailing lists, vendor advisory pages and exploit lists. In total, researchers tracked some 4,500 bugs.

However, two vendors disagreed with the findings, which prompted IBM to manually reassess "the CVSS (Common Vulnerability Scoring System) scoring, remedy information, and vendor information for every vulnerability that impacted the percentages that appear in this chart," Tom Cross, manager of X-Force research, said in a blog post. As a result, the company released a revised version of the report.

In a blog post, Google publicly expressed its frustration with maintainers of vulnerability databases.

The IBM report originally had said that Google had failed to patch nine percent of reported vulnerabilities and 33 percent of bugs deemed critical or high. The revised report lowered those numbers to zero and zero, respectively, actually making Google the most successful of the popular vendors at patching reported vulnerabilities.

Adam Mein of the Google Security Team said the original "33 percent figure" was due to a single unpatched bug, out of three, that was accidentally considered a security vulnerability.

"To make these databases more useful for the industry and less likely to spread misinformation, we feel there must be more frequent collaboration between vendors and compilers," Mein wrote in the post.

"As a first step, database compilers should reach out to vendors they plan to cover in order to devise a sustainable solution for both parties that will allow for a more consistent flow of information. Another big improvement would be increased transparency on the part of the compilers — for example, the inclusion of more hard data, the methodology behind the data gathering, and caveat language acknowledging the limitations of the presented data."

The amended chart reflected changes for eight of the 12 vendors. Some, though, were very minor.

A big mover was Sun Microsystems. According to the original chart, Sun failed to patch 24 percent of reported flaws and nine percent of bugs rated critical or high. The adjusted numbers dropped the figures to right and zero percent, respectively.

The chart originally showed Linux failing to patch eight percent of overall disclosures and 20 percent of critical and high disclosures, but the update showed those numbers falling to three and zero respectively.

See original article on scmagazineus.com