AusCERT2012: FreeBSD talks amateur bug hunting

In the war on vulnerabilities, small wins count.

But those wins aren't high on the agenda of professional bug hunters: They seek the bounties offering thousands of dollars to those who report the most dangerous security flaws.

They have made a living out of it too, scoring multiple high-paying bounties. Google, for example, paid $10,000 each to three bug hunters ahead of Pwn2Own this year, and Facebook has shelled out upwards of $5000 for lone bugs.

The search giant as of January had paid $730,000 over the two-year life of its Chrome bounty program. It's now including the Chromium browser in the program through which it will pay for the most critical vulnerabilities (think violations of the verified boot path or renderer sandbox escapes using Linux kernel flaws).

The emphasis on large bounties was illustrated in April research by DVLabs that found reported vulnerabilities in internet-based systems and applications dropped by a fifth over the last 12 months. A quarter of those counted were classified as highly severe and attained a score of between eight and the maximum 10 on the National Vulnerability Database's Common Vulnerability Scoring System. It says smaller vulnerabilities not counted were either fixed quietly by owners or not at all.

Persistent small bugs are risky because they can become critical overtime, says FreeBSD Project security engineer Colin Percival. He says small vulnerabilities reported in his secure backup service Tarsnap, “weren't critical when they were reported but could have become a problem later” given his plans for the affected code. 

He points to a cross site scripting vulnerability reported last year which at the time only affected the user who executed it, but would have later become problematic.

Percival had earlier found himself struggling to attract hunters to seek Tarsnap bounties worth up to $2000 for critical vulnerabilities. He suggested that users considered the service to be “already secure code”.

To draw eyeballs, Percival pushed the Tarsnap source code and opened a series of smaller bounties paying right down to $1 for cosmetic and typographical errors. Bug reports increased, and he received more than 300 over the last 12 months. “While [big bounties] have value, the open source world has a lot of bug reports from [novices] who tend to just see something wrong in code, then file a bug report,” he says.

But the move was not only about attracting amateur hunters.

Ugly code

The amateur typically lacks the die-hard staying power of the most devoted bounty hunters, and their steely nerve that allows them to pour over complex code. So to draw them in, code must written so that it is easy to read and understand.

“Casual coders won't sit down for a week; if they don't find something in half an hour, they will get bored and move on. You need to design code so they can say 'yes this is obviously right or no it's wrong”.

In FreeBSD, the security bugs that prevail are the ones hidden in ugly pieces of code that contributors don't want to read.

So what makes code pretty? It must be well-designed and modular – meaning that a file of 5000 lines of code should be broken into say 10 equal parts – and it should include comments explaining in plain English what the sections intend to do.

“Commenting style is something programmers love to argue about, but one of the most useful comments I could write was one that explained what the code does,” he says. “Though programmers will ask you why you are repeating yourself.”

Cleaner code and humble bounties were part of what Percival saw as a way of aligning safe practice in software engineering with that employed in other engineering fields.

“Think of how factories and so forth are built to avoid injuries: It doesn't target just injuries, it also targets unsafe behaviour. In software, you can't just target one bug, you have to target all bugs.”

Copyright © SC Magazine, Australia