The illusion of accuracy

It seems likely that I am one of the vanishingly small people in the country who has no interest at all in the latest Harry Potter book.

The last in a phenomenally successful series, its release was accompanied by midnight openings of bookstores, queues of fans in costumes and the like. In true 21st-century style there was also a copy leaked on the internet.

There were, of course, plenty of spyware-baited fake editions floating around but, apparently, some copies of the printed book were "accidentally" released early, and one enterprising fan decided to photograph them and distribute them online.

Although released only a week or so before the genuine article, it soon became a highly desired download on the peer-to-peer networking scene.

Presumably unknown to the budding literary pirate, the end result included some unseen and unwanted additions: metadata accompanying each image included details of the date, time, exposure and, crucially, the serial number of the camera used. This in turn means that, at least in theory, the camera can be traced to its original owner.

This sort of data creep is becoming more common, and day-to-day applications leave silent and hidden trails of activity all over the place. Usually the intentions behind this activity are entirely benign, but they can be troublesome nonetheless.

For example, Microsoft Word's habit of including much more data than you expect in its document files has caused plenty of high-profile embarrassment.

Blind trust in computer data can have serious consequences too. In the early hours of the morning on 11 March, a US school received a bomb threat to their student hotline.

The call was traced, and a 15-year-old boy, was arrested and detained for 12 days. The school's telephone system had logged the caller ID of his phone at the time the bomb threat was delivered. The school principal was confident that a dangerous criminal had been apprehended.

But the boy was innocent. The school had neglected to implement the Daylight Savings Time change to their logging system's clock, so their logs were an hour out. This is a very simple but serious example of how blindly trusting log data (or indeed any single source of evidence) can be a dangerous pastime.

Interpreting computer evidence such as log files and metadata needs careful handling. It is all too easy to be seduced by the precision offered; the exact time and date a web page was accessed, the specific "click trail" a user followed while browsing a suspicious site and so on. But this precision is illusionary. While the data is usually valid, its link to the real world, from the keyboard to the user, is often tenuous.

Log files don't show the real user, they give the network address or the user's authentication mechanism (username, password etc). Times listed are not the times the event happened, but the time indicated by the logging system. There is a subtle and fragile chain of evidence between event and logging data, and understanding it is essential to accurately interpreting the data. While things usually are what they seem, they can be very different.

When discussing the accuracy of artillery, the military uses the concept of "circular error probable" (CEP), which is an imaginary circle into which there's a 50 per cent chance your munitions will fall. The larger the CEP, the less accurate the system. Like weapons, logging systems have their own CEP equivalents, but it is rare to see any discussion of such errors.

On the occasions where I have had to present log evidence, one of the first things I put in any report is a summary of potential sources of errors.

Like all evidence, computer evidence is open to interpretation and error, but the very nature of computing can make it appear precise and reliable. Although in the legal world, strict standards are applied regarding its admissibility, in the corporate world things are often not so clear cut. It is essential to know the difference between precision and accuracy and to act accordingly.

Nick Barron is a security consultant. He can be contacted at nikb@virus.org.