Measure twice, cut once

The past year has been interesting in the penetration testing sphere of consulting, with a growing demand for the evaluation of security products.

While there is still a requirement for the bulk standard infrastructure-focused pentest, it would appear that many of my clients have become increasingly interested in the detailed security analysis of products they are about to purchase and distribute throughout their network.

In the past, many of my clients would have used pentesting and security assessment services purely as a method of validating the security and integrity of a deployed solution. If their organisation was particularly security-aware, they would ensure that each new deployment or code update was pentested before going "live."

It would seem that these security leaders have moved to the next step – third-party evaluation of security solutions prior to actual purchase and deployment.

For organisations looking to incorporate an enterprise-level software solution throughout their business, they now use pentesters and vulnerability researchers to "stress test" the final selection contenders.

Typically, we are provided with up to eight commercial packages (usually from smaller, specialised solution providers) with which we build small test environments.

Each product then undergoes a black-box zero-knowledge pentest to find probable weaknesses (using a mix of automated attack tools and fuzzers combined with some manual techniques), followed by local debugging of application components that appeared to have misbehaved during the pentesting.

Having completed the first rounds of security testing, we normally produce a report providing a comparative analysis and the likelihood of future exploitation.

Typically, after a few weeks, the client comes back with a second-phase requirement. This normally calls for the pentesters or security researchers to work directly with the selected vendor and its development team, explaining the security flaws to them in detail and giving technical advice on how to fix them.

Several rounds of testing and retesting of submitted fixes or updates usually follow before a substantially more robust and secure solution is "approved" and made available to the client.

While not all organisations would expect to go to this detail in evaluating each new enterprise-level solution, I strongly recommend they do. If past experience is anything to go by, even the best performing product solutions I have reviewed were initially poor and would have been trivial to compromise.

But at the end of the process, the selected product is something that the client (and any other client of the solution provider) should be comfortable adding to their network.

Closely related to this new type of requirement is the increasing number of requests made by the acquisition and merger departments from many of the larger global organisations. They are also using penetration testing as a way to understand the security implications of software or appliance products prior to purchasing.

However, in these cases, the security information is used for slightly different purposes.

Most of the time they wish to know whether the technology they plan to purchase could be integrated and released within their own commercial products without introducing any additional security flaws.

In addition, they like to understand how significant the uncovered vulnerabilities are, and how much development effort must be expended to make the product more secure.

I'm sure much of this information is also used in final negotiations prior to a final sale figure being agreed.

The role of pentesting and product security reviews within large global organisations has grown, and their perspective on security has matured.

This is an exciting period for IT security professionals and consultancies. Security testing is fast becoming a mainstream service to internal business processes, and I'm sure many more opportunities will be coming our way.

Gunter Ollmann is director of professional services at Next Generation Security Software