Certifying Australia's best pen testers

This article first appeared in SC Magazine's March edition

Penetration testing is a mystery to many businesses. Organisations seek ethical hackers to identify vulnerabilities in their systems before criminals do but they often can’t see the line that differentiates a green tester from a veteran.

Price isn’t a reliable indicator of quality, nor is the size of a pen testing business. And because inexperienced testers can impress clients by breaking into some systems, substandard tests may appear good – at least until the client is hacked.

Now a group of security professionals, industry personalities and customers from both sides of the Tasman want to identify skilled penetration testers with the CREST (Council of Registered Security Testers) certification, founded in Britain in early 2008.

To earn it, testers will have to pass a gruelling hacking test and may pay thousands for the privilege. In return, CREST Australia promises to promote certified professionals to the country’s largest and wealthiest corporations as the best in the business, able to seek out every nook and cranny that hackers might use to steal sensitive data and cause chaos.

CREST Australia has bold objectives, hopeful supporters and quiet critics. Its supporters are influential, and include some of the most experienced and successful penetration testers, wealthy corporations and the Australian Federal Government.

The non-profit group began taking shape some years ago through informal conversations within security industry circles but it only came to life late last year when its then-unofficial board was formed. That board pitched the certification to CERT Australia, an information security clearing house within the Federal Attorney-General’s Department, which offered to bankroll the initiative.

CREST Australia was registered with the Australian Securities and Investments Commission on 15 February.

When it launched in 2008, Britain’s CREST had 15 member organisations, including Ernst & Young and the British National Health Service (NHS). At the time, a CREST advisor from the NHS described it as a means to ensure organisations avoided “getting someone off the street who ends up crashing the system”.

By September last year, CREST became the official certification body of the Communications-Electronics Security Group (CESG), the British Government’s information assurance authority. CESG is responsible for supplying penetration testers for the most sensitive jobs – classed up to the ‘Secret’ classification – for government agencies, the military, law enforcement and critical national infrastructure providers like energy and water utilities. Britain now has some 28 CREST-certified organisations.

The Australian Attorney-General’s Department did not return repeat phone calls and emails to discuss its support of CREST Australia. The value of the Federal Government’s monetary contribution is unknown, and it was yet to be issued at the time of writing.

CREST testers are likely to appeal to the department’s CERT Australia, which is positioned as an information-sharing hub and data breach guardian of the same high-end and critical infrastructure organisations that the certification is designed to target.

“Where I see CREST positioned is to companies who may be targeted by pretty sophisticated criminals or nation states. Those companies have a position in our economy, so that if their security is poor, it is bad for the country,” CREST Australia chief executive officer Alastair MacGibbon says. “That is ultimately where the government will be most interested.”

CREST Australia will serve the top-end of town, MacGibbon says, describing big ASX-listed and private companies that are prepared to pay good money for the right test, and not to the numerous basement pen testers. “They want more assurance of the quality and skills of pen testers, and the ethics of the companies they work for,” MacGibbon says.

Paul McKitrick, chair of the New Zealand Internet Task Force that houses the working group to establish CREST New Zealand, agrees. “We are concerned that as the information assurance space grows, new players will enter and the high standard of the current market won’t be maintained. We are fortunate to have some quality boutique firms with talented people and we want that level to be maintained.”

Next: Known and unknown

Known and unknown

Much detail of what the fledging Australia and New Zealand chapters of CREST will do is unknown because both are in formative stages. They are eager to declare that final decisions ultimately rest with board members and are several months away, if not more. What is known is that the CREST bodies will require pen testers and their organisations to pass – and pay for – rigorous audits and examinations on a recurring basis. And chapter members have independently flagged their ambition to preserve uniformity where possible with the CREST model in Britain. 

If the British structures are maintained, it will likely mean CREST-certified pen testers could be recognised and find easy work within Australia, New Zealand, Britain and Canada, where a CREST chapter is also forming. McKitrick and others also are considering setting up buffers to prevent large Australian firms from “cannibalising” the smaller New Zealand market, and an trans-Tasman auditing model, under which Australian CREST companies will be audited by kiwi pen testing firms and vice-versa.

CREST examinations, audits and fee structures will need to be tweaked to suit local laws and regulations, even if chapters on both sides of the Tasman agree to implement them as uniformly as possible. In Britain, CREST charges £7000 ($A10,355) a year for company membership, £1600 ($A2367) plus tax for the senior CREST Certified Tester exam, and £395 ($A584) plus tax for its entry-level CREST Registered Tester exam. CREST certifications are valid for three years.

While CREST Australia will receive government funding, the New Zealand Government has yet to offer monetary support. But McKitrick says it has substantial industry support and has established a budget, though he did not reveal the figure. “We planned it early as a two-year project because we did not want to later make a knee-jerk reaction,” he said.

McKitrick, MacGibbon and others involved in the CREST initiatives say the certification was chosen by consensus after other security certifications were considered, including Britain’s TIGER certification which rivals CREST.

By the year’s end, CREST New Zealand hopes to set up a grandfathering program, under which pen testing firms will be encouraged to become CREST-certified. It is now drafting criteria for the program. McKitrick says the formation of CREST New Zealand has been predominantly led by pen testing customers, with a smaller representation from suppliers. “We wanted to start this from the buyers’ community because we wanted to make sure it is objective, unbiased, and that no one could say we had colluded with pen testing firms to set the bar too high for others.”

The CREST Australia board includes MacGibbon, colleague Nigel Phair, Datacom TSS general manager Richard Byfield, Telstra CISO Glenn Chisholm, AusCERT general manager Graham Ingram, StratSec CEO Tim Scully, and NGS Secure Asia Pacific general manager Wade Alcorn. The CREST NZ working group, established from the taskforce in late 2010 but not yet a registered company, is composed of representatives from the Bank of New Zealand, Kiwibank, the Department of Internal Affairs, the National Cyber Security Centre, and pen testing firms Insomnia Security and Lateral Security.

Next: Pressure test

Pressure test

Adam Boileau, pen tester for Insomnia Security and board member of CREST New Zealand is one of the few in Australia and New Zealand to have road-tested the CREST Certified Tester exam. “The technical level of them is not that high,” he says. “What it really tests is your ability to work under time pressure and without internet access, which isn’t something many testers are used to and most people will fail the first time around.”

It was this race against the clock, coupled with a lack of internet access for half the exam that saw the respected veteran pen tester flunk. And NGS Secure’s Alcorn says others have too. “These were skilled testers, really experienced guys,” he says.

Pen testers are rarely pressured for time during professional tests. Their work is thorough and considered, and they have online access to download tools and access information as it is needed. Moreover, penetration testing is anything but an exact science, and the industry is composed of professionals who vary as much in experience as they do in their approach to hacking.

The CREST Certification Examinations are billed as placing pen testers “recognisably at the top of [their] game”. Certifications are divided into two tracks, with the Infrastructure Certification Examination assessing “capabilities in the field of general infrastructure and operating system security assessments” and the Web Application Certification Examination assessing testers’ ability to find vulnerabilities in bespoke web applications.

Both tracks contain essential written components including 90 multiple choice questions answered closed-book and offline, and three long form questions conducted open book and online. Written and practical components of the examinations are sat consecutively, lasting a day. 

Boileau and Alcorn back the certification, warning that the exam environment is something that candidates will need to prepare for. They note the preparation might come at a cost. “The preparation time for some of these exams could be two or three weeks and if you’re taking that out of your consultancy every few years, that’s a reasonable amount of cash for a boutique,” Boileau says.

Those weeks, however long study may take, are worth the cost, McKitrick says. “I think many will see it as a cost of doing business and it has benefits to the tester and the company they work for.”

Next: Commendations and critiques

Commendations and critiques

SC Magazine Australia has spoken to more than a dozen seasoned penetration testers about the CREST initiative and found most in support. They say it has the potential to stomp out some of the basement traders and will demystify the penetration testing industry for customers.

MacGibbon says those cheap and simple pen tests have a place, but not with big businesses. “When you buy a vehicle, you have a wide choice, and you basically know what you are getting. That’s not the same in security. But the cheap and cheerful car has its place and it’s the same in security.”

The CREST certificate will also help forge a career path for university students into the penetration testing industry. “Students and others will have a career path that’s going to help them get into the industry and learn the right skills,” Alcorn says. In the same vein, CREST could help address information security skills shortages in both Australia and New Zealand, McKitrick says. “There is a skills shortage and that would allow us to import talent, and we would know without necessarily going through the entire interview that they are up to scratch.”

Both CREST chapters have said they intend to keep fees as low as possible while ensuring the non-profit companies are sustainable.

Critics – experienced testers who requested anonymity – say the standard is irrelevant to those who have the experience and reputation with the same customers CREST aims to serve. To them, CREST is a cost. But they have reserved further opinion until details emerge later. “We know our stuff already and don’t need a certificate to tell us or our clients that,” one pen testing director said. Indeed, many claimed to enjoy repeat business from clients which they say CREST would not affect. Others were concerned about how it may affect doing business in New Zealand.

Wil Allsopp, principal consultant with Verizon Business and a veteran penetration tester, says the abilities of a pen tester is best learnt by reading their CV. “My opinion is that certs for pen testing are pretty worthless all round for experienced testers; you should be able to look at someone's CV and within a short time interviewing them have a pretty good idea of where they are.”

Allsopp has taken the CHECK test, equivalent to senior CREST infrastructure certificate and run by CREST in Britain. His criticisms came despite having hired and been hired based on the certificate. His criticisms about the technical relevance of the exam are shared by others contacted by SC Magazine Australia.

“CREST … assumes that pen testing is a single discipline that can be baseline evaluated,” he says. “It isn't and you can't. To be a good pen tester takes time and experience because you need a very broad knowledge of operating systems, databases, languages and so on.” That broad experience doesn’t translate well in an exam, Allsopp says, arguing that a question asking ‘what stored procedure would you use to compromise an MS SQL 2005 server’ is unfair when examiners accept only one answer (xp_cmdshell) of dozens.

Copyright © SC Magazine, Australia