In a fortified, bulletproof facility in North Sydney, security engineers who once skirted the limits of hacking laws now work to catch those attacking NSW’s state critical infrastructure.
The engineers were talented hackers from Brazil, India, Russia and the former Soviet states, some of whom breached networks while honing their security skills.
Now they spend hours on end pouring over monitors that collect data feeds from networks across all but a handful of the state’s utility companies.
Despite a lack of university degrees and formal security certificates, these young men were hired by managed service provider Earthwave, which counts government bodies as half its clients.
The former hackers -- experienced exploit writers and systems poppers -- are now charged with recruiting new blood for Earthwave, and seem to view education portfolios as padding.
Instead, Earthwave's recruitment process is designed to draw out technical prowess. In early rounds of hiring, hopefuls have to beat a live capture-the-flag hacking challenge to demonstrate their skills and progress to further challenges.
Recruiters appear to have looser restrictions than those publicised by some high-profile security firms.
While it doesn't employ current black hats anywhere in the business, all job vacancies are open to former hackers without criminal records.
Convicted, former hackers are barred from the security operations centre (SOC), but may work in R&D, strategy or sales roles.
One Earthwave hire was a skilled security researcher from India, where he had scratched out a living by reselling his neighbour’s hacked wireless connection.
Another -- who works in the sales and strategy areas and not in the security operations centre (SOC) -- hacked the Australian Taxation Office in the early 90s and was later arrested and jailed.
The former hacker, who could not be named, said the security infrastructure protecting the ATO was then almost non-existent.
"It was just too easy ... We saw a few classified things, but we didn't break anything. It was just incredible to us to see how easy it was to get around."
Others working behind the bomb-proof walls of Earthwave’s multi-million-dollar SOC shared similar tales.
Many had learnt security the same way: as geeky school kids fascinated and obsessed with exploring the seemingly limitless freedom that the internet afforded.
At least two Earthwave staff members were former colleagues in hacking gangs of the 1990s.
"I learnt much more doing my own research in the dead of night than I did in university," said one, on the condition of anonymity. "I teach myself still rather than go to courses."
According to Earthwave's chief executive officer Carlo Minassian the former hackers' experience in reverse engineering and writing malicious code helped them to identify complexity in what others may see as mundane attacks against customer networks.
"You need a hacker to catch a hacker," he said. “These guys from India and Brazil, they have to work damn hard to make money and they are really very good."
Minassian founded Earthwave more than 12 years ago with his brother and two friends who were qualified CISSPs at the time, after his family fled Iran during the Iraq invasion of the 1980s.
About half of the 40 SOC engineers were overseas hires who immigrated to Sydney to work for Earthwave. Minassian and another local engineer who runs the technical interviews said foreign hackers were generally more skilled and worked harder than their Australian counterparts.
Hacking laws may be more relaxed, or perhaps even non-existent where some staff were hired. But Minassian insisted that every effort was made to ensure candidates were not engaged in illegal hacking.
"It is very important that the people we are hiring are not active hackers. We suss them out over many meetings -- they are sometimes naive and will often easily give up what they are doing."
Engineers working within the SOC must first pass “comprehensive” background checks by the Department of Defence and hold appropriate clearances that allow them to access sensitive government information.
Earthwave also has several staff who perform research and development from their home countries and must pass what the company claimed were rigorous audits by staff.
They were denied direct access to sensitive customer SOC data and were instead fed sanitised feeds to help engineers correlate data sets and identify potential threats.
But candidates could not work within the SOC if they had criminal records or were found to have maintained blackhat activities. Minassian said those people were sniffed out during interviews.
On a large monitor pinned to the front wall of the SOC, coloured lines envelop a map of the world linking dozens of countries to Sydney IT networks. Each line illustrates IP addresses that were attacking Earthwave customers.
Using intelligence gathered on the attacks, engineers could hone in on physical locations of the offending machines. That information could then be relayed to police.
Read on to page two for how Earthwave investigated attacks by staff and former staff of McDonald's and a Federal Government agency.
Some of the more interesting attacks were conducted over McDonalds' 860 stores that offer free wireless networks, monitored by Earthwave.
“My colleague gets a call from some company who says someone had logged on from our IPs and deleted their entire file server and web server,” Minassian said.
“The attacker had logged on using a VPN back to his office from a [McDonalds] Stanmore store and used a content management system to delete everything. Then he logged on to his Facebook and Twitter.”
Within an hour, Earthwave had the attacker's name and friends list. The unnamed victim, which would normally require legal authority to obtain details of the attacker, had quickly pinned the assault on an IT staffer who was fired days earlier.
“It was hacking 101. He thought he was safe, but we had his logs.”
Another recent investigation saw engineers tracking IT staff within a Federal Government agency who for months had stolen and forwarded sensitive internal emails to hundreds of staff.
The culprits were caught siphoning emails from the agency’s board and chief executive to external Hotmail and Gmail accounts, and sending those messages to hundreds of staff inboxes from there.
“We went in after hours and put in an appliance which captured every packet that went through. We found the IP addresses linked to those emails and they called the police,” Minassian said.
Earthwave also investigates more serious offences, including a large child pornography ring in which some members accessed monitored web sites from McDonalds stores.
The current investigation spans multiple countries and involves law enforcement from the FBI and Interpol.
Malware writers were increasingly deploying new malware variants against SCADA (Supervisory Control and Data Acquisition) networks, according to a chief engineer within the SOC.
He said those targeted attacks, while generally ineffective, were often a prelude to wider attacks.
“They might feel that SCADA, with all the media hype, is less secure -- which is not necessarily the case -- so they try to target them first.”
But the majority of attacks targeted vulnerabilities that admins had neglected to patch.
In the last month, engineers had spotted an uptick in attacks against Cisco firewalls within the finance sector that targeted vulnerability that was patched years ago. The hole enabled attackers to access the management console on the firewalls.