AusCERT2012: Anti-virus a SCADA kill switch

“Catastrophic failure” would have hit a major US gas utility last year if security technicians had run anti-virus to remove a prolific malware infection.

The malware had embedded itslelf into dozens of machines within the unnamed utility provider and created malicious files that bore identical filenames to those that were critical to the operations of the core SCADA (Supervisory Control and Data Acquisition) system.

Had AV been run, it would have removed both the infection and the critical files, grinding energy production to a halt.

“If they had run AV, they would have erased the core files for the SCADA system destroying its ability to function,” says Mark Fabro, president and chief security scientist of Lofty Perch.

“But we have seen entire catastrophic failures of very, very big SCADA systems.”

One such failure affected a large US critical infrastructure operator within the natural resources sector. Technicians had run an AV scan which quarantined and removed files in an underlying operating system that crippled the SCADA system.

But another operator was more lucky. In a previous instance, Fabro had intercepted and physically prevented security technicians from running AV to remove a worm found in machines critical to SCADA systems.

The security team had planned to “parachute in” and deploy AV to kill the infection as part of normal incidence response. But in doing so, administrative rights would have been enabled, allowing the malware to propagate throughout operational shares and spread through the critical infrastructure network.

“The only thing that stopped them is that only the master SCADA, and not the secondary systems, were running in administrative mode,” Fabro said.

“We had to physically stop the IT security team from logging in as administrator because the moment they would have done that, the worm would have rushed through the SCADA system and eradicated the functional files.”

The US power plant had averted disaster by not reacting on gut instinct when it received news of the infection from a vendor technician. He discovered the malware on the flight home from the plant when he plugged a USB stick, last used on the SCADA system, into his laptop. His AV flagged an infection.

Fabro was called and together with a crack flyaway team, took off to the plant. It was their job to establish the extent of the infection, how it got onto the network, and how it could be removed.

“This was a non-trivial task,” Fabro said.

The multi-tiered, high-powered facility could not be taken offline for system imaging during the forensic investigation. The team was further constrained because it had to work around plant operations, engineers and even the weather.

Further, the network was a homogeneity of different vendor systems, some with and some without AV.

The existence of the malware was confirmed in two principal devices at the primary facility, and the team developed a theoretical map of what the malware would likely happen and where it would spread.

Over the following month – much of it spent waiting for an opportune time to access the systems – the team would use live memory analysis to tap into memory and perform analysis on rebooted systems, identify infected systems, utilise backups, and recreate the the infection process.

The team had tracked the likely origin of infection to four removable drives, including an iPhone, which were plugged into the system at the time it was built. They recovered serial numbers for the suspect USB sticks and even photos left on the system that were owned by the engineers thought responsible.

Like similar infections, the malware was run-of-the-mill and not tailored to take down the SCADA system. It was, as Fabro describes, a “pain in the arse” botnet malware designed for denial of service attacks which, left unchecked, would have disrupted the SCADA systems by draining resources.

For Fabro, the investigation highlighted the importance of tailored incident response for critical infrastructure, the need to synchronised clocks which is vital in forensic investigation, and the realisation that operating systems underlying SCADA applications could be vulnerable.

“There are about 250 vulnerabilities in vendor-specific SCADA platforms, but if someone can hack into the underlying Windows or Unix environment using a vulnerability, they can go up into the SCADA system just as if they broke into the system itself.”

Copyright © SC Magazine, Australia