The faulty HPE SANs behind the Australian Taxation Office’s three-day December outage will be flown back to the vendor’s US headquarters to undergo “forensic testing" to find out what went wrong.
ATO commissioner Chris Jordan told senate estimates this afternoon that he was still incredulous that the seemingly simple storage infrastructure fell over and caused so much damage to the tax office’s online services.
“The SANs had only just been installed in November 2015, and were still considered state of the art,” he said.
“On the spectrum of IT services, storage is something that usually that just works. It is not considered high risk.”
The remediation effort was just as problematic, with another lengthy outage in early February linked to the restoration of storage systems and the 1PB of data that was lost during the initial collapse.
The ATO has commissioned PwC to review the incident and deliver a report on how to prevent this kind of high-profile disaster from happening again.
Jordan revealed that the ATO’s long-term supplier HPE would also conduct its own hands-on audit of what caused the 3PAR SANs to fail.
He said as soon as it was safe to do so, the two SANs will be sent back to the US for “forensic” testing, which is most likely to occur in June.
However, the public may never know the full story behind what locked taxpayers out of the ATO's online services, with the commissioner cautioning that some parts of the PwC report - which is due this month - might not be published due to “legal commercial sensitivities”.
He reiterated HPE’s claim that the SAN failure that undermined the ATO had never been seen before across some 67,000 organisations worldwide using the same equipment.
The vendor has since promised that it will make it up to the ATO by replacing the storage devices with what it has promised will be “the most solid equipment in any Hewlett-Packard installation in the world”.
The ATO chief said HPE has been “quite open in acknowledging the failure of their services” to the government agency.
