Root cause of ATO outages still unclear

The Australian Taxation Office is still yet to receive a "definitive answer" on the root cause of the high-profile twin outages to the agency's 3PAR storage area network.

Three separate investigations have been undertaken into the December 2016 and February 2017 failures.

An internal ATO review was made public last June, and a full copy of a commissioned post-incident review by PwC was published towards the end of last year [pdf].

The PwC report raised questions over the status of the third investigation - a root-cause analysis that was to be conducted by HPE, now known as DXC Technology.

This investigation had been delayed due to difficulties extricating some of the components from the ATO's environment.

The ATO earlier this year said the faulty SAN had been provided to DXC for forensic analysis at some point after last July.

It had previously warned that it could not be certain of the root cause until this analysis took place.

The agency's chief information officer Ramez Katf today told a parliamentary inquiry the root cause was still unclear.

"We have not received a definitive answer as to whether the cables were defective themselves, or the way they were installed was the defect," Katf said.

DXC has been contacted for comment.

The forensic investigation is crucial because DXC says the ATO was the first customer in the world to experience such an issue with its SAN.

The ATO's own investigation revealed a confluence of events contributed to the outages.

It reported that the fibre optic cables feeding the SAN were not properly fitted, software bugs on the SAN disk drives meant stored data was inaccessible or unreadable, back-to-base monitoring tools weren't activated, and the SAN configuration was more focused on performance than stability or resilience.

The result was that multiple cables failed across eight of the ATO's 200+ disks, impacting all of its systems.

The ATO reached a commercial settlement with DXC over the failure.