Primus reveals cause of data centre blackout

In an interview with iTnews today, chief executive Ravi Bhatia ended almost two months of speculation over the causes by laying out a chain reaction of events that led to the failure.

Bhatia told iTnews that external power to the data centre was throttled due to the extreme weather conditions in Melbourne.

"The severe weather conditions imposed exceptional demand on the public grid and our captive diesel generation capacity," Bhatia said.

"In this period, the power supplier had reduced the data centre substation capacity significantly below our customary load without informing us and without prior notice."

The extreme weather also meant power demands on the centre were higher than normal.

Customer and plant demand on the now-reduced power supply pushed ambient temperatures in the centre beyond its design limitations, according to Bhatia.

Generator capacity also declined because of the higher-than-usual temperatures.

"A sequence of events happened. The high-tension fuses [in the substation] went and the diesel generators started against a heavy load," Bhatia explained.

"The windings in the generator were already hot from the higher ambient temperature so the thermistors tripped and cut in to protect the generator from blowing up."

The tripping of both the external and back-up power supplies appears to be the primary reason for the outage.

Bhatia has confirmed that Primus had one diesel generator at the time of the failure.

It has since put in a second diesel genset at the 55 King Street site.

At the time of the failure, the number of generators on-site was a significant source of debate among customers.

Primus has also distributed some of the load from the centre to its new data centre located across the road.

This site includes an additional three diesel generators for backup power supplies.

Bhatia said the new genset [generator] redundancy will enable the centres to cope at ambient temperatures exceeding 50°C.

"Melbourne Internet Data Centre has operated with an excellent record over the last five years with availability exceeding 99.99 per cent," Bhatia said.

"The incident in February is unusual."