University of Tasmania loses $5m HPC cluster to floods

The University of Tasmania lost its $5 million 'kunanyi' high-performance computing (HPC) cluster to last week’s freak storms in Hobart after the data centre housing it flooded.

Kunanyi as it was assembled in the eResearch data centre (Credit: Iain Sheppard/Youtube)

The cluster, which was built by Huawei and contains 256 E9000 nodes and 7168 cores, was located in a “new purpose-built research data centre at the University’s Sandy Bay campus”.

It was only brought online last year.

Hobart experienced record rainfall and flooding on Thursday evening last week, with the university's Sandy Bay campus among the hardest hit.

The campus was only reopened on May 14. Its engineering and corporate services buildings were among the worst affected.

The engineering building was home to the kunanyi HPC cluster, the largest HPC at the university.

A brief advisory sighted by iTnews states that “the data centre housing kunanyi has been inundated with water as a result of the extreme weather event that occurred on Thursday.”

Photos circulating on social media from the university’s School of Technology, Environments and Design showed water levels peaked at about shoulder height in some rooms, waterlogging compute infrastructure at desk height or below.

“Extensive flood damage occurred outside of the eResearch data centre,” University of Tasmania CIO Jeff Murray confirmed to iTnews.

“The data centre itself was not exempt and has suffered some damage.

“The [kunanyi] cluster is currently out of service.”

It is not clear if any part of kunanyi is salvageable.

iTnews understands the most likely scenario is that the cluster will need to be fully replaced, though a final decision on that is yet to be made while the university continues to mop up.

Murray said that “the university’s current focus is on providing sufficient [HPC] capacity to its researchers”.

“We are working on a number of avenues that will see capacity incrementally restored over the coming days and weeks,” he said.

“Both [our] Katabatic and Vortex HPC clusters were unaffected and continue to provide computing capability to the research community.”

Both HPCs, however, are substantially smaller than kunanyi, being each based on a 512-core SGI Altix Ice 8200.

In addition, of the two, Katabatic is shown to be suffering from undisclosed “performance issues”, while Vortex is running at full capacity.

Murray said that “no other production services were impacted nor was data lost” due to the flooding of the eResearch data centre.

Kunanyi was announced in December 2016. It was built to handle research in “fields such as engineering, health and medical research, with a focus on genomics.”

It was also designed with expansion in mind.

The university said at the time that it could “accommodate a tripling in size of the HPC environment when required.”

A time lapse of the HPC's assembly is available here.