Melbourne researchers consider new 768-core machine

The Walter and Eliza Hall Institute of Medical Research (WEHI) is considering a new 768-core high performance computing (HPC) cluster to boost its resources for bioinformaticians.

The new cluster -- the Melbourne-based institute's fourth -- will be used by cancer researchers to make sense of tumour genome sequence data.

DNA from cancers is fragmented and sequenced using next-generation sequencing platforms.

“This generates hundreds of millions of short DNA sequences, which are then mapped back to the human reference genome,” WEHI's head of IT services, Dr John Wastell explained.

“To make sense of this data, we have developed data analysis tools which run on our HPC servers for finding chromosomal rearrangements in the tumour.”

WEHI currently operates three HPC clusters: One dedicated to bioinformatics (280 cores); one dedicated to structural biology (164 cores); and one dedicated to proteomics (204 cores).

All the clusters are built with commodity Intel processors and use UNIX OS. The institute also has "a number" of 64-core blade servers with 0.5TB RAM each to run bioinformaticians' smaller jobs.

In addition, WEHI uses the supercomputing resources at the Victorian Life Sciences Computation Initiative (VLSCI) located at Melbourne University but prefers in house infrastructure.

During the past three years, WEHI has largely replaced its data centre infrastructure, relocating to a new facility and investing heavily in networking, high-density blade infrastructure, virtualisation and storage. 

“We transitioned our entire network to Juniper, with 10GbE connectivity to all blade servers,” Wastell said.

“OSs are a mix of Windows Server and LINUX. Our virtualisation platform is Hyper-V. We still have a few racks of legacy servers to be virtualised which we are working through as time permits.”

The institute has approximately 700TB of fibre channel disk and is investing in adding a tape tier to its NAS, with a view to shifting infrequently used data off the disk and onto tape.

“A nice feature of tape is that it draws no power whilst not being accessed,” Wastell said. “We have purchased a pair of Quantum AEL6000 tape libraries which will enable us to scale to many petabytes in the tape tier.

"Backup and disaster recovery concerns dictate that we replicate all data to our secondary site at Bundoora. We have been fortunate to be able to implement a 1Gb/s fibre link between the two sites to facilitate this.”

WEHI's plan for a fourth cluster follows several other HPC announcements in Australia in recent months.

Melbourne’s Swinburne University of Technology last month deployed 86 new supercomputing nodes to complete its two-part, $2.6 million ‘Green II’ machine.

Meanwhile, Geoscience Australia unveiled plans to use the National Computing Infrastructure's 1.2 petaflop x86 supercomputer to understand the impacts of mining on the Earth's surface.

Compared to Europe, the US, Japan and China, Australia is a small market for HPC.

However, there is common acceptance in industry that HPC installations can provide significant benefits to many industries such as medical research, manufacturing, climate research, oil and gas, banking, defence, and so forth.

In 2010, research firm IDC noted in a report commissioned by the European Commission that a holistic HPC strategy could contribute as much as 2 to 3 percent to the region’s GDP by 2020 (pdf).

Globally, IDC expects the HPC technical server market - which does not include the storage, networking, software and services revenue associated with HPC - to expand by 7.1 percent in 2012, which would be a record-breaking year.