Inside the build of WA's $200m medical research cloud

When $200 million was committed for the building of two new state-of-the-art facilities for the WA Institute for Medical Research, it was a once in a generation opportunity for the state’s medical scientists to be kitted out with IT that could last them well into the future.

WAIMR North building on Perth's QEII Campus.

Research, by its nature, tends to be funded by sporadic bursts of capital investment, and research organisations tend to get very little in the way of ongoing OpEx funding.

So while most enterprises attempt to shift spending into OpEx (cloud) models, WAIMR had to make good with the money while it was still on the table.

Enter Titan ICT Consultants, contracted to manage the rollout of a fixed and wireless network through the two new buildings. As the project evolved, Titan ICT consultant Bret Watson was appointed as the interim CIO for the WAIMR project.

Watson saw a once-in-a-generation opportunity for the Institute to offer its researchers far more than a building and a network.

The 40+ research bodies that will co-mingle at WAIMR are attempting to solve a complex set of problems - the genetic and environmental causes of a range of diseases such as cancer and diabetes. The organisation itself is complex - its research bodies are funded via various arrangements with state and national governments, universities and other sponsors.

Most of these organisations were supported, at least in terms of client computing, separately -sometimes by the IT departments of universities, sometimes barely at all. Most, Watson said, “lived in a hand-me-down technology world.”

Becoming a facilities provider

When the original funding came through, WAIMR had simply planned to provide facilities to house the research, rather than build an IT department.

The first is a ten-storey building on the QEII medical campus in Nedlands, Perth, due to go live in mid-October; and the second a five-storey clinical research building adjacent to the Fiona Stanley Hospital south of the city, which goes live in early 2014.

Watson interviewed the research organisations that will soon share the two buildings and discovered they had pressing IT needs stretching far beyond a networked building with a BYOD policy.

“There was great vision around the building itself, but somebody had to ask the question - what is the building delivering to these researchers?” Watson said. “What is the network connecting to?”

He realised networking a building so researchers could connect to services delivered over the public internet would not benefit them any more than their current scenario.

Some researchers ran data analytics projects that required server and storage capacity on-hand.

“A number of researchers, for example, also work in-sicilo on a computer-based form of research called gene expression mapping,” Watson said. “The biggest file we discovered on their network was actually 4TB - which was the gene expression map for the cane toad.”

Titan ICT wrote a business case proposing the WAIMR become a facilities provider, suggesting it also morph into a shared service provider for data storage and server processing, and also handle user authentication services.

Watson was essentially proposing the build-out of a community cloud.

Aspiring to Bankwest Place

His plan covered everything from power required for an internal data centre, right up to shared storage and applications.

“There was a desire and a need to move large files around, fast,” he said. “We calculated that it would be cheaper to provide those kind of network speeds within the building rather than via a hosted situation.”

Co-lo hosting was ruled out, as at the time, “there was no co-lo space available for love or money,” Watson said. Resources giants such as Chevron, Inpex, BHP, and Rio Tinto had guzzled all of it.

Further, co-lo or public cloud services would have committed WAIMR to an OpEx model it couldn’t sustain into the future.

“We’re spending known capital versus committing to an expense based on money researchers would have continually had to find from somewhere,” he said.

Rolling out dark fibre to access third party services simply didn’t stack up to the cost of keeping IT in the two buildings.

“When you have a greenfields build, the cost of constructing a computer room inside it is – comparatively - not that great,” Watson said. “You’re not facing the usual problem of retrofitting.”

Watson planned for two 50 square metre shared computer rooms, with the in-row cooled racks physically sub-divided into lockable sections for the various research bodies.

Watson’s network design was based on 40GB links within the data centre (from distribution to core), 20GB between the buildings (across 2 x 10GB diverse paths) and 20GB of bandwidth at the access layer.

The access network was designed on the .1x standard to ensure researchers could plug in (fixed) or log in (wireless) from anywhere in the facility and have their access credentials recognised and calls routed.

“We wanted to build something akin to Bankwest Place,” Watson said.

Combined, Watson calculated the researchers had around 100 TB of data storage in use at their facilities, with quite a deal more stored on the personal devices of researchers. He designed a system with 350 TB of shared data storage in each building, governed by the build of a research data management system - a digital repository and document management system based on the open source DSpace archival system built by HP and MIT.

On the applications front, the scope was limited to Microsoft Lync and Exchange for email, unified communications and room/equipment bookings, Active Directory for single sign-on to shared systems, plus a research community portal based on the open source social networking engine, ELGG, to encourage wider collaboration. The rest was left to research bodies to decide for themselves.

In the wash-up, Titan ICT’s plan was accepted on the back of a five-year calculation of total cost of ownership. WAIMR has since put these specifications to tender for an aggressive rollout over September and October.

IT suppliers have been asked to deliver a turnkey solution, beyond which WAIMR - which now plans to recruit a permanent IT manager - will manage.

“The tenders sought systems whereby additional capacity can be added at any time on a per-node or per-disk basis,” Watson said. “The researchers can request a new VM by raising a ticket.

“But ultimately it is up to WAIMR to decide how they enforce quotas, should they ever need to. It’s a light-touch utility model.”

The rollout of WAIMR's network and private cloud has been submitted as a project to watch for the iTnews Benchmark Awards. Have you submitted your project?