Inside the Commonwealth Bank data centre

In an interview with iTnews, the Bank's executive general manager for enterprise services, Nick Holdsworth, provided a rare insight into CBA's data centre holdings and its strategy moving forward.

Holdsworth said power efficiency, cost reductions and consolidation were key business drivers in the bank's last major data centre refresh, which started in 2007.

"Like others in the industry, we had some older sites and also some mini sites," Holdsworth said.

"What we looked to do is to consolidate and refresh some of those sites over a number of years.

"It really started when we moved out of an older site to a new site in Sydney.

"We used that as an opportunity to drive a lot of consolidation and equipment refresh and also to create a more virtualised data centre outcome".

The CBA has two main data centres in Sydney as well as a host of smaller ‘satellite' centres affiliated with the Group. Some come under the direct control of CBA while others are operated by third-party providers, but all have strict control mechanisms attached.

The main centres are each "a couple of thousand square metres" in floor size and are designed around a 20-year lifespan, "although one would expect there may be technology advances that drive upgrades or refresh requirements during that time," Holdsworth said.

They typically also operate in an active-active configuration, connected by high-speed links. This type of capacity-sharing arrangement is especially critical for managing demand peaks in online real-time systems such as CommSec and NetBank, Holdsworth said.

The centres use under-floor cooling and maximise its effectiveness in part by separating plant from the computing equipment.

"We've taken all non essential equipment out of the raised floor environment, so the amount of space we have to cool is substantially less," Holdsworth said.

"The floors really are exclusively for processing equipment."

He said other cooling options had been considered at the time of the data centre build, but several of the newer options such as free or Kyoto cooling weren't yet on the market.

"If we were to build a site again today it would be different," he said.

The hardware environment is a mix of mainframe, x86 and blade server architectures. CBA is using VMware as the virtualisation component of its consolidation push, but Holdsworth said the Bank "isn't wedded to any [hypervisor] in particular".

"We do quite a lot of virtualisation on blade servers and see some real benefits and efficiencies in doing that," he said.

CBA also backed its mainframe environment, revealing there had been internal discussions recently around consolidating more workloads into a mainframe environment.

"Often people use mainframe in a derogatory sense," Holdsworth said.

"In fact I think there's some potentially effective uses for mainframes today. In some ways it's an efficient way to consolidate workloads.

"In many ways people are hankering to get back to well-managed and controlled environments. After things became really distributed, we saw the cost of that [approach] and in many ways the drive to consolidate to increase the management and control of high-availability platforms is kind where we came from [with the original mainframes].

"As we look to consolidate and get the mix of cost effectiveness and delivering service outcomes right, we'll continue to use mainframes and view them as another platform [available to us]."

Despite the use of blades, Holdsworth still views average rack density at between one and two kilowatts (1-2kW) -- far lower than the likes of Global Switch and Digital Sense.

"I think 2.5 or 3kW is pretty heavy," he said.

"We're continuing to see average densities go up but I guess the interesting aspect is most of the new servers [when they are released] actually save power even though the density increases.

"If you keep up refresh cycles you can manage your power allocation reasonably well and buy yourself a lot of headroom [in terms of your power allocation] by doing it".

Read on to page two for details of CBA's predictive IT build strategy.

CBA's longer term plans for its data centre presence appear to be to reduce its reliance on the space altogether.

At the very least, the Bank appears to be pursuing a strategy to convert capital expenditure in the data centre into an operating cost, by purchasing IT infrastructure and capacity from third-party providers where appropriate rather than buying more space and kit to do it themselves.

"We'd like to think that instead of our data centre requirements continuing to expand, we'd like to see them start contracting," Holdsworth said.

"We don't want to just rely on leasing, renting, buying or building data centre space to run everything. But in saying that, we'll still have a lot of things we want to manage internally."

As this suggests, Holdsworth is the first to admit it's not a model that will sustain all of the Bank's workloads, but it's one that CBA is currently exploring with its suppliers.

"More and more we say ‘why can't we buy infrastructure as a platform? Let's not buy and build it ourselves. Why cant we tell you we need capacity and load it in?'" Holdsworth said.

"I think suppliers are becoming much more receptive to those types of conversations."

Holdsworth said CBA continues to explore both public and private cloud-based computing services.

"We've done quite a lot of thinking around cloud but you've got to use the right tool for the right workload," he said.

"The type of workload you push to the cloud are things that don't have security clearance issues. They could just be really short bursts of a particular workload or you could place workloads in different places depending on the time of day or price of service.

"We're experimenting with that now".

CBA already runs compute capacity planning in its operations and has its outsourced providers bring storage and server capacity online up to three months ahead in anticipation of future demand.

"We try to keep a close eye on demand by forecasting it and trying to build out infrastructure at least two or three months ahead of forecast demand," Holdsworth said.

"Most of the providers we work with are happy to build out based on demand. [The other advantage is that] we only start paying for that equipment when we switch it on."