Inside the CBA's Oracle-as-a-service project

How the Commonwealth Bank saved on shared database.

The Commonwealth Bank has deployed PeopleSoft 9 financials suite - the first production application to use the second generation of the bank's shared database service.

Nicholas Tan, head of infrastructure and platform solutions at the CBA delivered attendees at the Oracle OpenWorld conference an in-depth look into the bank's journey to offering internal business users access to IT resources on a utility-style basis. 

Since 2007, the CBA has consolidated some 300 Oracle database schemas onto just three database instances.

The shared service essentially means that multiple bank applications access the same database instance on a large machine rather than being provisioned with smaller standalone database servers, and business units are charged by the IT department on the basis of how much of the resource is used.

The bank began its journey with Oracle using a cluster of Sun Microsystems E25K enterprise class servers running Oracle's 10G database for such applications as the bank's CRM (ComSee) and online banking (NetBank) systems.

In May 2008, the bank embarked on a project to deploy Oracle database-as-a-service using commodity SPARC-based mid-range servers running the Oracle 10G database.

Then in July 2010, the bank deployed its latest shared database platform - the Intel-based Exadata database machine, which is used for applications running the Oracle 11G database.

This latest generation of the bank's shared database platform is deployed on two Exadata machines - a full rack being used for development and production instances (including the PeopleSoft implementation), and a half-rack located 50km away being used for both disaster recovery and testing.

BILLING AS A SERVICE

The Commonwealth Bank's business units are billed for use of the shared database platform according to CPU-usage per month, plus an additional charge for monthly data storage.

Tan told attendees the bank's chargeback model "took a lot of time to work through".

The IT team began by suggesting a variety of metrics to pull out of the system. These included database time, database CPU, physical reads and logical reads.

But Tan explained that these granular methods of calculating chargeback proved too confusing for business users.

"The business doesn't understand them," Tan said. "So we settled on a basic CPU per-month charge - if only because it was simpler to understand compared to other available measures from monitoring tools such as DB Time."

The IT team set a base hosting fee of 0.5 CPU per month, with monthly usage being calculated in 0.5 CPU increments above and beyond the base.

Tan said that to charge in increments as low as 0.1 CPU "would be overkill".

Storage utilisation is also measured and charged on a monthly basis, and the cost of disaster recovery services are built into the monthly bill.

The bank went to considerable effort to ensure buy-in across the business for the as-a-service model, combining a top down effort at enterprise architecture level as well as a bottom-up series of one to one change management briefings for business unit system owners.

Some business users, Tan said, had slipped into a mode of "set and forget" for cost of supply and were naturally surprised by their use of database resources under the pay-as-you-go model. 

"There were many conversations along the lines of 'I haven't budgeted for that'," he said.

But in the medium term, most business users found that the pay-as-you-go model saved money as their business unit didn't have to pay for resources during low utilisation periods.

It also alerted business managers to areas of IT waste. Using Oracle Enterprise Manager, for example, a business user could be provided a list of top ten bad database queries. Business users now had a cost incentive to reduce this overhead.

DBA's [database administrators] were soon sent messages such as "please fix this code, it's costing me a bomb," Tan said.

"Once you have an incentive to look closely at the use of these resources - you tend to end up with lower costs in total."

By time-sharing the database resource, the bank has now 'booked' more CPU-hours than the number of CPUs in the machines - a far more efficient use of the asset.

Read on for more on business benefits and how the bank ensures system reliability...

BUSINESS BENEFITS

The 'Oracle-as-a-service' strategy has already paid for itself, Tan said.

The project broke even in its first year and was cash positive in its second.

Tan said the CBA had enjoyed a "significant reduction in [the cost of] servers, associated licenses and hosting charges". He said the bank only had to increase its database operations team by one to two people despite adding hundreds of new services.

On a per new application basis, the Oracle-as-a-service approach costs 40 to 50 percent of the cost of building a standalone database server for each app, he said.

It has also led to far speedier provision of new services.

Tan said Oracle database is "traditionally hard to provision" at the speed in which users demand new applications. Under the CBA's strategy, Oracle database "can now be delivered on tap," he said.

"A new development environment can be available within hours, not weeks or months.

"It has removed the need for expensive servers and the bespoke implementation time involved with a highly reliable solution."

For any application, he said, the strategy has allowed the bank to remove the line item for building, deploying and integrating databases out of the project business case.

In maintenance terms, Tan said the shared platform means the bank only has to "patch its database once" and can leverage a single implementation of toolsets across many business systems.

The strategy also provides IT a single point of control for database.

SECURING RELIABILITY

Tan's presentation was met with significant interest by other end user customers at OpenWorld. Streams of questions followed, mostly by users concerned as to how they might secure and guarantee the performance of applications running on a shared database environment.

One end-user asked what would happen if one application on the platform went haywire and impacted the performance of others.

Tan replied that the bank employs a dedicated team of operational DBAs to manage the platform.

"We have a gun team that runs the database," Tan said. "If any business app goes rogue, we turn that service off - which is much the same as if a normal [dedicated database] server had a problem."

The bank uses the resource management and session management modules of Oracle database to "constrain the resources made available to a particular workload in terms of I/O," Tan said.

This enables the bank to protect transaction processing applications (such as online banking apps) from the long running queries in decision support workload (such as queries to or reports from the data warehouse).

Tan said the shared service approach also enables smaller applications to get the "same DBA attention" as larger ones.