ING DIRECT puts core banking on x86

Achieves 'bank in a box'.

IT architects at ING DIRECT have rewritten the enterprise software development rule book after migrating the company's core banking applications off mid-range systems onto the commodity x86 platform.

The project has led to the development of the bank’s widely publicised “bank in a box", which provides the capacity to replicate a virtual instance of the banking platform for developers to patch, modify or extend in an isolated sandbox.

Ben Issa, head of IT strategy at ING DIRECT, sat down with iTnews in Melbourne this week to discuss the nuts and bolts of the solution.

Though not as old as the large and complex legacy systems in use at Australia’s big four retail banks, ING DIRECT's systems were hosted on mid-range servers running HP-UX until last year.

Issa credits chief information officer Andrew Henderson for backing his IT team's attempt to migrate onto the x86 platform, which became the catalyst for more fundamental changes to come.

“From the day he said ‘I'll back you’, within 20 days, we had the entire back-end running on x86,” Issa said.

Issa then saw the opportunity the bank’s developers could enjoy once the core banking application was sitting in a virtual container on the standardised platform.

"We hear discussions around big bang complicated response to these same problems," he said.

"But that is a complicated response to complicated infrastructure. What if you rationalised it to a point that you don’t need high end solutions? What if you could get away with commodity solutions? You don’t need a Ferrari to drive to the shop."

The "bank in a box" was built using technology partners including:

• Cisco's UCS for server processing and memory grunt;
• NetApp’s FlexClone for a data storage array with smart replication capabilities;
• Microsoft’s Hyper-V as a hypervisor that would work with the bank’s existing development tools; and
• Dimension Data as systems integrator to pull them together.

The result allows developers to make changes in a source code repository using Microsoft’s Team Foundation Server and test those changes against a virtual instance running the bank’s entire transactional banking system.

Test and dev

Using virtual instances for test and dev purposes isn’t particularly new. The beauty of ING DIRECT's solution is that multiple developers can take an end-to-end snapshot of the banking system, make changes, and have those changes harmonised against another cloned image to ensure they work in the production system.

Since November 2011, developers and engineers at ING DIRECT have executed some 395 copies of the banking system for everything from system patching, to the development and testing of new transactional capabilities, to replication of a customer’s environment to hunt for bugs.

Issa told iTnews that prior to the project, replicating the core banking system’s servers, applications, configurations, DNS records, database structure and 5.5 terabytes of data might have taken ten developers 270 days.

The same can now be done in a virtual instance in ten minutes, he said.

At the time of writing, there are 39 active full copies of the banking system running at ING DIRECT. Using FlexClone, the total data hosted isn’t much more than the 5.5 terabytes in the production array.

“The technology allows you to take a copy but not consume any more data other than incremental changes," Issa said.

"If you think about how much data one developer will usually change in a program, it might only by two megabytes. That delta is nothing in the scheme of things. And the developer is actually changing what’s in the source code repository, not what is in the container.”

In other words, the bank doesn’t have to worry about VM sprawl or massive data replication.

“We don't have to tear down every copy. We can either tear it down once we’re finished with that project or park it for later use,” he said.

The bank still has governance in place but focuses around release management before code goes into production, not so much for code from delivery.

“You have to ask yourself when you're going to put controls in, what is it that you're governing?" Issa asked.

"If it is sprawl, why is sprawl is a problem? Is it the cost of multiple software licenses? I say if cost is your problem, why are you solving it with governance? There are other ways to solve the cost problem.

"You go to your vendor and say, 'I want my delivery environment as agile as possible. We expect you to support us, and while we’re happy to pay in production, in development we need you to think different'.

“So yes, we have governance, but its lightweight and fit for purpose - not red tape for the sake of red tape.”

Read on to see how the solution has changed test and development on other projects.

Issa said the solution is such a large change that attempting a calculation around return on investment is akin to trying to calculate a similar return on the difference in time between finding information in a library and searching it on Google.

“We tried to do the maths on ROI but the numbers are so huge it looks absurd and meaningless,” he said.

“The fact of the matter is, we can get more projects out. The real net result is we have changed the game.”

The project has enabled the bank to justify why it keeps a 170-strong system development and testing team in-house rather than offshoring as many of its competitors have.

“We can have our developers innovate,” Issa said.

“We can ask our developers to put their money where their mouth is - here is the capacity to show us, let’s see what you can build."

The bank has now formed DIRECT Labs, an initiative that encourages the entire team to conceive of and test radical new ideas - sometimes without authorisation.

“The way we deliver has changed fundamentally,” Issa said.

“A developer might say, in their own time out of curiosity, what would happen if I kill a database in the middle of a transaction?

“The old way, the cost of writing that test case if very expensive. Being innovative came at a cost to the organisation. But now, if it costs relatively nothing, why not try the idea out? Why not test if you can re-architect an application to gain a millisecond better response time?”

The new architecture means that multiple projects can be running and easily tested against each other in a sandbox before going into production.

DIRECT Labs doesn’t necessarily need to involve additional monetary rewards for developers, he said.

“Developers are still remunerated for the hours they do. A developer is truly rewarded by having their ideas come to fruition," he said.

"There is a reward when you walk into a contact centre and see people using your change, and they turn around and tell you that whoever wrote this is a legend.”

Time to dream

Issa said the solution has sparked a wave of new thinking within the development team.

The bank, he said, could, reconsider having cold standby servers synchronised for disaster recovery. An entire core banking system might in fact one day fit on a USB stick.

It might not be practical to expect that today, according to Issa, but to prepare for such a possibility isn’t so out of this world.

Issa concluded that too many organisations assume they need to pay large amounts of money to top-end software vendors to achieve the agility he is enjoying.

“Why does IT have to be so complicated?” he asked.

“Our architecture principle at ING DIRECT is to adopt proportionate to our scale. So we won't drop a supercomputer into our data centre any time soon.

“Do I really need highly complicated patch management software? Why? Is it because I have five SANs and six processor models? Perhaps I’m attacking the wrong problem.

“In IT, the biggest constraint is the speed of light,” he said. “That is the only frame of reference we play in. We don’t need to be constrained to how we worked yesterday. 

“The technology to solve many of these problems exists, what's missing is the audacity to do it.”