Why agile works

After several years of uphill battles and aborted starts, Australia’s most progressive organisations have taken to agile software development in a big way.

The digital teams at banks such as ANZ, Bankwest, Commonwealth Bank, NAB and Suncorp; insurers such as Allianz and SunSuper; telcos such as Telstra and Vodafone; retailers like REA Group and Carsales; and the digital team at media conglomerate Fairfax - each have unlocked value from this new method of work.

Over the next two days, iTnews asks those charged with leading these teams why agile works, how to sell it to the business and how to take on the culture change required on the journey to DevOps and continuous delivery.

Why agile works

Agile attempts to solve a common mismatch between what the business demands of IT and what gets delivered.

Under traditional approaches to software development, a business analyst is likely to spend months detailing the requirements of a system before it is developed, handed to testers, and then onto IT operations to be put into production, only to discover that the business’ requirement had changed during the process.

Andrew Walduck, CIO at Australia Post, believes the days of delivering these large, complex, ‘complete’ software solutions within a specified timeframe are over.

"The number of times I've seen operating models where you start with requirements on one side, you dump it into operations on the other, and it fundamentally misses the point,” he said.

Agile attempts to bring the business into the development process by including its representatives (product owners) in regular meetings with the development team to assess progress.

Agile also breaks the development and testing work down into smaller chunks (or ‘iterations’), that tend to be delivered by self-managed teams, each of which pushes new features out into production in time-boxed sprints.

The model enables the most essential feature required from a solution (the ‘minimum viable product’) to be delivered earlier, and allows the business to change its mind about the remainder of the features along the way based on customer feedback.

“What I most love about agile is that we get the right thing to customers earlier,” says Pete Steel, CIO of retail and business banking at the Commonwealth Bank. “We are able to identify the wrong things earlier and save a lot of waste.”

Suncorp has been on the agile journey for longer than most Australian organisations, to the point where today the methodology is “embedded into how we work,” according to Michael Moseley, the bank’s digital platform manager.

Moseley said the key benefit of agile is that it “empowers teams to act autonomously” and encourages them to experiment while they are working.

On the insurance front, Allianz is also using the agile scrum methodology on a large online transformation program, with the help of an agile coach.

Andrew Mitchell, IT development manager at Allianz, said both IT and the business find agile a more productive and enjoyable way to work.

“It’s been an exciting 12 months for us,” he said. “You really do feel the energy walking out onto the floor. We have a number of screens set up showing the scrum progress, and everyone is doing the scrum ceremonies. The business comes onto the floor and they feel that energy too.

“It’s a sign on the hill,” he said.

False starts and new beginnings

Talking to those that have successfully implemented agile programs in their organisation, it’s easy to understand their sense of achievement. More often than not, attempts to introduce agile to well-established industries fail the first time, and succeed shortly thereafter.

“Sometimes we feel like we’re making it up as we go along,” said one attendee at a recent IBM roundtable on continuous delivery, hosted by iTnews. “It was only after talking to other early adopters they realised they we are all in the same boat."

Those that have succeeded all stress that adopting agile successfully requires massive cultural change.

Mitchell said Allianz has enjoyed its first wins over the last 12 months after an unsuccessful attempt at employing agile three years ago.

“One of the lessons IT organisations need to learn is that agile is actually about being more disciplined in what you do,” Mitchell advised.

“If you’re not more disciplined, you will fail. That’s what we experienced three years ago. We needed more discipline around automated testing and absolute discipline around ceremonies and retrospectives every two weeks.”

It is very important to employ agile coaches to reinforce this discipline, he said.

“If you try to do that yourself, you tend to make it up as you go along and you typically end up with Wagile - or ‘Waterfall Agile’. Next thing you know somebody is saying, ‘let's test what we did in the next iteration!’

That’s when the agile coach steps in to right the ship.

“For me, agile succeeds if you’ve got the engineering discipline,” Mitchell said.

Agile coaches, agrees Steel, “are absolutely essential”.

The Commonwealth Bank also “false started” on its first attempt at agile some five years ago, and has enjoyed success over the last three years.

Steel advocates adherence to the three E’s - education, experience and exposure.

CommBank now employs several hundred staff - from both technology and the broader business - participating in scrums on a day-to-day basis and earning their agile stripes.

“Walking out into the action on the floor is quite exciting,” Steel said.

Continuous Delivery

Having teams exposed to an agile mode of working is the first step in a long journey.

Mac McIntosh, technical sales leader for IBM Rational Software in ANZ, said that while agile addresses a gap between development and the business, it is part of a much broader journey to continuous delivery and 'DevOps'. 

Organisations that begin with agile project management methodologies like Scrum and KanBan tend to expand into more technical practices from eXtreme Programming like SCM (automated change controls), TDD (test-driven development), Build Automation and ‘continuous integration', he said, where agile teams continually merge and test code into a mainline stream.

“The challenge is for IT teams is to now go beyond foundational practices like version control, automated builds and continuous integration.”

Organisations are now exploring and implementing integrated capabilities like automated testing, service virtualisation, automated deployment and cloud to enable them to create a continuous delivery pipeline to production, potentially at the push of a button, he said.

This approach supports constant iteration, experimentation and feedback from users to continually refine an online service. 

Continuous Delivery - and agile more broadly - requires software delivery executives to convince the CFO to embrace a steady stream of funding for development activities. This can prove challenging in organisations where funding is allotted on a project-by-project basis, backed by well-defined ROI models that rely on lumps of CapEx attributable to a pre-determined outcome, fixed deadlines and fixed costs.

Walduck advised his fellow CIOs to choose a quick win to prove out the value of the model before seeking a more regular stream of funding.

Walduck managed to win six months of funding at Australia Post to stand up its first product using agile. It was a developer centre (list of APIs) that allows e-commerce sellers to extend Australia Post’s postage calculators and ordering service into their web stores.

“We used this first product to demonstrate agile could be trusted,” he said. “Now we have multiple continuous delivery teams funded.”

Claudia Lajeunesse, head of platform solutions at FIIG Securities spent close to six years prior implementing agile at superannuation firm, Sunsuper.

“It was an amazing cultural journey,” she told iTnews. “It was an amazing fight. You cannot underestimate the cultural impact.

“If I sit back and reflect on our agile journey, I spent a lot of time with the executive team explaining what benefit that was to them and bringing them on that journey and letting the [dev] team get on with it,” she said.

Lajeunesse recommends software delivery leads resist the temptation to focus on the details of agile when selling into the business.

Her success at Sunsuper was based on “tailoring our conversation to the business level,” which broadly means focusing on outcomes rather than “how you got there.”

The other major stakeholder to consider when looking to get an agile practice off the ground is those charged with managing risk.

“In the financial services industry, risk is critical,” Steel explained. “Risk can have a lot of legitimate concerns: at first this agile business is cowboy stuff to them. It sounds like cowboy stuff if it’s only described as moving quickly and throwing things into production and thinking about the consequences afterwards.” 

CommBank and others overcame this hurdle by inviting risk staff onto the floor to participate in scrums.

“Some of those with the potential to be the fiercest critics of agile [at CommBank] became advocates very quickly,” Steel said, because “they have a voice.”

Invariably risk teams realised that agile represents “a much better way for them to be working as well,” he said.

“That was an amazing turnaround for a quite conservative part of the industry, and they could make a difference earlier in the process than ever before.”

What does agile success look like? Read on to find out...

What does success look like?

Every organisation has a slightly different definition of what continuous delivery means from a time-to-value perspective.

Steel cites continuous delivery as the next step in the journey for Commonwealth Bank. He sees continuous delivery achieved via the automation of deployment processes to the point that iterations can be put into production at pace.

CommBank’s agile teams currently work on six-week release cycles - a short time-frame for any financial services industry let alone a big four bank - but Steel thinks there is still work to be done to get to what could be construed as ‘continuous delivery’ in the context of a regulated industry obsessed with risk.

“We’ve got iterations down to six weeks from three months,” he said.

“There is a lot of coordination required to get a major innovation out the door every six weeks. We want to shorten that for certain types of functions such as standalone customer innovation, less so for the deep plumbing inside the group. There will always need to be more scheduled releases for the big iron and systems of record.

“For systems of engagement, we want teams to ship a change that night if it’s ready. We want a train to be able to leave every night, so long as it is based on compartmentalised, safe releases. We aren’t quite there yet. When we have that going it will be fantastic. There is a lot of momentum and excitement in that area.”

Vodafone Hutchison Australia has spent the last 12 months experimenting with the ‘continuous delivery’ model.

Embracing agile allows VHA to experiment at low cost, put a digital product or feature into the market and see if customers will buy it. If customers bite, the focus becomes how to improve it and do more of it.

“Over the last 12 months we’ve looked at how to move from a cycle time of putting software out every six to nine months to now where we can put changes out daily without any downtime, whenever we want,” said Craig Rees, head of technology and product architect at Vodafone. “That’s been a huge journey.”

Bankwest has meanwhile driven agile practices from four teams up to 20. Another 10 would come close to fitting the definition, but not close enough for Ed Cortis, head of systems delivery and a vocal advocate of agile practices. These teams are nonetheless hosting stand-ups and retros, visualising their work, and engaging in practices other organisations would call agile.

A key indicator of the bank’s maturity has been the appointment of iteration managers, who work alongside project managers to focus on releasing services at a greater velocity.

The milestone for agile success, Cortis says, is when such practices are driven by ‘push’ rather than ’pull’.

“The tipping point is when it is no longer me standing on a soapbox, preaching the agile message,” he said. “The tipping point is where somebody comes to you and says, I heard you did good stuff with this, please come help us? That’s when you know it’s no longer an uphill battle.

“We’re starting to see agile practices leak out to the rest of the organisation,” he said. “There are coaches working with the rest of the business.”

For Walduck at Australia Post, continuous delivery means delivering many hundreds of small changes to its online services within the typical defined timeframe allocated to a software delivery project.

This allows for rapid experimentation when the business isn't entirely sure on the business outcome, he said.

“We can adapt and listen and work with our business colleagues in a far more integrated way to evolve ideas rather than define everything upfront,” he said. “We must move to a model in which we are more granular in how we execute and deliver. We need to test, try and learn.”

“As we reinvent our business, as we push into new products and services and as our environment becomes less certain, these adaptive and iterative techniques that are grounded in customer research enable you to realise outcomes sooner.”

Allianz has meanwhile moved from a six-month delivery window to three, and more recently to monthly.

“That’s good progress,” says Mitchell, but “but it’s still a massive effort every time you deploy. I’d like to get deployments into production during the day by the end of this year, if not in the next three months.”

The measure of success for continuous delivery is determined by the length of your longest iteration or sprint, he said.

Success is determined whether you can "press the button" at the end of that sprint and push code safely into production, he said.  

"Once your test team or continuous integration team says it’s done, once your TDD [test-driven development] suite says it’s done, once you’ve run all your performance tests, just deploy it! Why do you need any more governance and ITIL-style hoops to go through? You should be able to trust your process at that point and should be deploying at the end of every agile sprint. That is continuous delivery, in a nutshell."

Mark Cohen, Head of Digital at Fairfax has a one-year target for his team being able to be able to build, test and deploy code into production every day by year end.

“This doesn’t mean I expect the team to be releasing 50 times a day, it matters more that we are capable of it,” he said. “One release a day is the benchmark. We haven’t yet hit the target for saying that we practice continuous deployment, but we’re inching our way there,” he said.

Getting there, says Cohen, requires culture change on a scale few IT organisations have been brave enough to traverse before.

Many of these software delivery leads view the concept of DevOps - and its emphasis on greater collaboration between software development and IT operations – as the required ingredient.

This will be the focus for Pt II of this feature series.