Black swans and white lies

Opinion: REA Group CIO Nigel Dalton discusses the history of agile.

The city of Perth, Australia plays an important part in understanding the global spread of agile software development.

It was there; on the aptly named Swan River in 1697 that Dutch explorer Willem de Vlamingh made the horrifying discovery that the commonly used European metaphor of a ‘black swan’, meaning the very essence of impossibility, was false.

Thirty years later, two birds were captured North of Perth and taken back to Europe as proof beyond doubt that someone needed to quickly invent a new phrase to mean ‘impossible’.

Since then, a popular business book from 2010 (The Black Swan by Nassim Nicholas Taleb, Random House) has updated the meaning of the black swan metaphor as an economic phenomenon –

"I stop and summarise the triplet (of meanings): rarity, extreme 'impact', and retrospective (though not prospective) predictability. A small number of Black Swans explains almost everything in our world, from the success of ideas and religions, to the dynamics of historical events, to elements of our own personal lives."

Agile is the greatest black swan to hit Western business in the last 100 years. Rare, having an extreme impact on business as we know it, and thoroughly explicable in retrospect.

Had I not personally encountered extreme business failure from slavishly applying Henry Ford and Frederick Taylor’s century-old command and control management techniques, I too, as the typical 50 year old mid-level executive helping run an Australian business in 2013, would be dubious of card walls, scrum-masters, and daily standups that resemble the prayer meetings of some cargo cultworshipping all the library index cards leftover from the 20th century.

Since those days of working in a startup in San Francisco in 2000, where no amount of documentation of product or functional specifications in a templated 30- page Word document, or comfort-giving enterprise architecture diagrams drawn in PowerPoint could tie down a business model and set of customers which was changing weekly, I have been drawn to agile’s history in an attempt to understand the extreme emotional reaction it creates among my cohort:

It looks like anarchy.

Nobody is in charge.

The developers are just voting on whatever they want to do.

You can’t do a proper big project that way.

There’s no commitment to hitting hard deadlines or spending.

It’s ruining the workplace – just look at the walls covered in cards and blu-tack.

We have shareholders who expect proper governance and plans.

You have to separate the duties of developers and operations people – it’s the law.

The auditors won’t sign it off – there’s no documentation and we have depreciation schedules to meet.

Projects must have end dates, or they’ll go on forever.

We tried agile, but it didn’t work.

All little white-swan lies designed to protect the careers of middle-aged men in charge of business who just want a quiet drift into their retirement at a beach house in Portsea or the Sunshine Coast.

Agile software development grew from a moment of modern cross-cultural encounter 300 years after de Vlamingh, when an article published in the Harvard Business Review titled The New New Product Development Game explained in detail how Japanese product development had turned to small, empowered, multi-disciplinary product teams working to crushing timelines, to stay in the market.

The black swan had been spotted – but in my time at University studying economics in the early 1980s, we embraced the Western falsehood that Japanese industry had copied, cheated, manipulated its labour forces, controlled the exchange rate and ensured collaboration within a closed market by massive cartels called keiretsu.

The truth had a much more human and systematic foundation - Konosuke Matsushita, the Chairman and founder of Panasonic summarised it brilliantly in the early 1980s to a group of visiting American industrialists:

“We are going to win, and the industrial West is going to lose - there’s not much you can do about it, because the reasons for failure are within yourselves. Your firms are built on the Taylor model. Even worse, so are your heads.

With your bosses doing the thinking while workers wield the screwdrivers, you’re convinced deep down that is the right way to run a business. For the essence of management is to get the ideas out of the heads of the bosses and into the heads of labour.

We are beyond your mindset. Business, we know, is now so complex and difficult, the survival of firms so hazardous in an environment increasingly unpredictable, competitive and fraught with danger that their continued existence depends on day-to-day mobilisation of every ounce of intelligence.”

Complex. Difficult. Hazardous. Unpredictable. Competitive. Fraught with danger.

In retrospect, our turning to agile to cope with the madness of our high-speed digital-obsessed 21st century makes all the sense in the world.

Nigel Dalton works as CIO at REA-Group in Melbourne, and blogs at lunatractor.com.
More opinions, analysis at iTnews' agile topic page.