IT shops across Australia are struggling with increased demands for digital innovation while battling reduced budget and cost pressures.
These businesses are often competing with more nimble companies which can innovate rapidly.
Many are turning to agile development practices as a possible answer to gaining momentum for planned innovation efforts and as an alternative method to reduce the cost of developing new systems.
The result for many organisations has been mixed, and some say a lack of success can sometimes be reflective of a company’s culture. It’s been recommended to have a disciplined project management office (PMO) work side by side with an agile methodology.
I don't see this as a religious war over which is better, as clearly there is a place for traditional waterwall and agile to both co-exist. The trick is that there is insufficient sharing of best practice.
So what does work and how does one go about figuring out the best approach?
In CIO circles, Spotify is the benchmark of agile software engineering.
The music streaming service is an incredible success story and has disrupted an industry where innovation is at its heart.
Spotify, from a standing start in 2008, has grown an impressive user base of 60 million users and 15 million paid customers. The company is renowned for retaining an innovative spirit despite rapid growth.
There are now around 600+ software engineers in Spotify, located around the globe in locations as diverse as Stockholm, Sydney, New York, Boston, and San Francisco. A size and scale that is not that different to many multinational corporates.
I sat down with the company’s Stockholm-based agile coach, Anders Ivarsson, to find out more about his role.
DG: I've heard about Spotify as a world leader in agile. What does Spotify do that is unique?
AI: Spotify as a company has a genuine belief in and understanding of agile values and thus boasts a very agile culture. It can be seen in all levels from top management down to our individual contributors.
We're putting a lot of effort into keeping a culture of high trust, high transparency and where people are passionate about always improving and getting better results.
Within our product development, we have an organisational structure that has really helped us have autonomous teams, while also scaling to quite a large size.
It has also allowed us to keep being quite fluid and experiment with improving how we work, both within the teams and as a whole organisation. This is what most people are interested to learn about from us - how we've structured into squads, tribes, chapters and guilds.
There’s more information on the company’s engineering culture on the Spotify labs blog.
DG: Is this home grown and is it still evolving?
IA: This is very much home grown and is a constant evolution. We are always experimenting with new solutions and ideas, and adapting to new challenges and problems that come up.
DG: What issues still remain to be refined?
AI: When it comes to agile, I think we still could be a lot more disciplined around agile tech practices. As an organisation, we've achieved high levels of autonomy, but a challenge is to find and keep alignment between all the autonomous parts without reducing the autonomy.
As we keep growing, there is an ongoing challenge with clarity of vision and intent while also allowing for innovation and initiative to solve real problems.
DG: How are team assignments made? By whom and how?
AI: Just like many other things at Spotify, the answer is that ‘it depends’. Sometimes squads are spawned from an existing squad and take over part of their mission and backlog.
At other times we have a new idea - that might have come from anyone in the company, developed during one of our company-wide hack weeks, or perhaps realised through analysis of user behaviours in our product - that we want to test. We then spin up a new squad with the mission to solve that problem.
The squad themselves are most often heavily involved in finding their mission and defining what problem they should tackle.
DG: Has your team experienced any scaling limits to the agile model?
AI: We believe that each tribe should roughly follow Dunbar's number – that is maximum 100 to120 members which allows everyone to know and interact with one another.
That gives some upper limits to how big a tribe can become, but we’ve also seen tribes grow beyond that size and come up with ways of organising themselves to make it work.
I think the general pattern of highly autonomous teams with clear a mission works on all levels, so may likely scale beyond the current size.
DG: What is Spotify looking for in new hires?
AI: When we recruit to Spotify - no matter if it's for developers, product people, UX or other roles - we always look for technical excellence and skills, as well as a cultural fit and willingness to really participate in teamwork.
Finding the people with this mix of skills, experience and attitudes can be challenging and we put a lot of effort into constantly improving our recruitment.
DG: How long does it take to orientate new recruits to your culture and approach?
Anders: It really depends. A lot of people come here and feel at home right away. But since we're growing so fast and are now quite large - plus we also keep constantly tweaking the way we work and how we're structured, it means it can be quite hard to get an overview of the whole organisation, what we're doing and how things work. This can be a bit of a shock to some at first.
We're running boot camps with every new engineer to give them a quick introduction to help easy onboarding, assist as to where to find information, etc - and that has really helped people get started more quickly.
DG: How well does the team collaborate globally?
AI: Cooperating across geographical barriers, especially across several time zones, is always tricky.
Rather than having a distributed team working on the same thing from multiple locations, we often try to find enough people to build a small team in each location that can move more independently and autonomously, and thus reduce the need for constant synchronisation.
DG: Do you think it is possible for other organisations to mimic the ‘Spotify method’?
AI: I receive emails from companies that after reading our articles or watching our culture videos have been greatly inspired. Those companies have started making changes beneficial to them.
I also get a lot of questions from companies that have been trying to implement some of the things we do, but are looking for clarifications on details or have found places where it doesn't fit well with their current culture or way of working.
My personal take is that while it's great to look at what other companies are doing to find good behaviours and inspiration, you always have to start where you are and make gradual changes and improvements that fit within your context.
I don't think complex things like organisational design can just be copy-pasted like a blueprint for what will work.
David Gee presented his investigation into Spotify’s agile approach at the 7th CIO Strategy Summit.