Flight Centre Corporate now has a single observability strategy and tool

Flight Centre’s Corporate travel division has elevated observability from a technical tool to a strategic platform that supports its productivity and customer experience ambitions.

Flight Centre Corporate's CTO A/NZ Grant Currey

The division will detail its strategy and a year-long global consolidation of Datadog at the vendor’s summit in Sydney today.

“We have a massive investment into productivity across the organisation so that we can continue to scale quickly globally and provide a consistent experience regardless of market,” Flight Centre Corporate’s A/NZ CTO Grant Currey told iTnews.

“It doesn’t matter whether you're travelling through one of our our key corporate brands [like] FCM Travel in Tanzania; whether you’re travelling with Corporate Traveller in Sydney; [or with] Stage and Screen Travel, which is one of our media brands and you’re flying for the NRL or cricket or something; that experience is absolutely consistent regardless of where you're on the world.

“To achieve that, we've had to look very carefully at how we tackle productivity, how we consolidate our systems, where we position our systems geographically and the tools that we put across our systems to provide these insights.”

Observability in the past operated at a country or regional level, underpinned by best-in-class products selected by the in-country technology teams.

However, since the pandemic, Flight Centre as an overall group has made a concerted effort to “globalise and centralise” its IT function, Currey noted.

This has led to a number of consolidation programs around its tooling.

Currey said that the corporate division has “grown organically through nationally-oriented businesses”, now serving 95 markets worldwide.

“As we’ve scaled, it’s created operational challenges for us in how we manage our systems, throughput, capacity, growth: all the usual line items in an observability space,” he said.

“It wasn't that we didn't have observability - we didn't have this consolidated global observability strategy.

“We were operating in a world where we had a best-in-class tool for logging, for application management, for VM-level health [etc] and none of these tools knew about the other tool.

“Trying to trace anything across these was an absolute nightmare, hence consolidating it into a best-in-class tool that has all of this as best-in-class, not just parts of [observability].”

Over the past year, the corporate division has consolidated its observability tooling globally to Datadog, starting with Australia and New Zealand, before fanning out.

Some markets already were using Datadog as their selected platform, particularly in Europe.

The organisation has continued to refine and expand its use of the platform over time.

Currey said one of the key work streams has been connecting all parts of the technology organisation to Datadog to support their respective activities and interests.

Currey said the software presents a “great learning and development opportunity” for the company’s engineering team.

He also “wants our web and cloud admins, [and] our level two and three support people to be in Datadog, living and breathing what it has to offer". 

“I want my senior leaders, heads of tech leaders, etc to be aware of their stacks and to be across how their apps are performing at that next level of view," he said. 

“And I want the executive layer to be across finance. I want them to understand at a high level what do we have, how's it operating, how many times is it going down from an incident standpoint?

“I was really trying to tackle this at every layer of the tech function.”

He said the division, in partnership with Datadog, had “made really good progress” on establishing these views, although he noted the challenge of doing so given the geographic spread of operations.

Tackling FinOps

The Datadog platform is also now doubling as Flight Centre Corporate’s FinOps tool to keep its predominantly Azure cloud costs in check.

The platform has a ‘cloud cost management’ module that the corporate division has implemented.

Again, the company had been monitoring its costs before, including with Azure’s in-built tools, but it had been a difficult process owing to the number of tenancies, subscriptions and resource groups, as well as access controls for each.

“It actually made the overarching cost picture quite complicated to look at,” Currey said.

“We needed a tool that we could actively report on [cloud costs], create custom dashboards, [and] start actually driving a strategy of the application owners owning their cost.

“It wasn't about cost reduction - it was about return on investment.”

Currey said the module had proven valuable early on as “a lag indicator for problems that haven't immediately surfaced through other observability areas”.

“We had a situation where we had a rogue network device that started having high utilisation over a particular network link,” he said.

"It actually wasn't causing problems, so we had no alarms going off, and we had no customer complaints, but what we did see was our cloud cost reports showed a very significant change in spend.

“We dug into it and discovered that we actually had a technical problem.

“That could have become a [bigger] problem in the future that started to actually consume large volumes of bandwidth, and then we might have started seeing [it in the observability platform].”

The module has also helped to instil a “wastage mindset” among application owners and technology teams consuming cloud resources.

“We’ve seen quite a lot of optimisation, and it's allowing me to forecast a lot better in the year ahead as to to where we're going to be [on cloud costs],” Currey said.

“I liked the fact that this was another module in the broader category of observability, and I can link a change in cloud cost back to an event in metrics. I can see a log entry that's contributed to a storage account spiralling out of control.

“There's this traceability from APM [application performance management] or RUM [real user monitoring] right through to the cost piece and it's all available in one platform.

“That was the key thing around platform consolidation: having a single tool to give us this without having to join all these [views] together [ourselves].”