Australia Post is co-developing two ML models to prioritise its incident queue

Australia Post is working with a startup called Alpha Level to co-develop and production test two machine learning models that can help it to prioritise cyber security incidents for SOC intervention.

The government business enterprise (GBE) announced the partnership with Alpha Level at the end of last month as a pathway to “boost both the speed and accuracy of threat identification”.

While Alpha Level has an existing commercial product - an alert management system - Australia Post is not using it, instead co-developing entirely new models that may be commercialised later.

“We’re not buying a product from Alpha Level - it is actually a partnership,” Cartwright said.

“It’s probably a misused term, but what we are doing with Alpha Level is collaborating on some problems that most of cyber security has - independent of Australia Post - and we’re trying to develop new ways to expose and solve those problems.”

Cartwright said the initial work is focused on two machine learning (ML) models, both of which address aspects of the “overwhelming volume of alerts and events flowing into a SOC.”

The intent is to effectively prioritise the incident queue in the security information and event management (SIEM) process “in a way that focuses the attention of the analysts on what needs investigating.”

Out of billions of events recorded in the SIEM every week - from network traffic to security logs - thousands will trigger rules and be flagged as potentially malicious.

Most of those are either false or benign positives. The SOC winds up investigating a proportion directly - iTnews has elected not to reveal the exact numbers - which leads to a handful of “true positives” that require actual intervention.

One of the models being worked on with Alpha Level aims to filter out the false positives and benign positives faster.

“If you can somehow reduce that number, you’ve increased the effectiveness of your team,” Cartwright said. “They can concentrate on the things that matter.”

One of the challenges in this space is that signals associated with a more sophisticated threat may be weak or look like legitimate traffic. 

Cartwright indicated that ML could be a good way of detecting even weak signals amid a sea of otherwise legitimate system usage and traffic, and flag them for attention correctly.

The other model the partners are working on aims to reduce the noise in the SIEM, identifying “true positive” threats and putting them on top of the action list for analysts.

“What I’d like to do is have my SOC focus on those things straight away. I don’t want them to have to sift through a backlog of signals to get to [the highest priority signals], because response time is important. The sooner you can respond, the less damage is done in the environment,” Cartwright said.

The ML models, which the partners have been working on for about five months, are presently tagging incidents directly in Australia Post’s SIEM. 

Incidents that the models classify as false or benign positives are not culled.

“We’ve made a decision that we don’t want any of the alerts to disappear. We still think they need investigation until we’re more confident in the product,” Cartwright said.

“It’ll be very unlikely we get to a point where we say AI can decide that we shouldn’t look at this. It will be more [likely that the] AI will recommend to a human that this is low priority.”

Knowledge exchange

Cartwright said the partnership between Australia Post and Alpha Level came about when “dominoes lined up.”

“Many years ago ,when I was in the banks, I worked with the founder of Alpha Level, Dr Joshua Neil. I met him at Los Alamos National Laboratories in the US, I was there on behalf of one of the banks to look at the use of AI and ML in cyber security, and he had a PhD in statistical mathematics. He has a brain as big as a planet, as do most of the people in Los Alamos. It’s quite a fascinating place,” Cartwright said.

“We talked about some of the problems he was solving on behalf of that facility, which obviously attracts attention of all sorts, and how they’d developed models to detect anomalies and respond.

“That technology we then worked through in one of the banks in Australia. We worked with him to understand how it worked in a commercial environment versus how it worked in that highly restricted environment. We both learned a lot.”

Fast-forwarding through a few years, Cartwright says he got back in touch with Neil - who had just started Alpha Level - “interested in solving some cyber problems”.

That led to discussions and the result was the current partnership.

The partnership hasn't yet led to money changing hands, but Australia Post - and at least one other Australian and US company - are getting some thorny cyber security data deluge issues addressed.

“We’re not actually paying them for this at the moment,” Cartwright said.

“But the benefit to Australia Post is we’re getting their deep expertise. People like Josh are very rare in the world, so having that direct conversation with him and feeding the model and commercialising what we want versus what he thinks we want is very important to us. 

“At some stage they’ll come up with a product on the back of it. They’re a startup, they want to go-to-market with something, and then at that stage we’ll decide if we’ve gotten far enough along this track to justify the investment.”

Cartwright said that few Australian SOCs had access to the “calibre of thinking” that the partnership provided.

In addition to producing specific models, Cartwright said that “skills transfer” is another key deliverable under the arrangement.

He noted that the transfer is “not to the extent [that my analysts are] going to become statisticians … but they get to understand modelling, the pitfalls and the advantages, because there are pitfalls to using ML and AI - I’ve learned that over the past 15 years in cyber security. 

“There’s huge advantages, but you don’t get exposed to that outside of a vendor knocking on the door usually and saying, ‘Here’s my product, it’s got AI, buy it’. And everyone’s doing that. 

“So, this is actually educating my team too on what does and doesn’t work, what you have to think about. 

“That collaboration is gold.”