Telstra says timekeeping issue behind mobile network outage

Telstra has revealed a nationwide mobile network outage is due to a failure of timekeeping nodes used to synchronise its network operations, with work underway to recover them.

The network outage has had a broad impact, on Telstra and its MVNO customers, as well as on train services and payments.

Chief financial officer Michael Ackland told reporters in Melbourne that the telco is unsure why the nodes failed.

“At approximately 4.30 this morning, we identified an issue affecting some mobile calls and data services,” Ackland said.

“The issue is impacting a number of nodes within our network that keep time across the mobile network, and when these nodes are not operating as expected, other parts of the network can be affected, resulting in intermittent issues with some mobile calls and data sessions. 

“We’ve been able now to restore some of these nodes but it will take time for that to reset across all the thousands of servers across the network. 

“We believe that just under 90 per cent of calls and data services at this point are now working correctly.”

Telstra has not confirmed which part of its network is responsible for Wednesday's outage.

Mobile networks rely on precise timekeeping to function, and that timing infrastructure typically spans several layers between the network core and the edge.

A primary reference clock at the core derives its accuracy from global navigation satelllite system (GNSS) satellites, of which GPS is one constellation among several, and each satellite carries onboard atomic clocks accurate to within nanoseconds.

That reference signal then passes down through boundary clocks at aggregation nodes before reaching individual cell sites at the network edge.

A fault at any layer of this chain, from the GNSS receiver itself to a boundary clock partway through the transport network, could produce the kind of widespread service disruption customers are reporting.

Ackland said the telco is still trying to determine why the timekeeping infrastructure stopped working.

In response to questions, he said there was no indication that the root cause is cyber-related.

"At this stage, we have nothing to indicate malicious activity and we’ve been in contact with all regulators and government agencies to that effect, but we continue to investigate, remain curious and explore every avenue but we have nothing of that," Ackland said.

Triple zero calls

Various policing agencies have indicated that there may be issues with emergency calling, and Telstra indicated it is following up those reports.

However, Ackland said that emergency calls should not have been impacted.

"Triple zero calls follow a different settings in our network and therefore they were not impacted in the same way [as other voice and data services]," he said.

"We have seen a small number of reports that we are investigating and as part of our standard process, we conduct welfare checks when any call to triple zero fails, and we’ve been doing this."