ASIC plan for AI snoops on insurance calls strains hearing

Australia's financial watchdog might be dreaming of the day when call centre surveillance software automatically catches crooked insurance sales staff.

But there's still a way to go before AI-powered voice analytics can decipher the verbage that bubbles out of a sales boiler room.

That's the reality check bowled up to regulators and industry rapidly spitballing prototypes of new regtech solutions as banks, insurers and auditors all trying to find ways to automatically detect bad behaviour without creating a profit sapping compliance cost sinkhole in the process. 

At a closely watched regtech forum late last month, ASIC outlined its findings from a trial of voice analytics software applied to a sample of life insurance sales calls.

With a freshly sharpened set of teeth, the watchdog says it sees “great potential” in using voice analytics to automatically identify instances of potential misconduct in life insurance sales calls - but there's a catch.

The surveillance smarts will only really be effective if insurers start to record calls at higher quality.

ASIC had been hoping to determine whether AI could “replicate what it took several ASIC staff many months to do”, according to Barbara Buettner, senior manager for insurers in ASIC’s financial services group.

That is, could voice analytics (and voice-to-text) be used to recognise indicators of poor sales practice in life insurance calls?

The answer, according to Buettner, is a “qualified yes”.

“The tools were able to replicate a lot of what a team of people did for ASIC, but they were definitely better at detecting some examples of misconduct than others,” she said.

Call quality hits trial

One of the main issues encountered by ASIC and its technology partners in the trials - Daisee, Deloitte and Call Journey - was the quality of calls generally made it hard for the AI, and even human overseers, to understand the content.

“The challenges we encountered did affect the results of the trial - there's no other way of saying that,” Buettner said.

“Call quality did affect transcription quality, and it affected the ability to do some word searches and more complex searches. 

“Sample size was also an issue. I think, in an ideal world, you'd have not hundreds but thousands or tens of thousands of hours [of calls] to work with this technology and to train it. 

“Because of the call quality issues, we were down to the low hundreds of good [quality] calls, and that affected our ability to to use voice signals, which was quite an important component of the trial.”

ASIC had been hoping to use AI to analyse calls for sentiment or vulnerability. This may be identifiable by listening for pitch changes, silence, overtalk, hesitation, response length and other variables.

But the call quality in general wasn’t high enough to run “more sophisticated voice signal analysis”.

“Our sample sizes probably held us back to really explore that, and so I think on this one, we probably need to do a little bit more work,” Buettner said.

Some insurers also recorded calls for quality assurance purposes in mono. That required additional technology to try and separate the speakers on the call

“When you're measuring sentiment, if you have a mono recording, you're actually measuring how happy the agent is - and when have you ever heard an unhappy agent trying to sell?” said Mia Lander, a partner in Deloitte’s audit and assurance, governance, regulation and conduct advisory.

“So that will impact on your results.”

Software compensation only goes so far

All three tech companies in the ASIC trial encountered varying problems with call quality.

“One of the things that we found most challenging about this, of course, like my colleagues, was the recording quality,” Lander said.

“This is not new. We strike this all the time. 

“When you're looking at remediation programs, where you're going back through maybe up to 10 years worth of call recordings, those call recordings would not have been made in an optimal environment.”

All firms said they were able to “compensate” for those issues by using various algorithms and tools to clean up the audio or otherwise determine context in a way that reduced the number of false positives.

“Looking at the quality of recordings … there's a vast amount of variability in the data, and in fact, only 30 percent of the calls were of excellent or very good quality,” Daisee CEO Richard Kimber said.

“Despite that, we were still able to get a significant amount of insight and we were able to find all the issues that [ASIC wanted to recognise]. 

“But going forward, this is no longer going to be the case. Every company in the room should have a world-class call recording system.”

One of the issues is that current call recording systems were not designed with voice analytics in mind.

“If there's CIOs in the room who represent their particular organizations, most of the time that people want to record a conversation, it's because they have to, and they've got to store it, compress it, and it's got to be all squeezed,” Call Journey CEO Paul Humphrey said.

“Most people don't record a call because they need to analyse it so it's got to be at decent levels of quality. 

“In our experience, we still see a number of customers - quite probably a high propensity of people - with mono recordings, so we've had to work hard at our technology to engineer the capability to manage mono recordings and separate the two callers.

“But again, most of our customers will start adopting a change in the way they record their calls, just so that they can get much greater insights, and we are seeing that quite commonly now.”

Humphrey continued: “Conversation analytics can vastly improve with one simple industry practice, and that's high quality recordings. 

“Many of our customers are changing their recording capability now to be able to do this, and you can get very good accurate outputs from that perspective to cover those things under the misconduct rules.

“We do think that all participants in the industry should change their recording capabilities. That's just going to make it easier.”

The technology firms also used software to try to minimise background noise in recorded calls.

“I've been in a call centre,” Lander said.

“Something that you cannot control is customer end noise. So in this data set, we had customers calling from outside, we had customers calling with screaming kids, and those things will impact on the recording quality and therefore the analytics quality that you have. 

“If you're looking at trying to make something that's more accurate, then you need to think about all of those things.”

ASIC remains positive

Buettner acknowledged that “discrepancies” in call quality meant that voice analytics “might take a little while ... to be realistic” for production use.

However, she was confident that the challenges could be overcome.

“For ASIC, I think we could really scale up surveillances and investigations, listening to thousands rather than painstakingly listening to hundreds of calls,” she said.

“We will continue to explore how this fits into our regulatory toolkit. And people like me who run a stakeholder team would be very pleased if we don't have to spend eight months with 10 people listening in to call recordings.”

With Julian Bajkowski.