UBank brings AI to customer agents with RoboBrain

NAB-owned UBank is expanding its artificial intelligence focus with the creation of an “agent assist” enterprise search tool called RoboBrain.

The digital bank entered the AI fray in May last year with a chatbot to aid home loan applications, which it called Robochat.

It is now building on the internal competency it created for that project, and expanding the reach of AI in the business in the process.

Both Robochat and RoboBrain are powered largely by IBM Watson components. They also owe their genesis to the same process - internal hackathons designed to flush out potential use cases and ideas.

RoboBrain lets UBank staff "search across the various knowledge bases and information sources that we have, to provide real-time help for our agents as they’re servicing customers,” head of digital and technology Jeremy Hubbard told IBM Think 2018 in Las Vegas.

“RoboBrain is quite simply an AI-powered information sourcing tool for our customer service agents. A one stop, one screen solution for information search at UBank.”

The tool took eight staff about 12 weeks to build and deliver.

It consists of Watson Conversation, Discovery and Knowledge Studio, and a custom front-end that customer service agents use to search across internal documents and systems.

“We didn’t use the out-of-the-box Watson front-end solution,” Hubbard said.

“We built [one] in React and hosted on our Adobe AEM content management system. It didn’t take very long to put together - somewhere less than five days.

“It has authenticated access for users, so they can start to filter and save their favourites in items and see trends that are happening across the whole business.

“Importantly, they can just type in a question and it returns results in near real time.”

RoboBrain solves an immediate pain point for customer service agents: previously they had to search across multiple internal systems to find things like historical interest rates.

“As you can imagine a bank changes their interest rates a fair bit, so there’s normally a big spreadsheet that our agents use to get that data,” Hubbard said.

“They also have their own source for this which causes a problem in terms of compliance for us.

“Now they’ll have one spot where they can do this search. From thousands of rows of a spreadsheet they can get the very specific information they want.

“That works for all of our products and was a really quick win for us.”

When an agent asks RoboBrain a natural language question, it polls a range of documents and files stored in source systems and surfaces “snippets” that appear to best match the query.

The snippet comes “straight from the source system”, Hubbard said.

The system is able to search across multiple document types - for example, HTML, Word files, PDFs, CSVs, email templates and files stored in cloud services.

The documents load from the source systems into Watson Discovery, which is used to train Watson to ‘read’ a corpus of information.

Hubbard showed off the insides of UBank’s application of Discovery.

“It looks like we’ve loaded in 2270 documents, but we’ve actually only loaded in about 900 documents and we’ve used a document splitter that breaks those documents up - if it’s a HTML document, based on HTML tags,” he said.

“This serves a couple of purposes. One - it helps Watson in terms of how it searches for data, and two it makes it easier for us just to return the segment of the document that the query relates to, rather than returning the whole [thing].

“In loading these documents in[to Discovery], it’s really straightforward. We have crawlers that keep them up to date [looking for changes made to them in the source systems].

“It’s probably really important to understand that we’re still keeping our system of record for our knowledge bases as the original sources. We crawl those, and we load them into Discovery. We haven’t changed the core source.”

Agents can rate the usefulness of the snippet they are presented by RoboBrain, which contributes to the relevancy training within Watson Discovery. They can also opt to go to the source document.

Training the ML

Helping Discovery to understand what it is ‘looking’ at is essentially a glossary of terms that help the tool to annotate the documents and files it ingests.

“To help Watson understand more about our business, and to teach Watson about Ubank-specific domain knowledge, we used Watson Knowledge Studio,” Hubbard said.

In IBM-speak, Watson tries to recognise “entities” mentioned in a user’s input. It can then take actions to fulfil an “intent” - which is defined as what the user wanted to achieve from the query.

“One of the key things we did was define the entity types as they relate to UBank,” Hubbard said.

“For example, we defined authentication - how you verify yourself at a bank - as one of the entity types, and then we defined the sub-types of that, which - for us - are 1FA, 2FA, re-verification and identity, and verification. They’re the different types of authentication we do at UBank.

“With those types defined, we then defined relevant dictionaries in the pre-annotator. So we have our authentication dictionary, and essentially synonyms for those types that are used by Watson as it builds its machine learning model.”

UBank tested its definitions by applying them against a subset of the 900 documents ingested by RoboBrain.

“Watson will have its first attempt at tagging or noting those annotations within the source documents,” Hubbard said.

The system is able to make a number of annotations to documents - for example, mentions, relations and co-references.

Hubbard demonstrated the ability of RoboBrain to annotate mentions of 2FA in source documents.

“It picks up everywhere that 2FA is mentioned,” he said.

“Obviously it wouldn’t understand this without doing that prior work, and in fact it also shows the sub-type as well, so it understands not just that it’s about authentication but also that it relates to a specific type of authentication.

“This model is then applied back into Discovery against the full document set and that’s what starts to give us better results and make it feel to the agents that Watson really understands our business.”

Internal reception

Hubbard said RoboBrain, which launched this month, has “probably had some of the best uptake we’ve seen from anything that we have deployed at UBank”.

“In fact when we were prototyping and testing at UBank early on, well before it was ready to be released, we had [customer service agents] essentially begging us to let them use it now before it was ready,” he said.

“We have one simple tool to search across our systems and the results are fantastic.”

While it is early days, UBank will look to benchmark its return on investment on the project against three measures: “employee NPS [net promoter score], reduction in average call handling time, and thirdly and very importantly for a bank. a reduction in complaints and compliance issues".

However, it is not under specific pressure to generate an ROI on its AI projects to date.

“We invest our money in a 70/20/10 model: 70 percent on things that we have to do, 20 percent on things that are new, and 10 percent on things that we’re willing to risk,” Hubbard said.

“Both of our AI experiments to date have been quite small.

“We haven’t measured ROI on them in a traditional project sense based on the fact that these are in the 10 percent of things that we’re willing to risk.”

That could change as AI takes greater root within the UBank business.

“We’re finding the more we learn about Watson, the more we can see how we can transform our business,” Hubbard said.

“It’s helping us solve problems that previously we didn’t think were solvable.

“Through the evolution of RoboBrain we’ve also realised that natural language is a great way to convey complex information.”

Ry Crozier is attending IBM Think 2018 in Las Vegas as a guest of IBM.