ABS "extensively" tests AWS-based occupation autocoder for June launch

The Australian Bureau of Statistics is readying a new cloud-based tool that can consistently and accurately assign occupation codes to free text responses in time for next year’s census.

Stirling Jones from ABS.

The project - called the ABS whole of Australian government occupation coding service - was funded in 2023 and is intended to be functional this month [pdf]. 

Speaking at AWS Summit Sydney, coding redevelopment project manager Aidan Kent said the bureau is “really excited by what we’ve seen internally” in its testing - that is, “a highly performant model that is going to reduce our manual coding and increase our autocoding.”

The bureau uses a common example of classifying an electrician based on the job and duty descriptions that are typed into free text fields in the census.

“Whether the service sees something like an electrician or a sparky, the service knows what that occupation is and can code it appropriately,” Kent said.

Product owner of the whole of the Australian government coding capability project Stirling Jones said the difference between an official code and job title, and “what a person’s actually put on a form … speaks to the real value of this product.”

“It allows the Australian population to answer the form in a language and vocabulary that makes sense to them. Then, we take the task of turning that into our structured classification code so that we can produce comparable statistics,” Jones said.

The bureau previously had a hierarchical support vector machine-based autocoding solution in place called ‘the ABS intelligent coder’, but it was on-premises and limited in how many records it could handle.

In saying that, the 'intelligent coder’ “demonstrated the real value and potential uplift of using ML to perform this data coding process,” Jones said.

The intent of the ABS whole of Australian government occupation coding service program was to create a cloud-based autocoder harnessing AWS machine learning and large language model (LLM) technology.

This is not only for use during the 2026 census, but also by any department or agency across government that collects occupation data, such as via surveys or web forms, and wants to classify the data according to accepted standards.

Jones said that applying codes to occupational descriptions could otherwise be “burdensome or potentially compliance-driven” for departments outside of the ABS.

The new ABS service means that coding can be automatically and consistently applied, “which increases their data quality, but it also saves them time and effort.”

“Overall, what we’re hoping will happen is that as classification is increasingly adopted through the use of the service [and] that Australia’s data landscape around occupation will improve and increasingly become coherent,” Kent said.

Jones said that “extensive testing” has shown about 95 percent of records can be automatically classified, “leaving us with about five percent of records that need clerical or manual review.”

Ry Crozier attended AWS Summit Sydney as a guest of AWS.